Transcript comparing DAP4 and the CDM

Unidata’s CDM:
netCDF, OPeNDAP, HDF
Ethan Davis
10 January 2014
ESIP Winter Meeting – OPeNDAP BoF
OPeNDAP support in netCDF libraries
• netCDF libraries
– netCDF-C (and Fortran, C++, perl, R, python, ...)
– netCDF-Java (implements the CDM)
• OPeNDAP support
– Existing DAP2 support
• Client: netCDF-C and netCDF-Java/CDM
• Server: THREDDS Data Server (TDS)
– DAP4 development
• Client: netCDF-Java/CDM
• Server: THREDDS Data Server (TDS)
– Future DAP4 support (real soon now!)
• Client: netCDF-C
C Library Architecture
Application
Data Access API
Dispatch
NetCDF-3
NetCDF-4
HDF5
HDF4
OPeNDAP
…
CDM Architecture
Scientific Feature Types
Application
Datatype Adapter
NetCDF-Java/
NetcdfDataset
CDM architecture
CoordSystem Builder
OPeNDAP
NetcdfFile
THREDDS
I/O service provider
Catalog.xml
NetCDF-3
NcML
cdmremote
NIDS
NetCDF-4
GRIB
HDF4
GINI
Nexrad
Remote Datasets
Local Files
…
DMSP
NetCDF Classic data model
(aka netCDF-3)
• Multidimensional arrays of primitive values
– byte, char, short, int, float, double
• Key/value attributes
• Shared dimensions
• Fortran77
OPeNDAP and netCDF-3
• DAP2 carried this data model pretty well
– Some translation required
• String vs char*
• Shared dimensions vs Grids/Maps
• DAP2 extras
– Sequences, relational queries
• HDF-5 extras
– Groups, structures, variable length arrays,...
NetCDF Enhanced Data Model
(aka netCDF-4)
OPeNDAP and netCDF-4
• netCDF-4 extends classic model:
– Groups; Structures; Sequences/varLength arrays
• Mapping into DAP2:
–
–
–
–
Shared dims vs Grids
Primitive types
No Groups in DAP2
No relational queries in CDM
• DAP4 will carry the netCDF enhanced data model
– Still netCDF-4 doesn't do relational queries
• Something for CDM scientific data types (efficient?)
• Server-side (TDS) for efficient "relational" queries
Thank You
Commitment to Backward Compatibility
Because preserving access to archived data
for future generations is sacrosanct:
• NetCDF-4 provides both read and write
access to all earlier forms of netCDF data.
• Existing C and Fortran netCDF programs
will continue to work after recompiling and
relinking.
• Future versions of netCDF will continue to
support both data access compatibility and
API compatibility.
NetCDF Disk Formats
Thank You