Transcript GES DISC Experience with HDF Formats for MeaSUREs Projects

GES DISC Experience with HDF
Formats for MeaSUREs Projects
by James Johnson
NASA/GES DISC (Wyle Inc.)
April 17, 2012
What is MEaSUREs?




Making Earth Science Data Records for Use in
Research Environments (MEaSUREs)
ROSES-2006 Proposal with 29 projects
Create Earth Science Data Records (ESDRs):
long-term, consistent, and calibrated data and
products that are valid across multiple missions and
satellite sensors
GES DISC supports 7 MEaSUREs projects
GES DISC MEaSUREs Projects
Project
PI Name
Type of Data
File Format
Reprocessing and Goddard Satellite-based
Surface Turbulent Fluxes (GSSTF) Data Set
for Global Water and Energy Cycle Research
Chung-Lin Shie
L3: 1.0°×1.0° Grids
HDF-EOS5
Creating a Long Term Multi-Sensor Ozone
Data Record
Richard McPeters
L3: 5° Zonal Means
L2 Nadir Profiles
HDF5
HDF5
Consistent Long-Term Aerosol Data Records
over Land and Ocean from SeaWIFS
Christina Hsu
L3: 0.5°×0.5° Grids
L3: 1.0°×1.0° Grids
L2: Orbit Swaths
HDF5
GOZCARDS: Global Ozone Chemistry and
Related trace gas Data Records for the
Stratosphere
Lucien Froidevaux
L3: 10° Zonal Means
netCDF4
Earth Surface and Atmosphere Reflectivity
Since 1979 from Multiple Satellites
Jay Herman
L3: 2.0°×5.0° Grid
L3: 5° Zonal Means
HDF-EOS5
A Multi-Sensor Water Vapor Climate Data
Record Using Cloud Classification
Eric Fetzer
L2: Co-located
(AIRS and Cloudsat)
netCDF4
Developing Consistent Earth System Data
Records for the global terrestrial water cycle
Eric Wood
L3: 0.5°×0.5° Grids
HDF-EOS5
GES DISC Recommendations





GES DISC proposed recommendations for file names,
formats (HDF5, HDF-EOS5, netCDF4), and metadata
(CF-1 and S4PA)
Work with data teams to reach ‘recommended’
requirements
Goal to get consistency among the various MEaSUREs
data projects
Why? Primarily for benefit of users and also for
interoperability software such as OPeNDAP
Harness full power of 'self-describing' file formats
Problems and Issues




Ad-hoc HDF5 implementations cause problems with
OPeNDAP handler and other generic tools (particularly
if no CF-1 conventions used).
HDF-EOS5 has a well described data model for a
generic handler. Recommend CF-1/udunits conventions
for attributes.
netCDF4 implementations have not been tested yet,
except classic netCDF implementations using CF-1
conventions.
GES DISC would like to remove the HDF5
recommendation to prevent future problems.
GES DISC MEaSUREs Page


For further information about these projects:
http://disc.gsfc.nasa.gov/measures/
OPeNDAP access to available data at:
http://measures.gsfc.nasa.gov/opendap/