Transcript MODIS (and VIIRS Land) Current and Future use of HDF Robert Wolfe NASA Goddard Space Flight Center Code 614.5, Greenbelt, MD [email protected].

MODIS (and VIIRS Land)
Current and Future
use of HDF
Robert Wolfe
NASA Goddard Space Flight Center
Code 614.5, Greenbelt, MD
[email protected]
Terra and Aqua MODIS
• 3 Nadir Spatial Resolutions
– 250m (1,2), 500m (3-7), and 1km (8-36)
• 4 Focal Plane Assemblies (FPAs)
– VIS, NIR, SMIR, and LWIR
• 36 Spectral Bands
– Reflective solar bands (1-19, and 26),
thermal emissive bands (20-25, 27-36)
• On-Board Calibrators (OBCs):
–
–
–
–
Solar diffuser (SD)
SD stability monitor (SDSM)
Blackbody (BB)
Spectro-radiometric calibration
assembly (SRCA)
– Space view (SV)
• Science Applications
Terra (EOS-AM)
Aqua (EOS-PM)
Launched Dec.1999 Launched May 2002
First light Feb. 2000 First light June 2002
– Land, oceans, and atmosphere
– Nearly 40 science products generated
and distributed
Page 2
Current MODIS HDF Usage
• MODIS Level 1 (L1), Atmosphere and Land product formats
are
–
–
–
–
–
–
HDF4
HDF4-EOS structural metadata (swath and grid)
ECS core metadata (ODL string)
Mostly SDSs and Attributes (some VDATA)
Internal compression
Geolocation stored in swath products both internally (reduced resolution)
and externally (full resolution)
• MODIS Oceans (SeaWiFS heritage) product formats are
– HDF4
– “ECS core equivalent” metadata stored as attributes (matches SeaWifs
format)
– Mostly SDSs and Attributes (some VDATA for L3 intermediate products)
– External compression (i.e. gzip)
Wolfe - Nov 09 HDF Workshop
3
MODIS HDF4 Commentary
Commentary – my views, not official MODIS views
• “Simple” HDF Attributes should have been used for both
HDF-EOS structural metadata and ECS core metadata
If attributes had been used then:
– It would have been easier for users to read metadata fields using
the standard HDF library (without SDP library)
– It would have made it easier for Science Team developers to
read/write metadata
• Some lessons learned for Terra (and Aqua) were applied to
the Aura mission data formats
Wolfe - Nov 09 HDF Workshop
4
Planned MODIS HDF Usage
• MODIS will (likely) provide products in HDF5 in 2011
timeframe
L1, Land, Atmosphere:
– Initially, HDF5 will be offered as one format available as a Post-processing
option (i.e. like GeoTIFF) for MODIS Collection 6 products
• MODIS Collection 6 reprocessing will start early 2010 and be completed by end of 2010
–
–
–
–
HDF5-EOS structural metadata
ECS core metadata fields are likely to be stored as simple HDF5 attributes
Internal compression will be used
Later, PGEs (science software) I/O may be rewritten to read/write HDF5
• Oceans
– Switch to HDF5 will be driven by user community
• HDF5 will be used for Aquarius (produced by SeaWiFS group)
– SeaDAS user tool will be modified to read/write HDF5 as well as HDF4
Wolfe - Nov 09 HDF Workshop
5
VIIRS Land Bands
VIS
Wavelength (um)
Land Surface Temp.
Active Fires
Vegetation Index
Snow Cover (Binary)
Snow Cover (Fraction)
Surface Type
Surface Albedo
Ice Surface Temp.
Sfc Reflectance (IP)
NIR SWIR MWIR LWIR
M1 M2 M3 M4
I1
M5 M7
I2
M8 M10 I3
0.412 0.445 0.488 0.555 0.64 0.672 0.865 0.865 1.24 1.61 1.61
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
M11 M12 M13 M15 I5 M16
2.25 3.7 4.05 10.76 11.45 12.01
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
VIIRS Visible, NIR and SWIR Bands
• bbb
– ccc
CO2
O2
O3
O2
H2O
H2O
O2
H2O
H2O
H2O
H2O
CO2
Current VIIRS Land PEATE HDF Usage
• VIIRS Land PEATE converts NOAA IDPS provided products
from HDF5 to HDF4 to be compatible with MODIS
– Aggregates data to L1/L2 data to ~5 min. granules
– Simple HDF attributes used for metadata
• All VIIRS Science Data Record (SDR, L1) and EDR
Environmental Data Record (EDR, L2/3) software have
been converted to run with HDF4 data
– I/O wrapper is used in algorithms
– DDR (Diagnostic Data Record, L2/3) software reads/writes HDF4
• NASA Direct Broadcast (DB) Laboratory is modifying I/O
wrapper to read/write standard IDPS HDF5 products to
support DB community
Wolfe - Nov 09 HDF Workshop
8
VIIRS HDF5 Commentary
Commentary – my views, not official Land PEATE views
• HDF5 implementation by IDPS has several major flaws
– Complex data structures were developed to support aggregation by
time and by products
– Simple HDF5 features (dimensions, units, etc.) are not being used;
instead these fields are stored as HDF attributes
– Un-aggregated granules are much shorter than 5 minutes (~1
minute) so they are many (many) files to deal with
– For SDRs, each band is stored separately, unless specifically
ordered as band aggregates by users
Wolfe - Nov 09 HDF Workshop
9
Planned VIIRS Land PEATE HDF Usage
Note: The VIIRS Land PEATE is funded to help the NASA
science team to evaluate the VIIRS SDR and Land EDR
products, not to distribute them (or the DDRs) to the general
science community
• There will likely be a “convergence” MODIS/VIIRS SDR and
Land EDR HDF5 product formats
– A “rational” HDF5-EOS format will be used (see earlier MODIS Land
HDF5 discussion)
– MODIS HDF4/5 converter will be modified to also work with VIIRS
– Likely to be available in the early VIIRS post-launch timeframe (mid2011)
– This will only be available to the general science community if NASA
policy changes
Wolfe - Nov 09 HDF Workshop
10
Planned VIIRS Ocean PEATE HDF Usage
Note: The VIIRS Ocean PEATE is funded to help the NASA
science team to evaluate the VIIRS SDR and Land EDR
products, not to distribute them (or the DDRs) to the general
science community
• If the NASA policy changes, there will likely be a
“convergence” MODIS/VIIRS Ocean EDR HDF5 product
formats produced by the VIIRS Ocean PEATE (see earlier
MODIS Ocean HDF5 discussion)
Wolfe - Nov 09 HDF Workshop
11
MODIS/Aqua
Haze over China and Korea
Oct. 28, 2009
Wolfe - Nov 09 HDF Workshop
12
Thank you!