Transcript Software to access HDF5 Datasets via OPeNDAP

Software to access HDF5
Datasets via OPeNDAP
MuQun Yang, Hyo-Kyung Lee
The HDF Group
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
1
Previous work review
HDF5-OPeNDAP server:
•
•
•
•
Incomplete mapping from HDF5 to OPeNDAP
No comprehensive test suite
Only tested a few examples on two platforms
Incomplete documentation
HDF5-OPeNDAP client:
• No client at all
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
2
Previous HDF5OPeNDAP Support
HDF5
DAP
No Client
11/8/2007
Prototyped Server
HDF and HDF-EOS Workshop XI, Landover, MD
3
Proposed work
First Year
HDF5 server
Clients: Matlab etc.
DAP
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
4
HDF5
Current Status
HDF5 server
Clients: Matlab etc.
DAP
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
5
HDF5
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
6
HDF5 Server Status – New Features
•
•
•
•
Mapping Compound datatype to DAP Structure
*Mapping EOS Grid to DAP Grid
*Mapping HDF5 group to DAP
*Mapping HDF5 object and regional
references to DAP
D
O
N
E
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
7
HDF5 Server Status - Test
•
•
•
•
Code Cleanup – updating to the latest version of HDF5
Comprehensive testsuite – DejaGNU
Internal Aura EOS file testsuite
Inside OPeNDAP’s nightly build system
DONE
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
8
HDF5 Server Status – Special Support
•
•
•
•
Partial Support of CF conventions
Relative Object Path Support
Configuration Options
HDF5-OPeNDAP handler Website
DONE
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
9
HDF5 Server Status - Documentation
•
•
•
•
Mapping HDF5 group to DAP -- Done!
Limitations of HDF5-OPeNDAP handler – In progress!
Mapping Object reference to DAP – In Progress!
Experiments with OPeNDAP clients – In Progress!
In Progress !
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
10
Using OPeNDAP Clients to serve HDF5 data
• Matlab Client – in progress
• Testing OPeNDAP Clients to read NASA HDF-EOS5
files
Ferret
ncBrowse
IDV
ODC
GrADS
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
11
Several Highlights
• HDF5-OPeNDAP Handler Website
• Limitations of the current handler
• Demo: Visualizing HDF-EOS5 Aura data with existing
OPeNDAP clients
• Limitations of the existing clients
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
12
HDF5 OPeNDAP Website
• URL: http://hdfdap.hdfgroup.uiuc.edu:8080/
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
13
Limitations of the handler
• No support for HDF5 files that have a '.' inside a
group/dataset name
• No support for any variable length types except for 1
dimensional variable-length string array
• No support for ENUM,BITFIELD,OPAQUE types
• 64 bit integer (array) is not supported
• Both signed and unsigned char map to Byte
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
14
Demo
• Ferret
• NCBrowser
• IDV
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
15
Summary of accessing Aura L3G Data via OPeNDAP clients
Client
Problem
Solution
Notes
N/A
2D display only. No
way to latitude and
longitude information.
ncBrowse
N/A
Ferret
Geolocation fields should fit
Attributes with units
map geolocation value to
into longitude(-180~180)
should be provided
longitude and latitude
and latitude (-90~90) range
for correct display.
IDV
Strictly Follows CF
convention.
Need Short names(<15
characters for variables)
and need units for
attributes.
GrADS
Strictly Follows CF
convention. Maybe 3D(x,y,time) Grids only?
It entirely depends on
Renaming grid map data
libdap-nc client
variables?
library.
ODC
N/A
N/A
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
Beautiful user
interface.
It can't display data.
Data retrieval is OK.
16
Second year Plan
• High-level HDF5 client
• Working with GrADS team
• We really would like to hear feedbacks from Users
Please send emails to
[email protected]
[email protected]
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
17
Acknowledgement
This work was supported basing upon the Cooperative
Agreement with the National Aeronautics and Space
Administration (NASA) under NASA grant
NNX06AC83A and NNX06AG75A. Any opinions,
findings, and conclusions or recommendations expressed
in this material are those of the author(s) and do not
necessarily reflect the views of NASA.
11/8/2007
HDF and HDF-EOS Workshop XI, Landover, MD
18