Transcript Slide 1
Experiments with Remote Access to Scientific Data using HDF5 Kun Yan Robert E. McGrath ([email protected]) NCSA May 15, 2002 Today’s Talk 1. Background 2. Experiments with JSP, XML, HDF4, HDF5, etc. 3. Experiments with CORBA, Java, HDF5 4. Discussion Acknowledgements Kun Yan conducted these experiments as a Research Assistant at NCSA. This report is based upon work supported in part by a Cooperative Agreement with NASA under NASA grants NAG 5-2040 and NCC5599. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Aeronautics and Space Administration. Other support provided by NCSA and other sponsors and agencies. HTML Web browser Java Java C Java Server Platform Java Native Interface CORBA Server C++ HDF Library and File C XML Any Applet Java Native Interface C Java Java Other App. Other App. H5view, etc Any Any Client/Remote Distributed Product Demonstrated in Research Should work, but not demonstrated Server/Local Java 2. Experiments with JSP • Experiments with JSP, XML, XSL, etc. – Web access to Data and services. – Demonstration of browsing HDF5 datasets • Generate HTML or XML • Convert data from HDF4 to HDF5 on demand http://arabica.ncsa.uiuc.edu:7070/hdf5/ Tomcat • is a standalone web server that supports the Servlet and JSP specifications. • developed under the Jakarta project at the Apache Software Foundation. • integrated into the J2EE Reference Implementation. • Tomcat 4.x (Catalina, Servlet 2.3, JSP 1.2) • Tomcat 3.3 JavaServer Pages • JavaServer PagesTM (JSPTM) technology • rapidly develop and easily maintain, dynamic web pages • uses XML-like tags and Java scriptlets • application logic can reside in serverbased resources The Flow of a Request response request HTML Xalan.jar Xerces.jar client JSP response XML beans beans beans Application XSL DB lib How JSP and Beans Work together data 3. Experiments with CORBA • Created a demonstration CORBA server that reads HDF5 – Written in C++, linked directly to HDF5 library. • Created a pure-java client – No CORBA required on the client – Can be used from an applet within a standard Web browser Technology used • Orbacus 4.1.0 (C++) (free, full source) • Java 1.3 or above • HDF5.1.4.3, HDF5 C++ interface HDF5 CORBA Experiment -- Design Diagram CORBA Object Hierarchy HDF5 CORBA Experiment – IDL File 1. module HDF5{ 2. 3. 4. 5. 6. 7. 8. 9. 10. struct ObjInfo{ long oid; long fid; long type; string name; long numOfMembers; }; 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. struct DatasetInfo{ string name; long numOfDim; Sizes dimSizes; Sizes dimMaxSizes; long dataTypeClass; long dataTypeSize; long long storageSize; }; 26. 27. interface H5Obj{ void getObjInfo(out ObjInfo obji); typedef sequence<long> Sizes; struct DatatypeInfo{ string name; long type; long size; }; 28. 29. 30. 31. void getNumAttributes(out long attrNum); void openAttributes (out long attr); }; 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50. 51. 52. interface H5DatasetObj:H5Obj{ void getDatasetInfo(out DatasetInfo dsinfo); long long getStorageSize(); }; 53. 54. 55. 56. interface H5GroupObj:H5Obj{ ObjInfo getGMember(in long fid, in string gName, in long index); void openGroup(in string gName, out H5GroupObj h5gobj); void getDatasetInfo(in string dsName, out DatasetInfo dsinfo); void openDataset(in string dsName, out H5DatasetObj h5ds); void getDataTypeInfo(in string ttName, out DatatypeInfo dtinfo); }; interface H5FileObj{ void getTOC(out ObjInfo obji); void openRootGroup(out H5GroupObj h5gobj); }; interface H5FileAccess{ void openH5File(in string fileName, out H5FileObj h5fobj); void numOfGMembers(in long fileId, in string gName, out long gNum); string closeH5File(in long fileId); }; }; HDF5 CORBA Experiment -- Design Diagram C++ Servant Object Hierarchy H5FileAccess H5Obj H5FileObj H5DatasetObj H5GroupObj H5Obj_impl ObjInfo poa_ H5FileAccess_impl H5FileObj_impl H5::H5File poa_ H5::H5File ObjInfo poa_ openFile: closeFile: getTOC openRootGroup setFileName: getObjInfo setObjName: H5GroupObj_impl H5::H5Group poa_ openGroup getGMembers openDataset: getDatasetInfo: getDatatypeInfo: H5DatasetObj_impl H5::H5Daraset DatasetInfo getDatasetInfo getStorageSize HDF5 CORBA Experiment -How to Use the Client Applet • • • • • • • • • • Select file name from drop-down list (file resides corba server side) Select Flags and Access Property of the selected file Click on “open file” to open the file Click on “get file TOC” to get the file object info Click on “open root” to open the root group Click on “get group TOC” to get the root group object info Click on “iterate group” to get the group member’s info of the root and each subgroup Input dataset name including path, then click on “get dataset” to get the dataset object – Click on “get dataset TOC” to get the dataset object info – Click on “get storage size” to get the storage size of the opened dataset object Input dataset name including path, then click on “get dataset info” to get object info of the given dataset Input datatype name including path, then click on “get datatype info” to get object info of the given datatype URLs • JSP experiments: http://hdf.ncsa.uiuc.edu/HDF5/XML/JSPExperiments/ • • • • http://hdf.ncsa.uiuc.edu http://hdf.ncsa.uiuc.edu/h4toh5/ http://hdf.ncsa.uiuc.edu/HDF5/XML/ http://hdf.ncsa.uiuc.edu/hdf-java-html 4. Discussion HTML Web browser Java Java C Java Server Platform Java Native Interface CORBA Server C++ HDF Library and File C XML Any Applet Java Native Interface C Java Java Other App. Other App. H5view, etc Any Any Client/Remote Distributed Product Demonstrated in Research Should work, but not demonstrated Server/Local Java HDF5 CORBA Experiment -- User Interface HDF5 CORBA Experiment -- Examples of Object Info window Other HDF Tools HDF4 HDF5 file h4toh5 XML HDF5 file Viewer/ Editor XML ASCII HDF4 HDF5 file XML Generator Source code HDF DDL HDF5 file Dumper XML HDF4 file h4toh5 HDF5 H4 to H5 LIbrary HDF5 file h5toh4 HDF4 GIF file gif2hdf HDF5 flat file fp2hdf HDF5 Silly XMLTricks Netcdf to HDF5, via XML using XSL netcdf ncxdump nc.xml netcdf.dtd xalan nctoh5.xsl hdf5 h5gen h5.xml URLs • JSP experiments: http://hdf.ncsa.uiuc.edu/HDF5/XML/JSPExperiments/ • • • • http://hdf.ncsa.uiuc.edu http://hdf.ncsa.uiuc.edu/h4toh5/ http://hdf.ncsa.uiuc.edu/HDF5/XML/ http://hdf.ncsa.uiuc.edu/hdf-java-html