Transcript Registration/Visualization of Volume Data in netCDF format Ashraf Memon www.geongrid.org

Registration/Visualization of
Volume Data in netCDF format
Ashraf Memon
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
Acknowledgement
• Chuck Meertens (UNAVCO)
• Unidata
• NSF Grant No. 0225673 (GEON)
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
Overview
• Introduction
– Usecase
• Background
–
–
–
–
–
GEONgrid portal
netCDF data format
Opendap Server
Thredds Catalog
Thredds Data Server
• Architecture
– Overview
– Registration
– Visualization
• Action Items and Timeline
• Conclusion
• Questions
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
Introduction
• UseCase
– “Searching, Accessing, Integrating, and
Visualizing Distributed Data” – (Courtesy:
Chuck Meertens)
– Specific Example:
Searching, Accessing, Integrating and
Visualizing distributed volume data stored in
netCDF format and/or served on openDAP
server. Performing these tasks through
GEONgrid Portal.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
Background
• GEONgrid Portal
GEON Portal incorporates
- Search, Contribute, Access,
Integrate, Process, and
Visualization of resources (data,
tools, applications, and ontologies)
-Data Formats and Service types
supported
-Shape
-RDBMS
-GeoTIFF
-EXCEL
-OGC WMS Services
-ASCII
-Web Services
-Arbitrary Data Objects
(CUAHSI .zip file bundles)
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
Search for Resources
Current Types:
ASCII
CUAHSI Data
GMT Raster
GeoTiff
Relational Database
Shapefile
Tool
WMS Service
Web Service
Future to handle 4-D Data:
netCDF
OPeNDAP
Search Constraints:
Metadata Relation, Type,
Subject, Keyword
Spatial Coverage
Temporal Coverage
Ontology/Concept Relation
Working in the GEON Portal:
GEON Search with constraints
Sample search for:
Resource Type - “Relational Database”
Keyword - “GPS”
(Courtesy: Chuck Meertens)
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
netCDF
• NetCDF (network Common Data Form) is
an interface for array-oriented data access
for representing scientific data and a
library that provides an implementation of
the interface.
• Together, the interface, library, and format
support the creation, access, and sharing
of scientific data.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
netCDF
• A dataset has
dimensions,
variables, and
attributes.
• Attributes can be
global or apply to
individual
variables.
• There is a very
limited set of low
level data types.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
OPeNDAP
• The Open Source Project for a Network
Data Access Protocol (OPeNDAP)
• The DAP has been designed to be as
general as possible without being
constrained to a particular discipline or
world view.
• The DAP is a discipline neutral data access
protocol; it can be used in astronomy,
medicine, earth science,…
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
OPeNDAP Server
• Servers receive requests and provide responses via the
DAP.
• Servers convert the data from the form in which they are
stored to the OPeNDAP data model.
• Servers provide for subsetting of the data.
CODAR netCDF HDF4
Data
Data
CODAR
netCDF
Data
Matlab
DSP
Tables
SQL
FITS
CDF
Data
Data
Data
Data
Data
Data
Matlab
HDF4
JGOFS
DSP
FITS
JDBC
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Flat
Binary CEDAR General
Data
Data
ESML
FreeFrom
CDF
Data
CEDAR
www.geongrid.org ‹#›
OPeNDAP Clients
netCDF C
Ferret
GrADS
netCDF Java
IDV
VisAD
Matlab
Client
IDL
Client
Matlab
IDL
ncBrowse
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
Access
Excel
www.geongrid.org ‹#›
THREDDS Catalog
• THREDDS (Thematic Realtime
Environmental Distributed Data Services)
• THREDDS is
– a prototype system for cataloguing, browsing,
and accessing remote, retrospective, and other
data types through a set of middleware
technologies.
– is infrastructure needed for publishing and
accessing scientific data in a similarly
convenient fashion.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
THREDDS Data Server
HTTP Tomcat Server
IDV
Catalog.xml
THREDDS Server
•OPeNDAP
Application
•HTTPServer
•WCS
NetCDF-Java
library
WCS
Application
OPeNDAP
hostname.edu
Datasets
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
WMS
Server
www.geongrid.org ‹#›
THREDDS Data Server
• Integrates data access with THREDDS
catalogs and services
• Tomcat/Servlet, 100% Java, single war file
• Data input is netCDF Java 2.2 library
• Data output:
– OPeNDAP
– HTTP Server
– OGC Web Coverage Server (gridded)
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
UNAVCO/GEON PoP Server Details
Data, Models, Catalogs, Metadata
Data Access
http
srb
ftp
gridftp
OGC*
UNAVCO/GEON
PoP Data Server
1D/2D/3D/4D
OPeNDAP
Thredds
-Catalog elements
Digital Library
Metadata elements
Thredds
Catalog
OPeNDAP
Data
Servers
-netCDF
-Freeform
(via OAI or DLESE
Webservice)
WMS Catalog
*OGC=WMS/WFS/WCS
Dataset Catalogs and Metadata Access
*OGC: WMS/WCS/WFS
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
-Tomography
-GPS data/vectors
-Earthquakes
-Focal Mechanisms
-Strain rate
-Topography
-Image maps
-Geodynamics
-Faults
-Paleogeography
Plus IDV visualization
“Bundles” (.xidv files)
www.geongrid.org ‹#›
UNAVCO/ GEON Data Server
and Visualization Access
UNAVCO/GEON
Data Node
OPeNDAP Server
Example: Seismic
Tomography in NetCDF
format
WMS Server
Example: GPS
Seamless Archive
Postgres/PostGIS
Database
THREDDS/OPeNDAP
Catalog
Integrative Data Viewer
WMS Catalog
HTML File
GEON goal, by May, 2006: registration of netCDF
files and OPeNDAP servers into the GEON Portal;
Other
distributed data
servers
Local File
embedded GEONSearch within the IDV
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
Users view
GEON Portal
Volume Data
(ascii grid
or netCDF)
Data
Contribut
or
• Data/Model
Registration
Data
•Data Search /
Discovery
GEON Query &
Data Preview
User Contributed
“Concepts”
Thredds
Processed -Catalog
Data
-Digital Library
Metadata
Raw
Data
Data access
Data User
Metadata Access
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
UNAVCO GEON PoP
• Data/Metadata Server Node
• Data Conversion Webservice
www.geongrid.org ‹#›
Data Registration Architecture
OPeNDAP
Register OPeNDAP URL (URL, ADN, FGDC)
Register netCDF (URL, ADN, FGDC)
Register xidv (.xidv, ADN, FGDC)
User
Machine
Data
Registration
Service
.nc
.xidv
Archival
OPeNDAP
URL
adn
OPeNDAP
URL
THREDDS
WCS
fgdc
SRB
HPSS
.nc
GEON
Catalog
THREDDS
Data Server
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
Search, Access and Visualization
Webstart (.jnlp, ixidv)
User
Machine
Download, OPeNDAP, WCS, JPG
.jnlp(.xidv)
Tools
Repository
wms
Data Integration
And Visualization
Service
adn
fgdc
.nc
OPeNDAP
URL
GEON
Catalog
OPeNDAP
URL
THREDDS
WCS
Tools
OPeNDAP
URL
THREDDS
Data Server
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
Action Items
• Installation and Understanding TDS (1 week)
• GEON hosts TDS (WCS/OPeNDAP/THREDDS) Server of hosted
integrated data (1/2 week)
• Data registration system for .nc file (3 Weeks)
• Data Registration for OPeNDAP URL (3 Weeks)
• netCDF download from GEON-hosted data (possible http link to
data) (Already Implemented)
• GEON points to non hosted THREDDS server (2 weeks)
• GEON hosts THREDDS catalog of integrated data (only selected
data, proxy server-could be non hosted service as well as GEON
hosted) (2 weeks)
• GEON provides webstart to IDV with IDV catalog (points to hosted
and non hosted data) (1 week)
• GEON provides webstart to IDV with IDV catalog a default view
appropriate for type of data (1 week)
• GEON also provides a jpg preview from IDV scripting API (2 days)
• GEON to host WCS through TDS (1 week)
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›
Questions
• We are opening the phone lines for
questions, please call in at
1.888.629.9921 and put in the passcode
5383118#
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES
www.geongrid.org ‹#›