Transcript Open Data Access Protocol - University of Rhode Island

Open Data Access Protocol
Webb Pinner
May 2, 2005
CSC 536
Acknowledgements
Peter Cornillan - Graduate School of
Oceanography, DODS
James Gallager - OpenDAP.org
Tom Sgouros - OpenDAP.org
John Howland - Deep Submergence Lab,
Woods Hole Oceanographic Institute
Outline of Presentation
Background Information
The OpenDAP protocol
Oceanographic Dive Reply System
Summary
Questions?
Background
• DODS was created in 1993 by URI,
GSO, MIT, NASA and NOAA for use in
oceanographic datasets.
• It's purpose was to "define the structure
of a client-server based distribution
system for access to oceanographic
data over the Internet”
Background (cont.)
• DODS was quickly adapted for use by
the atmospheric data community.
• Because DODS no longer dealt with
strictly with ocean data the name was
changed to the Data Access Protocol
• 2002 the project was made open source
and thus OpenDAP was created
The OpenDAP Architecture
•
•
•
•
•
•
Provides uniform interface to data
Allows SQL-like data querying
Read-only access to data
Platform independent
Client/server model
For use over Internet (TCP/IP)
OpenDAP Implementation
• Uses http as transport protocol
– Server implemented as CGI extension in
Apache webserver
– Requests for data are sent to server in the
form of a URL
• URL sent to server describes exactly
what data is requested
– Includes which columns and/or data
constraints.
Example URL
• http://dap_server/cgi-bin/nph-dods/data/dataset1
–
–
–
–
–
http - protocol
dap_server - servername
cgi-bin/nph-dods - dap server software location
data - director where dataset is located
dataset1 - name of dataset
• Extension determines which service is called
• Constraints may be added to request specific
datasets
OpenDAP Implementation
(Services)
• Three basic services:
– Data Descriptor Service (DDS)
– Data Attribute Service (DAS)
– Data (DODS)
• Additional services:
– HTML form (HTML)
– ASCII representation (ASCII)
OpenDAP Implementation
(Constraints)
• Constraints specify which fields to
display and in what order to display
them.
– http://…/data.dods?date&time&temp
• Constraints specify search criteria.
– http://…/data.dods?temp>100&temp<175
Current Users
• NASA - Global Hydrology Resource Center
• NOAA - National Data Buoy Center (NDBC)
• NOAA - National Oceanographic Data Center
(NODC)
• University of Hawaii/SOEST - Sea Level
Center (SLC)
• Woods Hole - Upper Ocean Mooring Data
Archive (UOP)
OpenDAP DEMO
OpenDAP =
Middleware/Dist. System?
• Kinda, sorta, not quite
• Uniform interface to data (Middleware)
• User is unaware of OS, storage format
or access protocol (Middleware)
• Data distributed across multiple servers
(Dist. System)
OpenDAP =
Middleware/Dist. System?
• Read-only Access
• Each Dataset is independent of other
datasets
• No such thing as replication, commit
protocols, concurrency control, etc.
Oceanographic Dive Replay
System
Oceanographic Dive Replay
System
• Replay a dive complete with textual
data and images
• Retrieve data from remote servers
• Provide a graphical means to interact
with large ascii datasets
• Provide means to search dataset
WHOI - Virtual Vans
WHOI - Virtual Vans
• Replays a dive complete with textual
data and images
• Retrieves data from WHOI server
• Provide a HTML based GUI
• Provide means to search event data
Virtual Vans - Shortcomings
• Unable to search by any other dataset
• Unable to create custom datasets for
downloading
• GUI can not dynamically accommodate
new sensors.
• All datasets stored at one location
OpenDAP Replay System
• Use DAS, DDS information to
dynamically generate interface
• Use URL constraints to search based
on any data type
• Use URL constraints to customize
downloaded datasets
• Multiple servers may provide datasets
OpenDAP Replay System
Summary
• Introduced the OpenDAP protocol, it’s
history, architecture and current uses
• Quick OpenDAP demo
• Case Study: Dive Replay System
• Looked at existing system and its faults
• Discussed a new OpenDAP-based
systsm and its advantages
Questions?