Transcript Cyber-Infrasructure Overview

Development of a Community
Hydrologic Information
System
David G Tarboton
Jeffery S Horsburgh, David R. Maidment (PI),
Tim Whiteaker, Ilya Zaslavsky, Michael
Piasecki, Jon Goodall, David Valentine,
Thomas Whitenack
[email protected]
CUAHSI
HIS
http://his.cuahsi.org/
Sharing hydrologic data
Support
EAR 0622374
Outline
• CUAHSI and HIS data publication system
• WaterML and WaterOneFlow web
services
• Observations data model (ODM)
• Data publication system
• Analysis using HIS
• Implementation
• Performance
Consortium of Universities for the
Advancement of Hydrologic Science, Inc.
• 110 US University
members
• 6 affiliate members
• 12 International
affiliate members
(as of March 2009)
An organization representing more than one hundred
United States universities, receives support from the
National Science Foundation to develop infrastructure
and services for the advancement of hydrologic
science and education in the U.S.
http://www.cuahsi.org/
What is the CUAHSI HIS?
Discovery
Access
Modeling
Analysis
Hydroseek
HIS Desktop
HydroLink + OpenMI
MATLAB, Excel,
GIS, R, …
Web services and Schema
WaterOneFlow and WaterML
to transmit hydrologic data
in a standard way
Observatories
publication
and archival
of field
data
Data repository
HIS Server
ODM
store and
share
hydrologic
data
GetSites
GetSiteInfo
GetVariableInfo
GetValues
Service Registry
HIS Central
catalog
hydrologic
data services
and metadata
3rd-Party Servers
include
data from
others
An internet based system to support the sharing of hydrologic data comprising databases
connected using the internet through web services as well as software for data discovery, access
and publication.
WaterML and WaterOneFlow
WaterML is an XML language for communicating water data
WaterOneFlow is a set of web services based on WaterML
• Set of query functions
• Returns data in WaterML
NWIS Daily Values (discharge), NWIS Ground Water,
NWIS Unit Values (real time), NWIS Instantaneous
Irregular Data, EPA STORET, NCDC ASOS, DAYMET,
MODIS, NAM12K, USGS SNOTEL, ODM (multiple sites)
Slide from David Valentine
WaterML design principles
• Goal - capture semantics of hydrologic observations
discovery and retrieval
• Role - exchange schema for CUAHSI web services
• Driven by
– Hydrologists (community review)
– ODM
– USGS NWIS, EPA STORET, Academic Sources
• Conformance with Open Geospatial Consortium standards.
http://www.opengeospatial.org/
• For XSD pros, the WaterML schema is at
http://his.cuahsi.org/wofws.html
Slide from David Valentine
Values
• Each data value recorded in value element
• Timestamp, plus metadata for the value,
recorded in element’s attributes
qualifier
ISO Time
value
Slide from David Valentine
What are the basic attributes to be
associated with each single data value and
how can these best be organized?
Value
Offset
DateTime
Variable
OffsetType/
Reference Point
Location
Units
Source/Organization
Interval
(support)
Accuracy
Data Qualifying
Comments
Censoring
Method
Quality Control
Level
Sample Medium
Value Type
Data Type
CUAHSI Observations Data Model
Streamflow
Groundwater
levels
• A relational database at the
single observation level
Precipitation
Soil
(atomic model)
& Climate
moisture
• Stores observation data
data
made at points
Flux tower
Water Quality
• Metadata for unambiguous
data
interpretation
• Traceable heritage from raw
“When” Time, T
measurements to usable
t
A data value
information
vi (s,t)
• Standard format for data
s “Where”
sharing
Space, S
• Cross dimension retrieval
Vi
and analysis
“What”
Variables, V
CUAHSI Observations Data Model http://his.cuahsi.org/odmdatabases.html
Horsburgh, J. S., D. G. Tarboton, D. R. Maidment and I. Zaslavsky, (2008), A Relational Model for
Environmental and Water Resources Data, Water Resour. Res., 44: W05406, doi:10.1029/2007WR006392.
Water Chemistry from Laboratory Sample
Stage and Streamflow Example
CUAHSI HIS Data Publication System
Query, Visualize, and Edit
data using ODM Tools
Analysis
Discovery
Access
Hydroseek
GIS
Matlab
Splus
R
IDL
Streaming
Data
Loader
ODM
Database
Base Station
Computer(s)
Telemetry
Network
GetSites
GetSiteInfo
GetVariableInfo
GetValues
WaterML
ODM
Data
Loader
Excel
Sensors
Java
C++
VB
HydroExcel
HydroGet
HydroLink
HydroObjects
Service Registry
Hydrotagger
Harvester
WaterOneFlow
Web Service
ODM
Text
Contribute
your ODM
http://his.cuahsi.org
Water Metadata
Catalog
ODM
HIS Central
Dynamic controlled vocabulary moderation system
ODM Data
Manager
ODM
Website
ODM
Tools
XML
Local ODM
Database
Local
Server
ODM Controlled
Vocabulary Moderator
ODM
Controlled
Vocabulary
Web Services
http://his.cuahsi.org/mastercvreg.html
Master ODM
Controlled
Vocabulary
From Jeff Horsburgh
Direct analysis from your favorite analysis
environment. e.g. Excel, MATLAB
Direct analysis from your favorite analysis environment.
e.g. Matlab
% create NWIS Class and an instance of the class
createClassFromWsdl('http://river.sdsc.edu/wateroneflow
/NWIS/DailyValues.asmx?WSDL');
WS = NWISDailyValues;
% GetValues to get the data
siteid='NWIS:02087500';
bdate='2002-09-30T00:00:00';
edate='2006-10-16T00:00:00';
variable='NWIS:00060';
valuesxml=GetValues(WS,siteid,variable,bdate,edate,'');
2.5
x 10
4
Daily Discharge NEUSE RIVER NEAR CLAYTON, NC
2
cfs
1.5
1
0.5
0
1920
1930
1940
1950
1960
1970
1980
1990
2000
2010
Federal water data available through
HIS Central (http://river.sdsc.edu)
• Operational
• Development
– USGS
– NCDC
– SNOTEL
•
•
•
•
Daily Values
Instantaneous Values
Irregular Data
Groundwater
– EPA Storet
– MODIS
– NCEP North American
Model
– DayMet
HIS Implementation in WATERS
Network Information System
National Hydrologic Information Server
San Diego Supercomputer Center
• 11 WATERS Network test bed projects
• 16 ODM instances (some test beds have more than one ODM
instance)
• Data from 1246 sites, of these, 167 sites are operated by WATERS
investigators
Prototype Texas HIS
• TWDB is supporting a small project at
University of Texas to start building a
prototype Texas Hydrologic
HIS servers at
data sources
Information System
(State agencies,
River authorities,
Water Districts, Cities,
Counties….)
Web
Services
Texas Hydrologic Information Server (at TNRIS)
Texas Observations Catalogs and some state water datasets
Intermountain Constellation of Experimental
WATERsheds (ICEWATER)
The Inland Northwest Research
Alliance (INRA) Water Research
Consortium is establishing a HIS
Network to share experimental
watershed data to address
regional water resources
challenges
Great Salt Lake Information System
A common window for accessing, viewing and downloading
data and information in the Great Salt Lake Basin
Local Organizations
State Agencies
Universities
Federal agencies
Conclusions
• Data Storage in an Observations Data Model
(ODM)
• Data Access through internet-based Water
Data Services using a consistent data language,
called WaterML
• Data Indexing through a National Water
Metadata Catalog
• Data Discovery through federated map and
thematic keyword search system
The combination of these capabilities creates a
common window on water observations data for the
United States unlike any that has existed before.
HIS Team and Collaborators
• University of Texas at Austin – David Maidment, Tim
Whiteaker, Ernest To, Bryan Enslein, Kate Marney
• San Diego Supercomputer Center – Ilya Zaslavsky, David
Valentine, Tom Whitenack
• Utah State University – David Tarboton, Jeff Horsburgh, Kim
Schreuders, Justin Berger
• Drexel University – Michael Piasecki, Yoori Choi
• University of South Carolina – Jon Goodall, Tony Castronova
• Idaho State University – Dan Ames
• CUAHSI Program Office – Rick Hooper, David Kirschtel,
Conrad Matiuk
• WATERS Network – Testbed Data Managers
• HIS Standing Committee
• USGS – Bob Hirsch, David Briar, Scott McFarlane
• NCDC – Rich Baldwin
HIS Overview Report
• Summarizes the
conceptual framework,
methodology, and
application tools for
HIS version 1.1
• Shows how to develop
and publish a CUAHSI
Water Data Service
• Available at:
http://his.cuahsi.org/documents/HISOverview.pdf