Transcript Interoperable Information System of Systems for HTAP

Interoperable Information System of Systems
for
HTAP
Rudolf B. Husar and Rich Scheffe
With Erin Robinson
Presented at
HTAP Workshop, WMO, Geneva, January 26, 2006
Intercontinental Transport Events have
Influenced AQ Regulations ..and now Policy?
Web-Based Collab. in Virtual Workgroups
Near-Real-Time Data Access & Integration
US-Daily Avg. PM25 Dust:
Seasonal + Events
–> Air Quality Management
–> Science, the Process
–>Technologies and Tools
HTAP Characteristics
•
•
•
•
•
Stakeholders are autonomous countries, organizations
Diverse perspectives on the issues, solutions
Pollution sources and impacts are hemispheric
Information integration is key to success
Observations and models are ready for integration
The HTAP - GEOSS
•
•
•
•
•
•
Stakeholders are autonomous countries, organizations
Diverse perspectives on the issues, solutions
Recognition of the need to understand, act
Pollution sources and impacts are hemispheric
Observations and models are ready for integration
Information integration is key to success
Hence
GEOSS is a good system model for HTAP
HTAP is a suitable use case for GEOSS
Some Characteristics of System of Systems (SoS)
• Autonomous constituents managed independently
• Independent evolution of each constituent
• Displays emergent (novel, unpredictable) behavior
Summary
There is an ‘alignment of stars’ for integrating heterogeneous HTAP data
• System of Systems architecture is suitable for integrating HTAP data
– Standard data access is a key interoperability protocol
– Heterogeneous data can be non-intrusively standardized by mediators
– Service-based software architecture delivers tailored products to diverse uses
• Federated data and shared web-based tools are in use
– DataFed already includes over 100 datasets (emissions, ground, satellite)
– The system has been applied to EPA regulatory development
• However,
– DataFed is just one of the many mediator nodes, but these need to be connected
– Much more data would need to be federated
– HTAP model-data comparison would be an attractive use case
WMO-GAW could be an effective neutral facilitator for global data integration
using GEOSS (system of systems) principles
GEOSS Architecture Framework - DSS for HTAP
HTAP Reports:
Models
Model Estimates,
Obs. Evidence
Knowledge into
the Minds of
HTAP Knowledge
into the Minds of
AQ Analysts
Observations
Policy Makers
HTAP Knowledge
into the Minds of
Policy Analysts
Decision Support System
GEOSS
Architecture
Framework
Decisions
System of Systems: Interoperability
Connecting Machines and People
Interoperability Stack: Key concept of the Web
System components have to be interoperable at each layer
Amplify Individuals
Connect Minds
Open Architecture
Data Standards
Service Based
IP – Internet
Protocol
How do you agree on Interoperability?
Are WCS/WMS Space-TimeParameter queries
GetCapabilities
Query
Capabilities, ‘Profile’
Where? When? What?
Which Format?
GetData
Data
BBOX
When?
Time T1
What?
Temperature
Front
End
Ap
p
Ap
p
Ap
p
WCS/W
MS
Standards
GetData
Where?
Format
Client
Std. Interface
Back
End
Std. Interface
Server
What few things must be the same so
that everything else can be different?
OGC, ISO
T2
netCDF, HDF..
OGC, ISO
CF
CF, EOS, OGC
Convergence Protocols
Ap
p
Lets agree on
Space-Time-Parameter
Data Access Query
Protocol
From ‘Stovepipe’ to Federated Architectures
Data Providers
AIRNow
Model
DAACs
Info System
AIRNow
Compliance
Science
Info Users
Public
Manager
Scientist
• Current info systems are project/program oriented and provide end-to-end solutions
• Part of the data resources of any project can be shared for re-use through DataFed
• Through the Federation, the data are homogenized into multi-dimensional cubes
• Data processing and rendering can then be performed through web services
• Each project/program can be augmented by Federation data and services
Web Services and Workflow for Loose Coupling
Data
Datasets
Decision Support System
Std. Interface
Std. Interface
Obs. & Models
Control
Reports
Data Views
Web Service Interaction
Service
Broker
Service
User
Bind
Service
Provider
Integrated Data
Reports
Service Chaining & Workflow
Collaborative Reporting and Dynamic Delivery
Data
Datasets
Decision Support System
Std. Interface
Std. Interface
Obs. & Models
Data Views
Control
Reports
Integrated Data
Analysis Reports:
Information supplied by many
Needs continuous program feedback
Report needs many authors
Wiki technologies are for collaborative writing
Dynamic Delivery:
Much of the content is dynamic
Animated presentations are compelling
Movies and screencasts are for dynamic delivery
Reports
Co Writing Wiki
Screencast
Chemical Data
Ft. Collins, CO
AeroCom
Chemical Models
Paris, FR
ModelData Comparison
Workflow
Std
I/O
VIEWS
Std
I/O
Model-Data Comparison Workflow Software
Workflow Flow Program
Lego-like assembly of component
WMS, WCS OGC
Services
Sample of Federated Datasets
• HTAP Model-Obs “Test/Benchmark”ed
– A virtual data center design to meet the needs of the TF HTAP
model intercomparison and evaluation
– Archives of key observations and model data
– Tools for model intercomaprison
Tools..
Analyst Console
User Layout
Imagine…More Shared Global Obs & Models
2007
Global Data & Models
2007++
More Global Data & Models
The Network Effect:
Less Cost, More Benefits through Data Reuse
Programs ask/get Data
Data
Data
Data
Orgs Develop Programs
Program
Organization
Program
Data Re-Use
Network Effect
Public
Program
Data
Data
Data
Public sets up Orgs
Organization
Program
Pay only once
Richer content
Less Prog. Cost
More Knowledge
AeroCom
Obs &
Services
Other
Federations
Less Soc. Cost
More Soc. Benefit
CAPITA
Models
Obs &
Model
Federated Information System
Observations, Models, Services
Services
Obs
VIEWS
Models & Services
Agile Information System
Design Principles
Guiding Idea:
System of Systems
GEOSS
Domain of
Interaction
WMO/GAW & Stakeholders
Infrastructure:
Methods, Tools:
Internet, Middleware,
Web 2.0
Standards, Use Cases
Based on P. Senge et. al, 1994: Architecture of Learning Organizations(Link)
R. Scheffe
Apparent divergences?
Organizations have different missions
NPS
USDA
Protect ecosystems,
AQ WQ
Protect/optimize Ag
and forest resources
NASA
Explore fundamental
Earth System Properties
EPA (Protect human health & envi.
Improve air, water, ecosystem
NOAA
To understand and predict changes
in the Earth’s environment and …
CDC
To promote health and
quality of life ….
R. Scheffe
Stars aligned?
Confluence again..
Observation technologies
{e.g., satellites}
Collaboration (culture)
empire building?
Computational power
Alignment
Budgets, agency collaboration
resource/program accountability
Accountability, ↓ regulatory
assessments {e.g., NAS, CASAC}
Science, talent
{embodied in AQ models
and young geniuses}
Information technologies
{e.g., data sharing protocols}
Summary
There is an ‘alignment of stars’ for integrating heterogeneous HTAP data
• System of Systems architecture is suitable for integrating HTAP data
– Standard data access is a key interoperability protocol
– Heterogeneous data can be non-intrusively standardized by mediators
– Service-based software architecture delivers tailored products to diverse uses
• Federated data and shared web-based tools are in use
– DataFed already includes over 100 datasets (emissions, ground, satellite)
– The system has been applied to EPA regulatory development
• However,
– DataFed is just one of the many mediator nodes, but these need to be connected
– Much more data would need to be federated
– HTAP model-data comparison would be an attractive use case
WMO-GAW could be an effective neutral facilitator for global data integration
using GEOSS (system of systems) principles