Transcript Dynamically Coupling Sensor Web with Earth System Models - The Earth Predictive Systems (SEPS) Middleware to Support AIP2 Dr.

Dynamically Coupling Sensor Web with Earth System Models - The
Earth Predictive Systems (SEPS) Middleware to Support AIP2
Dr. Liping Di
Center for Spatial Information Science and Systems
George Mason University
[email protected]
September 26, 2008
CSISS
Page
LCenter for Spatial Information Science
and Systems
1
Overall SEPS Architecture
1.
2.
3.
4.
FTP, HTTP, WCS-T, WFS-T
SPS
THREDS, ECHO, CSW
SPS
5.
6.
7.
8.
OpenDAP, WCS, WFS
SOS
CSW, WFS, WCS
ESM state
9.
10.
11.
WCS, WFS
WNS
WCS, WFS
CSISS
Center for Spatial Information Science and Systems
Page 2
SEPS Standard Interfaces for Connection to Sensor Web
• ISO 19130 – Sensor Data Model for Imagery and Gridded Data,
provides the conceptual framework for enabling the sensor web.
• OGC Sensor Web Enablement (SWE) initiatives have
developed six implementation specifications for the sensor web
– SensorML, a general model and XML encoding of the geometric,
dynamic, and observational characteristics of a sensor;
– Observation and Measurements (O&M), a framework and GML
encoding for measurements and observations;
– Sensor Planning Service (SPS), by which a client can determine
the feasibility of a desired set of collection requests for one or more
mobile sensors/platforms;
– Web Notification Service (WNS) by which a client may conduct an
asynchronous dialog with one or more other services;
– Sensor Alert Service (SAS) by which a client can register for and
receive sensor alert messages, part of the OGC Sensor Alert
Service Interoperability Experiment;
– Sensor Observation Service (SOS) providing an API for managing
deployed sensors and retrieving sensor data.
CSISS
Center for Spatial Information Science and Systems
Page 3
Standard Interfaces to ESM
• The modeling community has developed the standard for
interoperability between models—The Earth Science Modeling
Framework (ESMF).
• ESMF is a high-performance, flexible software infrastructure to
increase the ease of use, performance portability,
interoperability, and reuse in climate, numerical weather
prediction, data assimilation, and other Earth science
applications.
– It defines an architecture for composing multi-component
applications and includes data structures and utilities for developing
model components.
– It defines standard function methods for organizing a model
component.
– It also defines an ESMF data structure (ESMF state) for
communication between components.
– The SEPS framework has to be able to provide the standard ESMF
interfaces to the model.
CSISS
Center for Spatial Information Science and Systems
Page 4
Standard interfaces among the SEPS components
• OGC Data Interoperability Protocols
–
–
–
–
Web Map Services (WMS),
Web Feature Services (WFS)
Web Coverage Services (WCS)
Catalog Services - Web Profile (CSW)
• OPeNDAP data access protocols
• W3C and OGC web geo-processing protocols
– WSDL, BPEL, UDDI, SOAP.
– The OGC web geoprocessing specifications.
CSISS
Center for Spatial Information Science and Systems
Page 5
Data Discovery and Retrieval Services (DDRS) Component
• DDRS is one of the two subcomponents in the framework that
connect to the EO sensor and data world.
• The major functions of DDRS are to discover and obtain data.
• It uses open standards for interfacing with data sources,
allowing a web sensor or an EO data system equipped with
standard interfaces to be easily plugged into the framework.
– For data discovery, OGC CSW will be used for all data sources.
– For data access to traditional sensor data systems, the OGC WCS,
OGC WFS, and OPeNDAP protocols will be used.
– For connection to web-enabled sensors, OGC SWE protocols (e.g.,
SOS, SensorML) will be used.
– The interfaces between DDRS and PIAS will be CSW for catalog
search and WCS and WFS for data communication.
• DDRS hides the complexity of the EO world behind its standard
interfaces. To EO systems, DDRS is their client. But to PIAS,
DDRS is the single point of entry for all required data from the
EO world.
CSISS
Center for Spatial Information Science and Systems
Page 6
Data Preprocessing, Integration, and Assimilation Services
(PIAS) Component
• ESMs need high-level geophysical products in a modelspecific form as input. Such products are normally not
available directly from sensor measurements, but are
derived from multi-sensor measurements and assimilated
with data from other sources.
• The major function of PIAS is to generate ESM-specific
products from data obtained through DDRS.
• In order for the SEPS framework to support easy plugand-play of ESMs, a general approach for product
generation is needed. We use the customizable virtual
geospatial product approach.
• ESMs will use products generated by PIAS in two ways: 1)
as the boundary conditions, and 2) as run-time ground
truth.
CSISS
Center for Spatial Information Science and Systems
Page 7
Science Goal Monitoring Services (SGMS) Component
•
•
•
•
•
SGMS takes ESM outputs and in many cases the geophysical products
generated by PIAS as inputs to analyze against science goals for
determining whether additional or refined geophysical products are
needed by the model to meet these goals.
Use of geophysical products in SGMS rather than sensor
measurements hides the complexity of the EO world from science-goal
analysis and enables dynamic plug-in and removal of sensors without
affecting SGMS.
Like PIAS, SGMS needs a general method to evaluate whether diverse
science goals are met. We will use the same approach in SGMS as that
proposed for PIAS.
Objective metrics such as information content and system uncertainty
will be used to ensure that SGMS can provide accurate description of
the additional or refined geospatial products needed.
The interfaces between SGMS and PIAS to be WCS and WFS. SGMS
will take ESM outputs in either netCDF or the ESMF state through
WCS, OPeNDAP, or FTP.
CSISS
Center for Spatial Information Science and Systems
Page 8
Data and Sensor Planning Services (DSPS) Component
•
•
The major function of DSPS is to translate the requirements for
geophysical products to EO data requirements and to schedule the
acquisition of such data.
The translation service uses geospatial processing models to trace
back and identify all raw datasets required for generating the products.
– From the specifications on geophysical products (e.g., time, spatial location,
accuracy), the service will determine the specifications for raw data.
– Then DSPS will find the best source for such data and schedule data
acquisition through working with CSW portal in DDRS.
– If the source is a data system, DSPS will provide information to CENS for
scheduling the ordering and retrieval.
– If the source is a schedulable Web sensor, DSPS will use OGC SPS to
schedule observations and register the observation event in CENS.
•
Because a geophysical product can be traced back to multiple raw
datasets, a request for the refined product may result in scheduling or
discovering simultaneous multiple-sensor observations.
– A coincident search capability is provided in the CSW portal to help DSPS
discover simultaneous observations.
CSISS
Center for Spatial Information Science and Systems
Page 9
Coordination and Event Notification Services (CENS)
Component
•
•
•
CENS works as a rhythm controller for the FF loop.
Since everything in the framework is implemented as a Web service,
the loop can be implemented as a large workflow (FF-loop workflow).
Two mechanisms, dynamic data retrieval, and asynchronous data
notification, enable the dynamic data flow in the FF loop.
– These two mechanisms share the same goal of using dynamic data but
from the ESM and EO standpoints.
– For dynamic data retrieval, a request is initiated from an ESM. CENS then
acts upon the request, searches, prepares, and downloads data, and then
generates and feeds the specific products to the model.
– For asynchronous data notification, a sensor or data event will initiate the
FF loop. A notification service will manage the event subscription and
registry and monitor the state change of the registered events.
– Event-based communication will be established between the server or the
notification service and the client or the ESM.
• The notification service keeps a register of events supported by the sensor web.
• OGC WNS and SAS will be the key service protocols for the asynchronous
mechanism.
CSISS
Center for Spatial Information Science and Systems
Page 10
Connection to AutoChem Assimilation System
CSISS
Center for Spatial Information Science and Systems
Page 11
SEPS AutoChem Use Case
The overall scientific goal is to enable the accurate and near-real-time prediction
of the transport of atmospheric pollutants through close interactions between
sensors and the model. Technically, this demonstration demonstrates the
capability of SEPS to dynamically feed data and serve result of the ESMF-based
AutoChem Atmospheric Chemistry Composition Modeling by following open
geospatial standards and specifications for Sensor Web; and this also
demonstrates the sensor planning capability through the
feedback loop of SEPS.
CSISS
Center for Spatial Information Science and Systems
Page 12
Data Sources for AutoChem
MLS
OMI
Aura
Aura
SAGE II
TES
AIRS
Aqua
SEPS Data
feed service
csiss
HIRDLS
Aura
OGC Standard Interfaces: WPS
13
Chem DB
CSISS
Center for Spatial Information Science and Systems
Page 13
Satellite observations for AutoChem
•
This table lists some of the sensors from NASA and
other satellites that collect data useful for atmospheric
chemistry composition monitoring and modeling.
Those data are available through NASA EOSDIS and
the other online repositories. In addition, climate,
weather observation and forecasting data from
THREDDS and NOMADS are also useful.
AURA
SAGE II
CLAES
Sensors
Observations
MLS Aura and
MLS
UARS
Atmospheric chemical species: OH, HO2, H2O, O3, HCl, ClO, HOCl,
BrO, HNO3, N2O, CO, HCN, CH3CN, SO2
HALOE
Ozone (O3), hydrochloric acid (HCl), hydrofluoric acid (HF), methane
(CH4), water vapor (H2O), nitric oxide (NO), nitrogen dioxide
(NO2), sulfate aerosols.
POAM
Ozone (O3), water vapor (H2O), nitrogen dioxide (NO2), aerosol
SAGE II
observation
CLAES
Ozone (O3), methane (CH4), water vapor (H2O), nitrogen oxides,
CFCs and other important chemicals, in the stratosphere
ACE and
ATMO
S
30 atmospheric constituents including O3, N2O, CH4, H2O, the entire
NOy family, and much of the chlorine and fluorine families,
including HCl, HF, and ClONO2
CRISTA
Ozone (O3), nitric acid (HNO3), dinitrogenpentoxide (N2O5),
chlorinenitrate (ClONO2), nitrous oxide (N2O), methane (CH4),
CFC-11, and water vapor (H2O),
ILAS
Ozone (O3), nitric acid (HNO3), nitrogen dioxide (NO2), nitrous oxide
(N2O), methane (CH4) and water vapor (H2O), profiles of
atmospheric temperature and pressure, as well as profiles of
aerosols and polar stratospheric clouds (PSCs)
Sondes
Numerous ozonesondes from worldwide sites.
NDSC data
LIDAR and microwave data from the network for detection of
stratospheric change for Ozone (O3), hydrochloric acid (HCl),
water vapor (H2O), nitric acid (HNO3), and carbon monoxide
(CO).
Aircraft data
All NASA aircraft campaigns from 1988 up to the present.
CSISS
Center for Spatial Information Science and Systems
Page 14
Automatic AutoChem Archival Data Feed
Service based on reusable workflow
• The on-line observations data are automatically fed into
AutoChem using a reusable workflow
– chain DDRS-CSW, DDRS-WCS and PIAS-WPS
subcomponents of SEPS
• The workflow, executed by BPELPower, performs the following
actions:
– order observations data from external archived data sources
(such as NASA ECHO)
– generate coverage description of image data
– subset different level data in HDF-EOS format
– convert different constituent observations into ChemDB.
• The SEPS service components have be used as is to structure
the workflow
– No special adaptation efforts are needed for supporting
AutoChem.
CSISS
Center for Spatial Information Science and Systems
Page 15
Creation of constituent data feeding workflow
• To build a workflow that serves data with standard
interfaces, the following steps need to be followed
– Step1: Query data from NASA GSFCS4PA using DDRS-CSW
– Step2: Generate coverage description using PIAS (Generator
WPS)
• http://csiss.gmu.edu/sensorweb/autochem/datafeed/generato
r.htm
– Step3: Subset different level data in HDF-EOS format using
DDRS-WCS
– Step4: Convert different constituent observation into ChemDB:
• http://csiss.gmu.edu/sensorweb/autochem/datafeed/converte
r.htm
CSISS
Center for Spatial Information Science and Systems
Page 16
Constituent data feeding workflow
(example – Aura OMI) (1)
Input data
from Aura OMI
in HDF file
format
CSISS
Center for Spatial Information Science and Systems
Page 17
Constituent data feeding workflow
(example – Aura OMI) (2)
• Output of the data
feeding workflow
– Table in the format
and arrangement
managed in
chemDB
– SQL-based
constituent
database
– Directly useable by
the AutoChem
Output is table in chemDB
CSISS
Center for Spatial Information Science and Systems
Page 18
AutoChem Model: opaque chain of modules
• AutoChem assimilation model
– Initialization of the parameters used to determine
the rate coefficients at the start of the code
– Temperature and pressure are used to calculate
the bi- and tri-molecular rate coefficients
– Local solar time, day of the year and latitude are
used to calculate the solar zenith angle and then
the photolysis rate coefficients
– Aerosol surface area is used to calculate the
heterogeneous rate coefficients
CSISS
Center for Spatial Information Science and Systems
Page 19
SEPS and AutoChem Model
• SEPS
– Service oriented architecture
– Web service – standardized software in the Web environment to
fulfill a well-defined set of actions
– Major re-useable and adaptable components
•
•
•
•
•
Two types of observation data service flows
Five components
Six OGC Web service interfaces
Seven external components
Nine information models.
• AutoChemSEPS
– Observations data  AutoChem model
– WCS/WFS data  AutoChem ChemDB constituent databases
– SEPS feedback segment using dynamical data retrieval mechanism
of CENS
•
•
•
•
Coordinate all the requests, searches, preparations, and downloads
Adapt data to be fit for the AutoChem model
Future – feeding will be bridged through ESMF
Future – when the AutoChem model in OOODS (Objectively Optimized
Observations Direction System) is in place, the asynchronous data
notification mechanism of CENS can be used to implement a sensor or
data event initiation of the FF loop
CSISS
Center for Spatial Information Science and Systems
Page 20
Connections of SEPS to AutoChem Model
• Data feed mechanism
– Re-use service
components
– All are completed ondemand, or virtual
• Future
– OOODS
• Simulate the directed
access assets
• Prioritize observations
constituents using target
location and priority
metric objects
– Complete FF-loop
• Feed forward
• Feed back
CSISS
Center for Spatial Information Science and Systems
Page 21
Atmospheric chemistry observation data service flow
• Two types of predefined BPEL flows are used in the prototype
– On-demand observations data flow based on CENS
– Real-time observations flow based on CENS
• Components for the on-demand observations data flow
– Coordinate and Event Notification Service (CENS)
– Chain of DDRS-CSW, DDRS-WCS and PIAS-WPS subcomponents
of SEPS
• To order observations data from external archived data sources (such
as NASA ECHO, GeoBrain CSW, NOAA CLASS and ESA EOLI
catalogue), subset different level data in HDF or HDF-EOS5 format,
translate different constituent observations data into ChemDB, and
transfer the model initial state to AutoChem model
CSISS
Center for Spatial Information Science and Systems
Page 22
Dynamic sensor data feed to ChemDB
• On-demand
observations data
flow
– Use existing SEPS
resources to
automatically form
the constituent
database
• Real-time
observations flow
(to-be developed)
– Use the real-time
sensor observations
by planning
CSISS
Center for Spatial Information Science and Systems
Page 23
Sensor Web Fire Detection Workflow
CSISS
Center for Spatial Information Science and Systems
Page 24
Fire Location Detection (FLD) Workflow
Workflow Resource
Design Deploy Execute
OfferingID
EO1SOS
GMU
WCS
Band110
Subseting
Band150
Subseting
Band210
Subseting
Band213
Subseting
JPL
WPS
Product
ThermalHyperion
L1G Process
GMU BPELPower Workflow Engine
CSISS
Center for Spatial Information Science and Systems
Page 25
CSISS
Center for Spatial Information Science and Systems
Page 26
CSISS
Center for Spatial Information Science and Systems
Page 27
CSISS
Center for Spatial Information Science and Systems
Page 28
Workflow example for fire detection（Hot Pixel）
1) Design an
abstract GeoProcessing Model
for fire location
detection
4) Fire Location
Chart
B110
B150
B210
B213
2) Retrieve four bands Hyperion data
using EO-1 SOS “GetObservation” and
GMU WCS “GetCoverage” operation
3) Using the BPEL Engine to Chain
EO-1 SOS, GMU WCS and JPL WPS
to generate fire location chart
CSISS
Center for Spatial Information Science and Systems
Page 29