Transcript Introduction to SensorML

Q2O – QARTOD to OGC
http://q2o.whoi.edu
Integrating QA/QC tests, flags and standards
into Sensor Web Enablement
Janet Fredericks
Woods Hole Oceanographic Institution
Martha’s Vineyard Coastal Observatory
August 26-28, 2008
NSSTC/UAH - Huntsville AL
This morning …
• Introduction to Q2O project: who, what,
why, when
• Present example implementation from
MVCO ADCP waves and CDIP data buoy
waves
• Discussion of QA/QC implementation
• And perhaps we can add the where!?!
OOSTETHYS (MVCO in Fall 2006)
Why OGC?
Sensor Observation Service
• “Provides an API for managing deployed
sensors and retrieving sensor data”
• Key here is access through a service
• Three mandatory “core” operations
GetObservation
Describe Sensor
GetCapabilities
SensorML
What is it?
SensorML provides standard models and an XML
encoding for describing any process,
including the process of measurement by sensors
and instructions for deriving higher-level
information from observations.
Processes described in SensorML are discoverable
and executable.
All processes define their inputs, outputs,
parameters, and method, as well as provide
relevant metadata.
SensorML models detectors and sensors as processes
that convert real phenomena to data.
MBotts
QARTOD is a multi-organizational effort to address the Quality
Assurance and Quality Control issues of the Integrated Ocean
Observing System (IOOS).
Motivation
• Expected increase in the number of observations, observing
systems, and users
• Need for simple, accurate and consistent quality content and
descriptions for the real-time data to convey expected level of
data quality to users
Methodology: Engaging the community to develop standards.
Courtesy Julie Bosch
Activity initiated by NOAA NDBC and CO-OPS in 2003
Series of Workshops
• Workshop format with plenary and breakout sessions
• Approach to tackling the QA, QC, and metadata issues evolved
• From separate breakout sessions for QA, QC, and metadata for
all observations combined
• To breakout sessions by observation focus group (covering
QA, QC, and metadata)
• Each observation focus group addresses the same guidance
questions
• Participation
• Approximately 80 participants per workshop
• Primarily observing system operators and data managers
• Representation from federal agencies, oceanographic institutions,
universities, and instrument manufacturers
Julie Bosch
The Workshops
QARTOD I: December 3-5, 2003 NDBC, Stennis Space Center, MS
• Task to develop minimum standards for QA/QC methods and metadata
• Considering existing observing systems and existing practices (presentations)
QARTOD II: February 28-March 2, 2005 CO-OPS, Norfolk, VA
• Focus on wave, in situ current and remote current (HF Radar) measurements
• Identify unique calibration, metadata and QA/QC needs
QARTOD III: November 2-4, 2005 SIO, La Jolla, CA
• Continued waves, in situ currents and remote currents work
• Added CTD focus group
• Primarily addressed QC with some metadata emphasis
QARTOD IV: June 21-23, 2006 WHOI, Woods Hole, MA
• Continued wave, in situ current and CTD work
• Added DO focus group
• Primarily addressed QA; increased emphasis on capturing metadata requirements
• International considerations
Courtesy Julie Bosch
Outcomes
Waves
• Results being incorporated into National Waves Program data
management plans
• Results compiled and submitted to the IOOS DMAC Standards
Process
In situ currents
• Results specific to ADCPs compiled and submitted to the IOOS
DMAC Standards Process
Remote currents
• HF Radar community continued independent of QARTOD
• Developed Data Management Standards for HF Radar
• Submitted to the IOOS DMAC Standards Process
New initiative
• Implementing QA/QC from QARTOD for in situ ocean sensors
using OGC Standards/Sensor Web Enablement
Courtesy Julie Bosch
more information…
http://www.qartod.org
NOAA-CSC RCOOS awarded Jan 2008
Integrate QARTOD QA/QC recommendations
into OGC Sensor Web Enablement:
•
develop data dictionaries;
set them up in SensorML profiles;
• demonstrate products by integration into
oostethys_sos and update or complement
cookbooks with QA/QC implementation
• Document and integrate results by providing
results to community building organizations,
such as MMI, ACT and QARTOD
Principal Investigators
Janet Fredericks, Woods Hole Oceanographic Institution (WHOI);
Julie Bosch, NOAA National Coastal Data Development Center/DMAC;
Michael Botts, author of SensorML, along with his OGC development
team from the University of Alabama at Huntsville (UAH);
Philip Bogden, the Gulf of Maine Ocean Observing System
(GoMOOS)/OOSTethys/OGC Oceans IE; and
Sara Haines, University of North Carolina/ Chapel Hill and SEACOOS
http://q2o.whoi.edu Funded 3 yrs
Semantic Interoperability
• Domain experts define best practices and
required vocabularies – working with IT
experts to keep interoperability goal in mind and
guide in developing relationships for RDF
definitions (http://qartod.org)
• Build buy-in within and across communities to
promote interdisciplinary potential (E.g.,
http://marinemetadata.org)
For each domain (e.g.., waves/currents)
• Meet with domain experts to develop
vocabularies, definitions and relationships
(develop data model/SensorML profiles)
• Present data model and profiles to broader
community
(develop tools and guides)
• Hold workshop to demonstrate
implementation and introduce/test
products
Q2O Schedule
• 2008 – complete waves implementation
and demonstration
• 2009 – complete in situ currents
implementation begin CTD/DO work
• 2010 – complete CTD/DO work and (if
funded) demonstrate in situ currents and
CTD/DO
Past Q2O Workshops
• February 2008 – Introduction of QARTOD
leads for each domain to the Q2O project
and the SWE development team to
QARTOD
• June 2008 – Met with waves and in situ
currents domain experts to discuss SWE
and the concepts; discussed data
dictionaries, use cases and reviewed
QARTOD recommendations
Tasks for This Workshop
• Review the data model for QARTOD
recommended tests and the preliminary
demonstration of SensorML profiles for
waves (MVCO TRDI-ADCP and CDIP
Buoy examples)
Or Space time toolkit, or NWS or science
user (Matlab) or NCDDC ….
SWESOS?
QC-flags
Manufacturer’s
info & processing
Serial Number
Model Number
Sensor
capabilities
Processing
history
Test
parameters
specific to
sensor
SWESOS
MVCO
Data Provider
info & processing
Set up and
Deployment
descriptions
Processing
history
Test
parameters
specific to
deployment
NDBC
QC-flags
SWESOS
QC-flags
NDBC
processing
Processing to meet the
requirements of
IOOS/NDBC etc
Testing not available to
a local node (nearest
neighbor, regional range
checks)
What do we have (know) to
start with?
What information can we
provide to data users or
systems (OOSTethys)?
How and with What do
we convey that
information?
- A sensor (wave buoy or
ADCP) with certain
characteristics
- A sensor history
- QA info associated with a
sensor
- Deployment
characteristics
- Methods to process the
data
- QC Tests to apply to the
data
- …
- What sensors we have
available as a service
Get Capabilities
Shown as snapshots of
information on multiple web
Pages, docs, etc
- Description of the sensor
- Description of where / how
/ when it is deployed
- List of the processing
methods used on the data
- List of the QC tests applied
- The criteria used in the QC
tests
- The results of the QC tests
- The data
- …
Show the data model (tests,
criteria), dictionary, and where all the
pieces and parts are located
- lists available data
(properties)
- uses SOS,
Observation Offering
- Describe Sensor
-Provides sensore
characteristics,
deployment
characteristics and
processing methods
-Uses SensorML
- Get Observation
-Provides the data
-Provides test results
-Points to file with
processing/test info
-Uses O&M
Show info conveyed
in the SWE xml files
Julie Bosch