Transcript Marine Data Representation systems T. Loubrieu, G. Maudire, S. Pouliquen, T.

Marine Data Representation systems
T. Loubrieu, G. Maudire, S. Pouliquen, T. Carval, IFREMER,
E. Charpentier, WMO
Scott D. Woodruff, NOAA
on behalf of JCOMM
Background
Involved in :
• European Operational Oceanography
(Mersea/MyOcean, ECOOP) information
management system design.
•
CORIOLIS operational data management system
ocean in-situ observation.
•
SeaDataNet, marine data management paneuropean framework.
Data Representation systems
•
•
•
Marine Observation networks (ARGO, OceanSITES,
GOSUD, VOS, GLOSS, GTSPP, Drifting buoys,
GRHSST…)
Integrated systems for marine monitoring and forecasting
(Mersea/MyOcean, ECOOP, …)
Overall framework initiatives for marine data management
(SeadataNet, UNIDATA/CDM, CSML).
Current Status regarding GTS
•
•
Most of the Marine data are being exchanged in real-time
through the GTS using the recommended formats (SHIP,
BUOY, BATHY, TESAC, TRACKOB, WAVEOB, BUFR and
CREX)
Migration to table driven codes has been engaged
especially with regards to buoy, Argo, XBT (BUFR) and sea
level data (CREX). Works remains for the VOS (BUFR).
ARGO overview (1/2)
 More than 3000 profiling floats
 Measure temperature, salinity, dissolved oxygen.
 Each delivers every 10 days one vertical profile
from 2000 meters depth to surface
 6 regional data assembly centers + 2 global data
centers
 Harmonized quality assesement procedures, near
real time and delayed mode delivery.
 Based on ftp repositories and harmonized netCDF
files for data and metadata.
ARGO overview (2/2)
GOSUD, OceanSITE are
extensions of the DRS
to thermosalinographs
and moorings datasets.
Drifting buoys data are
also managed at ISDM
and CORIOLIS data
center.
Voluntary Observation Ships
overview
 Observation from volontary ships
 Transmitted in real time (F-13 SHIP format).
 Transmitted in delayed mode (IMMT file format) to
Global Collecting Centers and archived at
ICOADS (IMMA file format).
 Data analysis, climate studies provided in
netCDF/CF files.
Global Temperature and Salinity
Profile Program (GTSPP)
 Provide a timely and complete data and
information base of ocean temperature and salinity
profile data of known and documented quality.
Global Sea Level Observing System
(GLOSS)
 A 'Global Core Network' (GCN) of 290 sea level
stations around the world for long term climate
change and oceanographic sea level monitoring.
 A permanent service managing delayed mode
data (PSMSL) : ASCII files.
 A station directory and link to regional data
management facility (non-harmonized).
 A fast-mode delivery service (OPeNDAP and
ASCII files).
GHRSST overview
 Remote sensing observations of sea surface
temperature (SST)
 Harmonized quality assesment and file
dissemination (L2 products) in real time through
data centers (ftp/http).
 Analysis, gridded products mixing datasets from
different satellites (L4).
 Central facilities for indexing the datasets (MMR)
and archiving and reanalysing them (LTSRF).
Mersea/MyOcean overview
 Monitors and forecasts the global and regional
ocean. Provides a service for application providers
(oil spill monitoring, ship routing, …) and policy
makers.
 Involves 5 modeling and forecasting centers and 3
thematic assembly centers.
 Information flows are federated through standards
back-end services and central facilities (catalogue,
viewing portal, system monitoring)
SeaDataNet overview
 An network infrastructure for managing the large
and diverse data sets collected by the
oceanographic fleets and the automatic
observation systems.
 Sets up reference services (vocabularies,
catalogues, users directory…) before setting up a
federating infrastructure for European marine
observation data sharing.
Suitability criteria
 Content
 Pictorial display
 Operation activity loging (warning, …)
 Operational support (real time, …)
 Transmission of information to users
 Storage of information within systems
 Compliance with standards
 Metadata availability
 Translation of data representation
Content
 For observation networks : datasets are gathered in each
DRS according to :
– Observing platform/sensor type (profiling floats, moorings, tide
gauges… remote sensing/SST).
– Delivery requirements (quality assesment / delay).
(SeaDataNet focuses on infrastructure, historically deals with
marine in-situ delayed mode data)
 For integrated systems : datasets are gathered in one DRS
according to :
– One service requirement (e.g. monitoring and forecasting
ocean)
– The processing chain is integrated within the DRS (from
observation sub-DRS to modelling and forecasting sub-DRS).
Pictorial display
 Most of time specific GIS web portal or preprocessed quicklooks with adavanced web
interfaces.
 More and more GIS web portal based on
OGC/WMS back-end services and generic web
portal interface (openlayer, mapbuilder).
Operation activity log
 Up to now, very few standardized exchanges for operational
activity and results, for example :
– Grey list for profiling floats status (ARGO).
– Textual information on web site (e.g. GTSPP, pressure warning)
– Automatic mail report for activity (end of process, status of
process) within a production unit (at CORIOLIS).
 Standardized monitoring engines, on-going development :
– Automatic check of data availability (according to SLA
requirements) : NAGIOS tools verify in real time OPeNDAP
services for Mersea/MyOcean.
– Later on, standardized report for activity of MyOcean production
units to Service Desk (e-mail or RSS).
Information dissemination (1/2)
 Usually functions divided into :
– Discovery
– Viewing
– Download
 Discovery disseminates products catalogues or
dataset inventories with ISO19115 and THREDDS
catalogues. CSW is likely to be used for
requesting these catalogues (to be done).
Information dissemination (2/2)
 Viewing :
– pre-processed quicklook can be browsed in specific web interfaces
– OGC/WMS is being more widely used for map quicklooks.
– No standard way of dealing with non-map view (vertical profiles, time
series, vertical cross section, …)
 Download :
– Most of time, ftp server with netCDF/CF files with specific extraconvention for file format (ARGO, OceanSite, …, Mersea/MyOcean)
– ODV ASCII file format for SeaDataNet
– MEDS-ASCII or various ASCII flavors for time series, profiles, …
– IMMT/IMMA binary file format for VOS (ICOADS)
– OPeNDAP interfaces for dynamic subsetting (using THREDDS Data
Server) : Mersea/MyOcean.
– OPeNDAP/Dapper interface for profile data at CORIOLIS (DAP4COR).
Storage facility
 Under responsability of production unit (with
Service Level Agreement to be defined so that a
subset of archive can be restored on user
demand).
 Or under responsability of NODC (National Ocean
Data Centers)
 For GHRSST, GLOSS, ARGO, …, a central
archive and reanalysis facility.
Standard compliance
 Metadata : XML/ISO19115 widely used with different profiles depending on
requirements (SeaDataNet, Mersea/MyOcean). THREDDS used for simple
metadata index (Mersea/MyOcean).
 Images : OGC/WMS (not extended for non-map representation)
 Download :
– netCDF file format with CF convention.
– OPeNDAP interfaces (supported and checked by information
management team in Mersea)
– OGC/WFS and OGC/WCS emerging (not used yet).
(requires further standardization/convention for non gridded datasets)
 References, background (to be used) :
– vocularies provided by SeaDataNet
– guidelines for usage of metadata standard for marine datasets.
– CDM / CSML provide federated data model for ocean/atmosphere
dataset.
Metadata availability
Harmonized metadata available within each project :
 By products (SeaDataNet/EDMED, Mersea
catalogue)
 By observing platform, numerical model
(SeaDataNet/EDIOS, ARGO/Metadata files, …)
 By dataset (SeaDataNet/CDI, Mersea inventory)
Translation of data representation
To be done :
 Rely on standardized data model for
ocean/atmosphere datasets (CDM or CSML) : to be
set up by each production unit.
 Rely on external authorities to translate references :
– Parameters names with vocabulary services.
– Geographical services for changing map projections or
geospatial reference datum.
Practical experiment : SeaDataNet
 Vocabularies web services :
– ontology available on-line. Parameters names,
categories…
– similar to GCMD vocabularies + deprecated flag instead
of complete removal of terms.
– Identification of terms with URN (within ‘sdn’ namespace)
 XML/ISO19115 extension :
– Usage of URN within the XML metadata description for
embedded controlled vocabularies.
Practical experiment : download
from Mersea
 Download integrated
service architecture :
Practical experiment : download
from Mersea (2/2)
Real time monitoring of the distributed system for improving the
reliability (status : ok, late delivery, down) : from product and
service catalogue, configuration of NAGIOS monitoring tool.
JCOMM starting or on-going
activities related to Marine DRS
 JCOMM and IODE have engaged in the « Standard
process » (ontologies, temporal and geospatial references,
platforms, quality control…). One meeting has been held in
Ostend, Jan. 2008.
 JCOMM Pilot project for WIGOS (2008-2010) :
– The Pilot Project will aim at making the appropriate identified data sets
interoperable with the wider WMO and IOC communities. It will develop
and agree on consistent standards to be used across the community (at
metadata and data level).
– Guidelines are expected from ET-ADRS.