Transcript Coriolis data center

EMODnet Chemistry 2
Service Contract MARE/2012/10 S……………
By
Dick M.A. Schaap – Technical Coordinator
Trieste – Italy, 3 – 5 June 2013, Kick Off Meeting
Requirements for the portal
gather all measurements of a particular chemical species with their
appropriate metadata within a given space and time window
include the physical conditions under which the measurements were
made (from EMODnet physical parameters portal or the GMES marine
core service)
visualise the measurement density in a given time and space window
visualise a time evolution of a selected group of measurements
show concentration plots for a given time and space window and also
along the coast
show inflows from rivers of nutrients. The user should be able to
select a section of coast, a country or a region (NUTS3) and obtain
time series of inflows of parameters expressed as mass or moles per
unit time per river (or section of coast)
calculate spatially distributed data products specifically relevant for
selected Marine Strategy Framework Directive Descriptors
Present portal
the present EMODNet Chemistry portal sits on top of the SeaDataNet infrastructure and
makes use of its services that have been adapted for specific EMODNet Chemistry
needs.
This is continued and expanded for the new project with mutual benefits for EMODnet
and SeaDataNet:
easy expansion of the geographical coverage of the EMODnet chemical data sets
to full European coverage via SeaDataNet nodes and potential new data centres;
merging with other types of marine data sets, because SeaDataNet already
handles a large volume of multi-disciplinary data sets, also through other
initiatives;
Perspective of long term sustainability of the infrastructure, because the
infrastructure and network are based upon the NODC’s and via crossfertilisation;
high quality and long expertise with handling marine data;
INSPIRE compliance;
optimal use of existing infrastructure and ongoing SeaDataNet II developments,
such as machine-to-machine services and visualisation services
efficiency and cost effectiveness by combining efforts and developments, not only
technically, but also organisation-wise;
Other data providers will be more willing to contribute data sets because of
SeaDataNet status and perspective
Present EMODNet Chemistry portal
It provides various services and functionalities to users for browsing
and viewing the chemistry data products and for identifying and
requesting access to the gathered chemistry data sets for the
European waters.
The primary services are:
CDI Data Discovery and Access Service giving facilities for
searching and retrieving chemistry data sets;
OceanBrowser Viewing Service giving facilities for viewing,
browsing and downloading Chemistry data products;
Sextant Products metadata catalogue giving facilities for
searching Chemistry data products and linking to the viewing
service.
CDI Data Discovery and Access Service
ISO19115 content standard and XML Schema migrating soon to
INSPIRE compliant ISO19139
Dedicated tools and facilities for generating CDI entries, format
associated data sets and populating the CDI service
Shopping basket mechanism for discovery, access request, and
downloading of data sets from distributed data centres (at present 85
data centres from 31 countries connected)
Downloading in harmonised
SeaDataNet formats:
SDN ODV ASCII, and
SDN NetCDF (CF) underway
Adopted in many projects
and ongoing improvements
Operational governance scheme
CDI Chemistry Data Discovery and Access Service
At present 3 search interfaces for human users:
Quick Search with dynamic drilling down of search results;
Extended Search with more flexibility for combining search
options, including free search;
Variables Vs Marine Regions with an interactive Matrix of variables
in specific marine regions.
OceanBrowser Viewing Service
This gives access to integrated maps of selected parameters.
Developed and operated by GHER group of University of Liège (ULg).
For EMODNet Chemistry a dedicated implementation.
The viewer provides access to the two kind of available products:
DIVA interpolated maps. Output images available as horizontal
sections and vertical sections. The latter can be selected by
drawing an appropriate transect; (via OpenDAP, NetCDF, ncWMS)
Time series plots.
Sextant Products metadata catalogue
used to describe the Chemistry data products. This facilitates
searching for specific data products and the exchange and use of the
Chemistry data products in other services, such as the Chemistry
OceanBrowser, and other portals with OGC WMS support.
Metadata format: ISO19115 - ISO19139 with SDN Controlled Vocabs;
CSW service based upon GeoNetWork
EMODnet Chemistry portal – present set-up
Portal features and planned extensions
Already many features for providing a gateway to data, metadata and
data products
Users can retrieve, view and download Chemistry data products
as digital maps and transects.
Users can query and identify the availability of Chemistry data
sets and view maps to understand the density of measurements
and the data gaps. Users can request copies of data sets from its
data providers.
Planned additional functionalities:
signing in: only for requesting data using CDI service; uses CAS;
work ongoing for supporting also OpenID and Shibboleth
Search: many options already provided. Plans to extend the CDI
search facility with sea regions and possibly user drawn
geographical polygon and larger shopping basket (10.000)
+ to integrate the Sextant catalogue in the OceanBrowser menu
Planned additional functionalities
Viewing:
CDI provides location viewing + WMS support: plans to enable
user created WMS layers;
OceanBrowser provides viewing of layers in the water column and
transsects: plans for OGC WFS services on top of the map WMS
service for selected stations.
it is planned to include visualisation of data by OpenEarth
techniques.
Planned additional functionalities
Coastal data:
CDI service allows users to search by a lat lon box: it is planned
to explore an extra spatial query functionality for searching based
on the distance along the coastline. Also an external WMS layer
with geographical features (e.g. towns, borders, river outlets) will
be added.
OceanBrowser service: planned to allow the visualization along
(or near) the coastline of a Chemistry data product. One axis of
such visualizations will correspond to the distance along the
coastline from a given starting point. Currently vertical sections
can be extracted along a user-specified path. Data extraction
along the coastline will be simplified for providing a list of
predetermined sections following the coastline. The display of the
along-coast figures will be optimized to take into account the
specificities of these.
Planned additional functionalities
Machine–to–Machine interfacing for CDI – metadata level:
Already supports OGC WMS – WFS for CDI metadata
Already supports CSW to interact with GEOSS and ODP portals
Following GENESI-DEC developments it is planned to add the
OpenSearch protocol to facilitate remote queries from another
portal; OpenSearch is an interesting protocol supported by major
portals such as Google, Yahoo, Twitter. OpenSearch request
interface is simple, consisting of a description of a HTTP GET
request with a series of optional key-value parameters that can be
used to constrain the search:
Free search
Geospatial (area or point + radius)
Temporal (from to)
Planned additional functionalities
Machine – to – Machine interfacing for CDI service – data
level:
Rationale: The CDI system provides a strong foundation for the
further operation and expansion of the SeaDataNet infrastructure and
the EMODnet Chemistry portal with now 85 data centres connected,
> 1.3 million data sets, interoperability with other major portals and
expanding support for all kinds of data types and disciplines.
However the present system is also limited in functionality and should
be extended with functionalities that can provide additional support
for users and user communities:
the present CDI system is mostly user driven with user interfaces
for discovery, shopping and downloading of data sets
the present system provides limited options for data viewing and
visualisation and is mostly focused on downloading of distributed
data sets; the viewing is limited to maps and WMS -WFS services
of data locations and related CDI metadata.
Planned additional functionalities
Machine – to – Machine interfacing for CDI service – data
level:
Therefore SeaDataNet II is making progress with a.o.:
Developing specific aggregate data sets as internal data buffers
that are centrally maintained with automatic harvesting using a
robot user in the CDI system
Developing advanced access and viewing services on top of these
aggregate data buffers for providing sub-setting, advanced
options for graphics and analytics
Robot harvester is already operational and has been tested for
automatically harvesting ca 800.000 T&S data sets as part of a joint
data climatology product development SeaDataNet – MyOcean
Robot harvester can be configured for specific data buffer profiles and
in agreement (SLA’s) with data providers AND possibly specific data
user communities (such as MyOcean, WG DIKE)
Planned SeaDataNet architecture extension of
relevance for EMODnet Chemistry
data centres at the basis managing a large collection and variety of
data sets and connected to the CDI system
CDI system at the portal for users to discover and request access to
these data sets
CDI system at the portal with machine interoperability to other
portals with an increasing service level from metadata exchange to
full remote data shopping and delivery
central data buffers (possibly hosted in the cloud) maintained by
robot shopping and data harvesting using the CDI system; these
buffers will concern aggregate data sets for specific parameters that
have been agreed with SeaDataNet data providers beforehand
Planned architecture extension
Oceanotron
OpenEarth
DIVA
ODV
CDI User CDI Robot
Interfaces harvester
Agreed
Settings
Advanced access
and
viewing services
Dynamic
Maintenance
Specific data
buffers
Possible EMODnet Chemistry extension
DIVA +
OceanBrowser service
Oceanotron &
OpenEarth services
QCd data
buffers
Regional
experts
CDI Robot
Harvester
ODV QC + aggregation
Specific data
buffers
Planned architecture extension
the access to these buffers of aggregate data sets can be virtual in
which case only results and graphics of analyses are made available
or as downloadable files / sets of files
the access to these buffers of aggregate data sets can be for public
use or only for specific user communities which can be arranged by
Service Level Agreements agreed between SeaDataNet data centres
and specific user communities such as e.g. MyOcean, WISE-MARINE
specific access and visualisation services on top of these buffers of
aggregate data sets for providing advanced query, visualisation and
analysis functions
Needs further analysis for loops with QC, duplicates etc – interfacing
to ODV, DIVA, OpenEarth, Oceanotron – buffer hosting (cloud?)
www.emodnet-chemistry.eu