Transcript Coriolis data center

NODCi - National Infrastructure for access
to Oceanographic and Marine Data and
Information
(RGI-014)
By
Dick M.A. Schaap – MARIS
Amsterdam, April 09
NODCi project summary
implementation of National infrastructure for access to oceanographic and
marine data and information
must be interoperable with international networks and developments
phase 1: definition (early 2005 – march 2006)
phase 2: implementation (mid 2006 – march 2009)
partners:
KNMI
TNO - Geological Survey
RWS
NIOO-CEME
NIOZ
Hydrographic Office
Deltares
MARIS (coordinator)
co-funding by Ruimte voor Geo-Informatie (RGI) programme (RGI-014)
International developments
Sea-Search: Pan-European network for oceanographic and marine data
management (2002 – 2005) – focus on metadata
SeaDataNet - Pan-European network for oceanographic and marine data
management (2006 – 2011) – continuation of metadata and access to data
EU - INSPIRE Directive – European Spatial Data Infrastructure
GEO-GEOSS initiative from the G8 ad-hoc Group of Earth Observations for a
Global Earth Observation System of Systems
GMES (Global Monitoring for Environment and Security) from ESA and EU
Research as European component of GEOSS
New EU - Marine Directive with plans for an EMODNET
UNESCO – IOC – IODE – Intergovernmental Oceanographic Commission
ICES – International Council for Exploration of the Seas
NODCi approach
Cooperating and tuning with the projects Sea-Search and its successor
SeaDataNet for standards and technology
NODCi as Netherlands node in the Sea-Search / SeaDataNet
infrastructure
Tuning with INSPIRE developments via European contributions with
SeaDataNet as Spatial Data Interest Community (SDIC) and nationally via
cooperation with RAVI and later GeoNovum
Exchanging progress and tuning with IOC-IODE, ICES, WMO, EGS and
other international groups and fora
NODCi activities
establishing a new NODCi portal connected to data management systems of
individual NODC members
populating the Sea-Search V0 and successive SeaDataNet V1 metadata
directories
establishing harmonised and transparant online access to the data collections,
managed by the NODC members, in line with SeaDataNet
establishing a pilot application service with a numerical model
establishing OGC compliant viewing services for spatial information for the
Netherlands part of the North Sea
preparing an official arrangement with NODC members for long-term
sustainability and operation of the NODCi infrastructure
dissemination, promotion and regular dialogue with potential users from
government, research and industry
More about SeaDataNet
SeaDataNet aims to set up and operate an efficient Pan-European
distributed infrastructure for managing marine and ocean data by
connecting:
40 National Oceanographic Data Centres (NODC’s), national
oceanographic focal points, and ocean satellite data centres, in
Europe
these Data Centres are mostly divisions of major national marine
research institutes and based in 35 countries, surrounding the
European seas
SeaDataNet aims to ensure a comparable quality of data sets and to
make data sets easily accessible on-line through a unique portal,
while the data sets are stored and managed at the Data Centres.
SeaDataNet Participating Countries
49 Partners = 40 Data Centres + ICES + IOC-IODE + JRC +
specialists in statistics, products and informatics
SeaDataNet Infrastructure overall concept
SeaDataNet Infrastructure versions
Version 0 = Continuation and maintenance of existing Sea-Search
systems and population of the Common Data Index (CDI) V0 system
Version 1 = Harmonised and upgraded metadatabases adopting ISO
19115 standard; Transparent data access involving all Data Centres
via implementation of Common Data Index (CDI) V1 system and
downloading services.
Version 2 = Adding OGC viewing services and further virtualisation
of data access.
SeaDataNet Services
SeaDataNet portal (www.seadatanet.org)
Pan-European metadata directories:
EDMED - European Directory of Marine Environmental Data Sets
EDMERP – European Directory of Marine Research Projects
CSR – Cruise Summary Reports
EDIOS – European Directory of Ocean observing Systems
EDMO – European Directory of Marine Organisations
Common Data Index (CDI) for detailed overview and harmonised
access to data sets, managed by SeaDataNet data centres
Generic regional data products, available by viewing services
SeaDataNet Interoperability
Interoperability is the key to distributed data management system
success. This is achieved via:
Standardised metadata formats, based upon ISO 19115,
supported by controlled Vocabularies
Standardised QC methods and Quality Flags, complying with
international standards
Standard software tools and protocols for interconnecting the
Data Centres and the Portal
Portable data management software to equip the Data Centres for
file conversion, analysis, and viewing
Standard formats for delivering data sets to users
Common geostatistics software (DIVA) for computing regional
gridded fields and statistics
Web services as key technology
SeaDataNet Common Vocabularies
Vocabularies User Client Interface
http://seadatanet.maris2.nl/v_bodc_vocab/welcome.aspx
SeaDataNet Data policy
Metadata
free and open access, no registration required
each data centre is obliged to provide the meta-data in
standardized format to populate the catalogue services
Data and products services
the general case is free and open access
however user must register once in order to get a personal login password
user agrees with “SeaDataNet User Licence”
licence is part of the SeaDataNet Data Policy, that is fully
compatible with the Directive of the European Parliament and of
the Council on public access to environmental information, the
INSPIRE Directive, IOC, ICES, WMO, GCOS, GEOSS and CLIVAR
data principles.
SeaDataNet Data Policy is an overarching policy, respecting any local
policy
Access to data is regulated by the User role and the specified Data
Access restriction
SeaDataNet V1 ODV and DIVA presentation and
analysis tools
NODCi activities
establishing a new NODCi portal
populating the Sea-Search V0 and successive SeaDataNet V1 metadata
directories
establishing harmonised and transparant online access to the data collections,
managed by the NODC members, by connecting the data management systems of
individual NODC members to the portal
establishing a pilot application service with a numerical model
establishing OGC compliant viewing services for spatial information for the
Netherlands part of the North Sea
preparing an official arrangement with NODC members for long-term
sustainability and operation of the NODCi infrastructure
dissemination, promotion and regular dialogue with potential users from
government, research and industry
NODCi portal
http://www.nodc.nl
Access to data - Common Data Index (CDI)
CDI is a fine-grained index to individual data measurements (such as
a CTD cast, moored instrument record, sealevel station timeseries)
CDI XML Schema is based upon the ISO 19115 standard
CDI V0 developed in Sea-Search; upgraded to V1 in SeaDataNet
CDI V0 = Central and homogeneous metadatabase, but for Data
Access users are confronted with the different user interfaces of the
NODC Data Centres (which can provide online search and access /
ordering facility / e-mail request)
NODCi CDI V0 – User Interface
NODCi CDI V0 – Data Coverage
CDI V0 coverage: ca. 42.000 CDI entries from 5 NODC
members
NODCi CDI V0 – Data Coverage
CDI V0 coverage: TNO collection = > 10.000 CDI entries for
geological samples and cores
Common Data Index (CDI) V1
CDI V0 XML Schema has been upgraded to V1 XML Schema,
supported by Common Vocabularies, editing/generation tool
(MIKADO) and upgraded XML validation services
CDI V1 has been adopted in NODCi phase 2 to give users not only a
homogeneous metadatabase of available data, but also a unique
interface for getting access to the data sets themselves, that are
managed by the distributed NODC members
This includes connecting the NODCi CDI portal with the data
management systems of the NODC members by adopting tools and
principles, developed in SeaDataNet
The NODCi implementation served also as learning curve and
provided SeaDataNet very useful feedback, which has lead to various
improvements and additional functionality
Delivery Services – CDI V1
Central and homogeneous metadatabase plus transparent access to
the distributed data sets via a unique user interface at the NODCi
portal and downloading services
Data Access components
Common Data Index (CDI) system including a Shopping and
Transaction mechanism
Using harmonised Data Transport Formats for data sets delivery
Authorisation and Authentication Services, based upon the Central
User Register and the Data Access Restrictions as given in the CDI
metadata
CDI V1 system components
CDI V1 Metadatabase and User Interface:
Searching and browsing of metadata of data sets, managed by
Data Centres
Central User Register - Web service:
Details of users, their organizations and addresses, license period,
Id-Passwords, Roles
Shopping Basket – User Interface:
Preparing a user request of multiple data sets, handling validation
of users and roles via the Central User Register, and routing
requests to the Request Status Manager
Request Status Manager – Web service + User Interface:
Processing and administration of all requests and data deliveries
(downloads), for users in communication with data centres. For
data centres to oversee all transactions.
CDI V1 system components
Data Centre Download Manager – Java Tool:
Handles communication with Request Status Manager and takes
care that requested files are made ready for downloading by User
(if OK) in the agreed formats via a local website address at the
Data Centre
NODCi CDI V1 – User Interface
Many options, but difficult for first-time users
NODCi CDI V1 – Quick Search User Interface
Easy, because very intuitive
NODCi CDI V1 – Data Access example
Check Status
In RSM
Search
Include in
Basket
Request
Confirmed
Results
Ready at DC x
Shopping list
Download
Submit + Authentication
Data
SDN
format
Pilot application service
NODCi GIS Viewer, combining North Sea topographic and bathymetric
maps from RWS and TNO via OGC-WMS
Viewer is based upon Minnesota Map Server
Viewer includes layers for oil & gas licences, pipelines and platforms
Viewer expanded with a demonstrator for an oil-spill simulation
model, based upon Deltares ‘Delft3D-Part
User can calculate and visualise oil spills
Oil spill model results are exported in NetCDF format and stored at
Deltares on ncWMS server
NODCi GIS Viewer enables to select model runs and to include the
WMS output as a layer in the GIS Viewer
NODCi GIS Viewer
NODCi Oil Spill entry form
Cross sector involvement
SeaDataNet cooperates and provides services for many other
European and international groups and projects, contributing and
safeguarding good data management. Examples:
EuroGOOS, maintaining the European Directory of Ocean
observing systems (EDIOS) and improving access to real-time
data. See www.edios.org
POGO, collecting and providing information on ocean-going
research vessels, and their operators, planned and completed
cruises. See www.pogo-oceancruises.org
GMES Marine Core Services for streamlining the provision of
long-term archives for optimising marine forecast services
EuroGeoSurveys for improving overview and access to
geological and geophysical data sets See www.eu-seased.net
Partners in the Humboldt project (GMES) contributing to a
European Spatial Data Infrastructure
Services for the international oil & gas industry for metocean data
(SIMORC). See www.simorc.org
Associated infrastructure projects for the Black Sea and Caspian
Sea communities (BlackSeaSCENE and CASPINFO)
EMODNET perspective
New Marine Directive includes the establishment of an overarching
European Marine Observations and Data Network – EMODNET
EMODNET will integrate existing and new to be developed
observation networks and data management systems into a
coherent Pan-European network for providing easy discovery and
access of data and derived products to users from science, industry
and general public
As part of EMODNET development recent 4 data pilot tenders have
been set out by the EU EMODNET Taskforce for hydrography, geology,
chemicals and biology
SeaDataNet subgroups have been successful in their application and
are awarded the projects execution
NODC as partner involved in the pilot for chemical data, that will be
developed using the SeaDataNet (and NODC infrastructure) in close
cooperation with the Marine Conventions (OSPAR for North Sea –
Atlantic Ocean) and EEA (Marine WISE and SEIS)
NODCi – FES proposal
New national funding programme – FES
NODC has applied for further development of the NODCi
infrastructure:
Expanding number of connected data holding institutes with
IMARES and the Hydrographic Office
Including socio-economic data sets
Additional functionalities for visualisation, presentation and
analyses of data sets
Developing and testing support and analysis tools for a number of
use cases, such as planning of new windfarms
Demonstration