Transcript GEOWOW-enhanced data discovery, access and exploitation

Project overview and
new achievements for GEOSS
GEOSS interoperability for
Weather, Ocean and Water
THEME[ENV.2011.4.1.3-1]: Inter-operable
integration of shared Earth Observation in the
Global Context
Duration: Sept. 1, 2011 – Aug. 31, 2014
Total EC funding: 6,399,098.00 €
Project Web Site: www.geowow.eu
GEO-X
January 14th, 2014
Roberto Cossu
ESA
[email protected]
Hervé Caumont
Terradue
[email protected]
EC Grant Agreement no. 282915
GEOWOW Vision
ooooo
Digital Earth Communities
A long-term vision for GEOSS - GCI evolution …
…considering feedback from all the stakeholders___
More user categories
data providers, data specialists, multidisciplinary scientists, decision makers
More flexible architecture
community components, resource enablers, innovation opportunities
2
A vision for GEOSS
Digital Earth Communities
• Present situation
– expert scientific advice identifies
the required datasets
– discovery: from the GWP the
datasets of interest
– downloads of individual
datasets to local computing
– heavy algorithm management &
chaining for each host workflow
& data input
• Future GEOSS operations ?
– expert scientific advice identifies
the required datasets
– assembly: needed datasets are
assembled on demand into the
GEOSS infrastructure (into a
cloud?)
– data as a service can live &
remain ‘in the cloud’, for the
application of algorithms and
calculation of indicators
– repeatable environments allow
for easy updates (in time or with
new concepts) & scientific sharing
3
GEOWOW components/enablers
Digital Earth Communities
• Components for Enhancing data discovery and access
[Brokering Framework Evolutions] including result ranking
and semantic-enriched search
• Components for Developing Enablers* [Developer Cloud
Sandbox, Cloud Controller], for the development of geoprocessing components that can interface HPC resources
*Enablers are customizable according to community-specific characteristics
4
Cloud Sandbox main concept
Digital Earth Communities
Think to …
…. a sandbox in a playground where
children can play with bucket and
spade to build a castle
A virtual work environment in GEOSS where
scientists can use dedicated tools to
build/test and run new models/algorithms,
with the ability to retry and modify
A developer’s cloud sandbox includes also
interfaces to Cloud services enabling data
intensive analysis
5
Digital Earth Communities
The scientist can manage the
sandbox service through a simple
interface, e.g., defining input
parameters, triggering the algorithm
execution, monitoring the
processing status
The scientist can access in a secure
way the sandbox service and code a
specific algorithm
Access to and sharing of
the sandbox service
Developer Cloud Sandbox
service
Community-specific clients are
able to easily communicate input
parameters, included input products,
and invoke the execution of the
algorithm (standard OGC WPS).
The sandbox environment is
sharable among trusted
communities for e-collaboration
Once the algorithm is
“consolidated”
it can be shared with the wider
GEOSS community as a SaaS
e.g. through the WPS interface
6
Digital Earth Communities
Developer Cloud Sandboxes - Achievements
7
Catalyzing resources
Digital Earth Communities
Support researchers to compute
indicators for policy makers
Take-up for future commercial
applications (under ad-hoc conditions)
Feed resources to Computing Clusters
to run global & regional marine
ecosystems assessments
Go through research oriented, not for
profit, uses of TIGGE-LAM data in
order to spread innovation
Expand uses of ESA satellites data
Development of new applications
leveraging innovative uses of earth
observations
8
Capacity Building
Digital Earth Communities
Ability to leverage ESGF’s CMIP5
Climate projections data, slice it and
process it
Support reproducibility of scientific
experiments & open science
Explore and visualize ECMWF data
from integrated Cloud appliances
Improve accessibility of key TIGGE
data for a wide user community and
better support ECMWF partner users
Experiment with ENVISAT
SCIAMACHY, global measuring of trace
gases in the troposphere and in the
stratosphere
Towards integration of Earth
Explorers missions: catalogs &
processors on Cloud Sandboxes
9
Data Sharing
Digital Earth Communities
Ocean Biogeographical
Information System (OBIS)
Transboundary Waters
Assessment Programme (TWAP)
indices and indicators
ESA Missions data (archived and
Earth Explorer)
TIGGE and TIGGE LAM, THORPEX
Interactive Grand Global Ensemble
Limited Area Model
ERA Re-Analysis archives
Earth System Grid Federation –
Model Intercomparison Project
Phase 5 (CMIP5) data
10
Digital Earth Communities
ESA’s experience using the sandbox
USER PERSPECTIVES
11
Sandbox Experiments @ESA
Digital Earth Communities
VEGetation ANalysis
through MERIS time
series
Cryosat-2 analysis in
preparation to
Sentinel-3
SCIAMACHY mission
reprocessing for
mesospheric Sodium
analysis
Algorithm
implementation,
testing and
evolution
Use of cloud
computing for
massive data
processing
12
#1: Vegetation analysis through
MERIS time series
Digital Earth Communities
Exploring Large Earth Observation Temporal Series for Land Change Events Detection
•
•
•
•
Some samples products are copied in the “sandbox”
An SSH connection to the sandbox allow the scientist to implement the algorithm and define a
workflow, installing any library of interest in the sandbox, e.g. “R”, MERIS toolboxes etc.
The scientist tests the algorithm on few samples
The scientist runs on the full dataset of interest (thousands of datasets) exploiting Cloud resources
if needed
For an easy access to data,
metadata and workflow
management
R libraries
13
#2: Cryosat showcase
Digital Earth Communities
•
Cryosat mission: three years of observations for climate
research
•
ESA has prepared a Processor Prototype generating:
•Radar Echogram
•SSH, SLA (W/O SSB), SWH, sigma0, wind speed
•
The objectives of the experiment using the SANDBOX are:
•to experiment in house research themes that will be further exploited
in the ESA-funded R&D projects
•to provide expert users with consolidated SAR geo-products to get
acquainted with the novelties and specificities of SAR Altimetry
14
#3: Sciamachy Showcase
Digital Earth Communities
Retrieval of Mesospheric Sodium densities from SCIAMACHY daytime limb spectral
•
•
•
•
The experiments allowed scientists to extent the results analysis obtained from a preliminary analysis (in the
temporal frame 2002-2007) to the full archive of SCIAMACHY data (2002-2012) .
This gives the possibility to identify trends.
Furthermore this analysis may allow the possibility to define a new inversion model for more accurate vertical
profiles.
Scientists are currently analyzing the obtained results.
15
Digital Earth Communities
Developer Cloud Sandboxes – What’s next
PERSPECTIVES
16
Coming next
Digital Earth Communities
Have a focus on results
dissemination: success stories,
best practices, online tutorials &
demos, and getting some
external users on the GEOWOW
Platform.
17
Project overview and
new achievements for GEOSS
GEOSS interoperability for
Weather, Ocean and Water
THEME[ENV.2011.4.1.3-1]: Inter-operable
integration of shared Earth Observation in the
Global Context
Duration: Sept. 1, 2011 – Aug. 31, 2014
Total EC funding: 6,399,098.00 €
Project Web Site: www.geowow.eu
EC Grant Agreement no. 282915
GEO X Plenary
Geneva, 14 January 2014