Transcript Spring 2006 CEOS WGISS

ECHO and EDG
Status
Sept 12, 2006
Beth Weinstein, [email protected]
Yonsook Enloe, [email protected]
1
Outline
What is ECHO?
ECHO Background - How did we get here?
Why use ECHO?
Current Functionality
Current Partners Participation
Upcoming activities
2
ECHO Mission/Vision Statement
ECHO Mission
• ECHO’s mission is to enable a global marketplace of Earth
Observation resources that will make Earth Observation data
utilization more efficient and will spark innovation. ECHO
provides Earth Observation communities with the ability to
publish, discover, access and integrate directory and inventory
level data and services through community-developed user
interfaces.
ECHO Vision
• ECHO will…
• be highly recognized, trusted and valued by the Earth Observation
community
• be a critical building block in distributed information, modeling,
decision support and public access systems
• have a low cost of participation to encourage broad community
involvement
3
ECHO as Middleware
ECHO is middleware for a service-oriented enterprise
Its capabilities can be accessed through publicly available APIs
• Based on industry standards for performing web-based computing
• Defined in the Web Services Definition Language (WSDL) and are
accessible through Simple Object Access Protocol (SOAP).
Custom clients are developed to interact with ECHO’s Earth
science metadata and extended services registry
• Clients written in most contemporary programming languages are
isolated from the underlying technologies
• Clients may call the ECHO web services much like a local function
call.
4
ECHO Background – How did we get here?
ECHO initiated as an enhancement to EOSDIS in response to:
• User feedback on complexity and limitations of the “system-wide”
view of EOSDIS data provided by EOS Data Gateway (EDG).
• Belief that the community could and would develop better client
capabilities tailored to their needs.
• Evolving NASA Earth science vision of multiple, distributed,
heterogeneous data and service providers.
• Availability of emerging technologies (e.g. web services).
Response was development of ECHO as enabling infrastructure.
• “Externalized” metadata and made it accessible via APIs that
supported development of custom clients.
• Extensible architecture that allows standard client and provider
interfaces to be added.
• Support for data services.
Centralized “clearinghouse” model based on industry feedback.
• Driven by performance and availability requirements.
5
Why Use ECHO?
Open system provides Earth science data and services to large,
diverse pool of users enabling scientific community interaction and
collaboration
Control in the hands of the data partner
• Automate mapping between your metadata and ECHO catalog
metadata
• Control visibility and access to your contributed resources
• Select the best spatial search approach for your data
• Check the history of orders and provide status on open orders
Users search for collection and inventory-level data
• Search and order data through a customized user interface
• Directly access online data and/or order data on media
ECHO offers high system availability
• 99% system availability
• Even if your system is down, ECHO users can still search your
metadata
6
ECHO Capability Today
User Registration and Login
Metadata ingest, validation, and reconciliation
Searching is accurate and flexible - support for Cartesian, geodetic and
orbital data description and subsequent search by these and the
following mechanisms
• Spatial Search (e.g. line, polygon, multipolygon, circle)
• Temporal Range Searches
• Keyword Searches
• Numeric Searches (e.g. cloud cover percentage)
• Product Specific Attribute Searches
Open interfaces (web service based) for human-machine or machine-tomachine clients (legacy interfaces are being phased out).
Data Access
• Direct On-line Access
• Brokering of Orders
• Price Quotes
• Subscriptions
Interoperability with other systems (OGC/NSDI Client support)
Service Catalog based on web services standards
7
ECHO Schedule
ECHO 7.0 operational
•Browse Data Insert, Update, and Deletes
•Multiple Collections and Groups for Access Control Rules
•Spatial Query Based on Lat/Lon Point
ECHO 8.0 operational
interface mechanism.)
(Phase out of XML APIs. Web Service APIs are primary
•Web Services API
•Asynchronous Queries
•Web Service brokering
ECHO 9.0 operational
interface mechanism.)
(Removal of XML APIs. Web Service APIs are only
•Improved performance
•More Comprehensive Error Handling
•Enhanced Security
•Metrics
ECHO 10.0 operational
•EOSDIS Evolution Items
Mar
2006
Oct
2006
Apr
2007
4Q
2007
8
ECHO Data Partner Status
ECHO’s Current Holdings (Aug 2006) from 10 Data Partners
• Collections
2,408
• Granules
66 million
• Browse
14 million
All NASA ECS DAACs are actively participating in ECHO (GES, LARC,
LP, NSIDC)
• Atmospheric Composition and Dynamics, Global Precipitation, Ocean
Biology, Ocean Dynamics, Solar Irradiance
• Radiation Budget, Clouds, Aerosols and Tropospheric Chemistry
• Land Processes
• Snow and Ice, Cryosphere and Climate
V0 DAACs are participating (ASF, GES, JPL, ORNL, SEDAC, PO)
• Synthetic Aperture Radar (SAR), Sea Ice, Polar Processes, Geophysics
• Biogeochemical Dynamics, Ecological Data, Environmental Processes
• Oceanic Processes, Air-Sea Interactions
• Population, Sustainability
MODIS Data Processing System (MODAPS) is operational
JAXA CEOP is in test mode
9
Data Provider Support in ECHO
FTP Push for submitting metadata
DTDs that govern required xml format available on
echo website for metadata submission
Required and optional Collection level and product
(granule) metadata
Browse metadata and imagery storage
Automated processes for ingesting metadata files.
Metadata update and reconciliation capabilities
Partner test system to test metadata and perform
validation before ingesting into operational system
ECHO Ops team support for data providers
10
ECHO Client Partner Status
Variety of client types
• Web-based clients
• Installed applications
• User interactive
• machine-to-machine
Variety of purposes
• General purpose (e.g. Geospatial and
temporal searching)
• Customized user interfaces (e.g. Targeted
to smaller communities, Facilitates specific
tasks)
• Back-end harvesting tools (e.g. Supports
client-side caching of key information)
• Additional value-added processing by clients
(e.g. Subset, Stitch, Resample, Reproject,
Reformat)
• Middleware components
NASA ROSES ACCESS solicitation client will
be selected in Fall 2006.
11
Custom Clients and ECHO
There are many different ways to use ECHO.
• Can use a subset of ECHO web service APIs to use only the desired
ECHO functionality (search, order, access, extended services...)
• Can search through entire ECHO metadata or service registries or a
subset
• Can search using full search criteria or limited search criteria or product
specific attributes
• Can customize the search results returned to subset of data or service
metadata or list of data access urls of result set
• Application-specific server can harvest ECHO metadata of interest and
cache it for value added processing by its own dedicated client
Clients can use ECHO Web Service APIs to register an extended service
in the ECHO Service registry and access this service through SOAP
Case example: OPeNDAP Group built a limited client access to ECHO
Service Registry in less than one month. This client access was used by
Matlab client and can be used by other OPeNDAP enabled clients. This
client invokes a search that searches for only data served by OPeNDAP
servers. Client queries by space & time, and is returned set of metadata
& OPeNDAP urls of result set.
Level of effort to build client to query ECHO using web services is very
small.
12
EDG Transition to ECHO’s WIST Client
Warehouse Inventory Search Tool (WIST)
• ECHO client being developed by NASA ESDIS
• General search and order interface
• Will offer all EDG functionality
• Public access to current ECHO operational version
WIST releases
• Preoperational WIST has been deployed for public access. While it has been
tested with ECHO, it is considered preoperational because not all of the EDG
functionality has been included yet.
• Next version of WIST released in November 2006 will include subsetting and
ASTER On Demand support.
• Operational WIST 1.0 will be available in April 2007 when ECHO Version 9.0 goes
operational.
• ECHO Partners (e.g. NASA DAACs) will continue to test and evaluate WIST as
new releases are made.
EDG to WIST transition
• Goal is to start the EDG-to-WIST transition plan in summer 2007 and turn off
EDG in late 2007.
• ECHO must meet criteria (e.g. search performance, available, up-to-date
metadata) before EDG is turned off.
• EDG and WIST will run in parallel for a period of time.
13
International Partner/ECHO Activities
CEOP (Coordinated Enhanced Observing Period) program
• Plans to use ECHO and OPeNDAP enabled clients and servers
• OPeNDAP enabled clients can directly query ECHO
• Satellite data used by CEOP will be represented in ECHO
JAXA is currently evaluating ECHO through its CEOP activity
Israel Space Agency will become an ECHO Data Partner. Also
expect to be a client partner and service partner later.
IRE RAS (Russia) will become an ECHO data partner, client
partner, and service partner.
ESA is implementing a prototype to access ECHO from INFEO
through ECHO’s web services APIs.
Dundee (Scotland) considering becoming an ECHO data partner
and client partner.
Continuing discussions with other international partners. Future
discussions with GEO partners also expected.
We welcome additional partners!
14
OPeNDAP/ECHO Activities
CEOP program
• CEOP will ingest metadata repesenting data of interest (from
JAXA, NASA, ESA, and Eumetsat) into ECHO.
• WTF-CEOP developing extensions to OPeNDAP based tools to
provide access to satellite data to the CEOP science community.
• CEOP Satellite Data Server will provide access to satellite data
served through a WCS to OPeNDAP enabled clients.
OPeNDAP prototype
• Direct search and access of ECHO through the web service APIs
by OPeNDAP clients
• Matlab client was demoed in July 2006 with operational capability
expected when ECHO 8.0 is operational.
• This capability was demoed at the WTF-CEOP session yesterday.
15
ECHO & GEO Tasks
ECHO can be a WGISS contribution to GEO – initial task list
Task DA 06-06: Spatial Data Infrastructures
Advocate use of existing Spatial Data Infrastructure components as
institutional and technical precedents, where appropriate, including standard
protocols and interoperable system interfaces, among other components.
Task AR 06-05: GEO Clearinghouse
Initiate development of a publicly accessible, network-distributed
clearinghouse, subject to GEOSS interoperability specifications to date, and
including an inventory of existing data, metadata, and pre-defined common
products.
Task WA 06-05 In Situ Water Resource Monitoring
Initiate the creation of a coordination mechanism within GEO for global in-situ
water observations, including ocean observations, and advocate synergy and
sharing of infrastructure among observing systems.
Task WA 06-07 Capacity Building Program in Latin America
Task WA 07-P2 Satellite Water Measurements
16
IDN (GCMD)/ECHO Activities
GCMD and ECHO are working together to share
information from its registries and give users a more
unified experience when interacting with the two
systems
GCMD Portal to ECHO data operational in 2Q 2007
17
ECHO Contact Information
ECHO International contact– Yonsook Enloe
[email protected]
Contact ECHO Operations (Ops)
• [email protected]
Visit the ECHO Project Website
• http://eos.nasa.gov/echo
•
•
•
•
Holdings Summary, Upcoming Functionality, APIs and DTDs
Reference materials and tools
Real-time systems status monitor
Operations metrics updated weekly
Join ECHO Mailing List: [email protected]
Schedule bi-lateral telecons to discuss potential
collaboration!
18