Transcript COD - Ocean Information Technology

Ocean Information Technology
New approaches to ocean data management
Neville Smith
BMRC, Australia
COD, Brussells, 25-27 Nov
COD, Brussells,
25-27 Nov
The Motivation … The Issues







We have only 5-10% of the needed telemetry
capacity.
Data Assembly - no broad agreement on how we
ensure coherent, integrated data sets.
Lack of agreement on standards, formats and
practices.
Poor modes of data transport, both routinely for
operational requirements and for science.
Poor or inappropriate uptake of modern
information technology.
No proper assessment of the future need.
…
COD, Brussells,
25-27 Nov
Some Foundations

Some good examples of D&IM at work
 IODE, JCOMM
 OBIS (CoML), Argo DM, ODINAfrica, US IOOS Data plans …

but
 poorly integrated
• “not invented here” syndrome  many realizations of the
same function/form
 Too much in-house technology, too little out-sourced
functionality
• Too little fusion / cross-sector adoption
 No / little engagement with scientific community
• Innovation not within
–
…
COD, Brussells,
25-27 Nov
Rationale (The Why)
• Approach IT as you would a Science Program






Effective telecommunications
Common standards, practices and protocols
(metadata management)
Data and product service matched to the
participants and users
Innovative data inquiry, access and delivery
Intra-operability and interoperability
…
COD, Brussells,
25-27 Nov
The Components … the “How”
1.
2.
3.
4.
5.
6.
7.
8.
Telemetry
Metadata Management
Data assembly, data set integrity, quality control
Data circulation and transport
Archives and archaeology
Applications and user interfaces
Capacity enhancement, training
Governance, oversight, metrics
COD, Brussells,
25-27 Nov
1. Telemetry and Communications
… getting the data “in” (instructions
out?)
a) Availability and capability
 Bandwidth, 2-way comms, connectivity, …
b) Requirements for climate, open ocean
c) Requirements for coastal regions
d) Polar / remote regions
Bandwidth
… enabling
COD, Brussells,
25-27 Nov
2. Metadata Management
Semantic Metadata
• Data about data …
• Its environment
• Its structure
Data
• Its meaning
• Keywords

Syntactic Metadata
IODE, JCOMM (CBS) and others have made considerable progress
COD, Brussells,
25-27 Nov
2. Metadata …
Developing an agreed vocabulary

–
–
–
–


Perhaps agreeing on a Markup Language: MarineXML or …
(e.g., IODE/ICES Project)
An essential element for intra-operability, interoperability
and exchange
Non-physical / unconventional: a challenge
Complying with mandated requirements
Complete, consistent, searchable metadata
– Keywords, geospatial references
– Adopt standardized, shared vocabularies
– Developing catalogues
– How to do highly effective searches in distributed
systems?
How to automate, encourage, …
COD, Brussells,
25-27 Nov
3. Data assembly, quality control, …

More formality, accepted procedures, …
–
–
From Level 0 (raw) through Level 3 (analysed)
TOGA, WOCE, JGOFS experience
Performance level
monitoring
COD, Brussells,
25-27 Nov
3. Datum and Data Set Integrity

Develop methodology for uniquely
identifying original data and variants
• Argo is being used as one test-bed
• Original data given and always retains “tag”
• Variations referenced against original

Accreditation
– Eg, at level 2: a) no QC; b) auto; c) scientific
– Institute A, A+, A++ system, esp. for c)
•
•
•
•
Recognize value adding
Recognize scientific involvement
Agreed peer-review system
…
?
COD, Brussells,
25-27 Nov
4. Data Circulation and Service


Timeliness, efficiency
Architecture for data/product serving and exchange
•
•
•
•

Data packets and sets arrive with standard metadata and “tag”
Systems like IDD are used to “push” data to routine users
The sources may be real-time or high-quality delayed-mode
Each ocean data distribution
centre of
hasDCS:
instruction sets for
Components
distribution; event driven
Data
Transport
Also request-reply (pull)
servers
– US Data
Communication
initiative …
OPeNDAP (DODS) is an IOOS “pilot” activity
– Discipline-neutral, low entry barrier
– Can immediately begin building DCS
– Must develop the “ocean data model”
(standards) in parallel
Parallel work on blending data push/data pull
COD, Brussells,
25-27 Nov
US IOOS Model
Circuits, data
conveyers
COD, Brussells,
25-27 Nov
Together  New Functionality






The XML standard (or an equivalent) should ensure
all data are properly described / characterized
Some of the needed metacode can be generated
automatically
The “tags” allow identification and removal of
duplicates
There is immediate identification of nonconforming data insertion
No need to be a major centre to participate
(provider or user)
Some traditional data archive (backup) could be
automated
COD, Brussells,
25-27 Nov
5. Archives, Centres

Reference other sessions
 Centres might specialise in a type of service
–
–
–
–

Routine versus ad hoc / itinerant
Sophisticated versus non-specialist
Strong link to User Interface theme
Catalogues and Portals: Modern Library
Irreplaceable observations, data products of lasting
value and associated metadata preserved in
perpetuity
– Accessible using standards, keywords
– Integrated into real-time and delayed mode data transport
mechanism

Data archaeology
COD, Brussells,
25-27 Nov
6. The User Interface

Must make data and information:
– Available, usable, accessible, …

Scope
– Assess “external” user requirements
– “Wizards” to simplify interface
– Assess “internal” user requirements
– Robust interfaces; guaranteed responses

Several existing innovative client interfaces
– Live Access Server and derivatives
– GIS Systems; Web Map Server
COD, Brussells,
25-27 Nov
E.g., GODAE LAS Product Server: UK
http://www.nerc-essc.ac.uk/las
Daily 1/9o FOAM N Atlantic data
Menu for UK
FOAM &
POLCOMS
data
SST for 7 Jun 2002
COD, Brussells,
25-27 Nov
7. Capacity, “education” …

Some of the greatest challenges are in the area of
community outreach and organizational behavior
rather than in technology! [US IOOS DCS SC]
 Challenges
– Breaking down traditional barriers
– Capacity enhancement: even, “fair”
– Adoption and acceptance
– Uneven access to technology, people
 Reference other presentations
COD, Brussells,
25-27 Nov
Conclusion

There is broad consensus on the need to act
 We have an outline of the strategy: why, how
 Enormous potential to impact science and
applications; not discipline bound
 Architects and engineers are already busy
 Critical that we have broad participation
 Starting now
– http://ioc.unesco.org/oit
COD, Brussells,
25-27 Nov
Thank you
MOI?