Transcript TRITON / TAO Data Access

Establishing a User-Driven, WorldClass Oceanographic Data Center by
the Right People, in the Right Place ,
and at the Right Time
L. Charles Sun
National Center for Ocean Research
20-24 June, 2005, Taipei, Taiwan
Outline
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
Time, Place, and People
Steps in Establishing an NODC
Mission and Role of an NODC
QC and QA
Products and Services
Information Technology
Organizational Considerations and Chart
“Collaboratory”
IDARS, Argo & GTSPP: Three examples of “Collaboratories”
Data Portal: “Gateway” to Ocean Data
Climate Data Portal: The Proven Prototype
Other Technologies for the Collaboratory
The Future
2
Time, Place, and People
Time: Since 1975 ~
 Place: The Center of the world
 People: We are the right people

3
Steps in Establishing an NODC - I
Recruit a team of interested parties to
propose a mission and organizational
model for the center.
2 Construct a draft mission.
3 Conduct negotiations with the potential
partners.
1
4
Steps in Establishing an NODC - II
Prepare a draft administrative
organization.
5 Prepare a final version of the mission and
information on partnerships for final
approval.
4
5
Organization Chart
Office of the Director
Director
Deputy Director
Associate Director
Staff
Ocean Dynamics
Chief
Data Base Management
Chief
Information Technology
Chief
Library
Chief
Data Processing
Research Data and
Product Development
Data Archival
Database Development and
Maintenance
Networking
Operating System Maintenance
Hardware/software purchase and
Maintenance
Service
6
Mission of an NODC
To safeguard versions of
oceanographic data and information.
 To provide high quality data to a wide
variety of users in a timely and useful
manner.

7
Roles of an NODC
Conventional role – as a minimum
 Contemporary role – in response to
advances in data collection and
information technology

8
Conventional Role - I
Receive data, perform quality
control, archive and disseminate it
on request.
 Keep copies of all or part of its data
holdings in the format in which the
data were received.
 Developing and protecting national
archives of oceanographic data

9
Conventional Role - II
Produce and provide inventories of
its holdings on request.
 Referral of the users to sources of
additional data and information not
stored in the NODC.
 Participate in international
oceanographic data and information
exchange.

10
Contemporary Role - I
Receive data via electronic networks
on a daily basis, process the data
immediately, and provide outputs to
the user or to the data collectors for
data in question.
 Report the results of quality control
directly to data collectors as part of
the quality assurance module for the
system.
11

Contemporary Role - II
Process and publish data on the
Internet and on CD/DVD-ROMs.
 Publish statistical studies and
atlases of oceanographic variables.
 Performing a level of quality control
on its data holdings

12
Quality Control and Assurance

Data can be detected easily by a data
center
Obvious errors such as an impossible date and
time and location

Data cannot usually be detected by a
data center
Subtle errors such as an instrument may be off
calibration
13
Information Technologies - I
Data Storage/Archive
 Data Processing
 Local Area Networking
 Wide Area Networking – the Internet
(and the GTS)

14
Information Technologies - II
Publishing DVD/CD–ROMs
 Graphics Capability (Graphical
Information System)
 Software Development &
Implementation
 Hardware procurement &
Maintenance

15
Products Development - I

Work with the client to determine
what the real need. Examples of data
products include atlases, datasets of
ocean observations filtered by area,
time and variables observed
16
Products Development - II

Review the world wide web sites of
existing NODCs for ideas and
examples of data and Information
products.
17
Services
Providing directory and inventory
information
 Acting as a referral center
 Receiving data for specific
processing followed by delivery of the
processed data

18
Organizational Considerations


A centralized data center
A distributed data center
Centers of Data : “Data Portals” or “Virtual Collaboratories”
Data Center
Center of Data A
Center of Data B
Center of Data C
19
What is a Collaboratory?
The fusion of computers and electronic communications has
the potential to dramatically enhance the output and
productivity of researchers. A major step toward realizing that
potential can come from combining the interests of the
scientific community at large with those of the computer
science and engineering community to create integrated, tooloriented computing and communication systems to support
scientific collaboration. Such systems can be called
"collaboratories."
From "National Collaboratories - Applying Information Technology for Scientific
Research," Committee on a National Collaboratory, National Research Council.
20
National Academy Press, Washington, D. C., 1993.
Acknowledgement
Soreide, N. N. and L. C. Sun, 1999:
Virtual Collaboratory: How Climate Research can be done
Collaboratively using the Internet. U.S. – China Symposium
and Workshop on Climate variability, September 21-24, 1999,
Beijing, China
Presented by Len Pietrafesa, North Carolina State University.
21
Collaboratory Infrastructure

Data Portal
– Computer and networking hardware and software
– Increased network bandwidth/speed
– Next Generation Internet (NGI) connection

Visualization
– Interactive Java graphics
– 3D, Virtual Reality, collaborative virtual environments
– immersion technology CAVE, ImmersaDesk...

Relationships:
–
–
–
–
Observing System Project Offices
Research community, Academia...
Other Collaboratory nodes
Steering Committee
22
Structure of the Collaboratory for Ocean Research
International
Steering Committee
Collaboratory
Partner
Collaboratory
Partner
Collaboratory
Partner
Collaboratory Partners & Customers
Providers of Data & Information
Users of Data & Information
Observations
&
Satellite
Groups
Modeling
&
Forecasting
Groups
New Users
Research
Groups
Educational
Administrators
General Public
23
*
IDARS
as an example...
• Real-Time Coastal Water
Temperature Data
• Real-Time Argo Profile Data
• Real-Time Global
Temperature and Salinity
Profile Data
• Time Series Data
• NOAA CoastWatch AVHRR
SST Images
http://www.nodc.noaa.gov/idars/
*Interactive
Data Access and Retrieval System
24
Argo as an example...
25
*
GTSPP
as an example...
* Global Temperature-Salinity Profile Program
26
Argo and GTSPP

Argo and GTSPP set a standard in the
international ocean data management community
 Data dissemination in near-real time
– Researcher involvement has assured data quality

Benefits of data dissemination
– Wide use of Argo and GTSPP data
– Traditional research, modeling, forecasting groups
– Related disciplines, educational, administrative, public

With recent advances in technology, we can do
much more...
27
Distributed Object Technology
Data servers and datasets are
objects – software packages of
procedures and data that contain
their own context
 Solid, commercial underpinning for
distributed object technology in the
ocean sciences

28
The Data Portal: a “gateway” to ocean data

Why do we need a Data Portal?
– Each center of data provides a highly customized Web
sites for their data
• but different datasets have different navigation and interface
characteristics
• so the user faces a bewildering spectrum of data access
interfaces and locations

Data Portal is single, uniform, consistent
“gateway” to ocean data in a common format
• User goes to a single location and sees a consistent interface
• Complements the customized data access
30
Data Portal/Visualization/Collaboration
Data &
Information Users
Traditional users:
Modelers
Forecasters
Researchers
New users:
Educators
Students
General Public
Uniform network access
Distributed data
Observed data
Satellite data
Data and information
products
Model outputs
Visualization
31
Data Portal
Data Server
One or more
Web Servers
User
Observing System Server
CORBA*
TAO data support
Web
Browser
Data
Java Servlet
Network
Client Support
Graphics
CORBA*
Java
Application
N
e
t
w
o
r
k
CORBA*
Common Object Request Broker Architecture (CORBA) is an industry standard
Middleware. CORBA is used in the NOAAServer software from which this effort
will leverage. Based on performance indicators, Java Remote Method Invocation
(RMI), an alternative middleware, could easily be substituted for CORBA.
32
Data
Data Portal
Data Servers
One or more
Web Servers
User
Observing System Servers
CORBA*
TAO data support
Web
Browser
Java Servlet
Network
CORBA*
Client Support
Graphics
CORBA*
Java
Application
Data
N
e
t
w
o
r
k
Drifter Data support
Data
CORBA*
Common Object Request Broker Architecture (CORBA) is an industry standard
Middleware. CORBA is used in the NOAAServer software from which this effort
will leverage. Based on performance indicators, Java Remote Method Invocation
(RMI), an alternative middleware, could easily be substituted for CORBA.
33
Data
Data Portal
Data Servers
One or more
Web Servers
User
Observing System Servers
CORBA*
TAO data support
Web
Browser
Java Servlet
Network
CORBA*
Client Support
Graphics
CORBA*
Java
Application
CORBA*
Data
N
e
t
w
o
r
k
Drifter Data support
Data
In-Situ/Satellite Data Servers
CORBA*
In-Situ/Satellite data
support
Data
Model Output Servers
CORBA*
Model data support
Data
Gridded Data Servers
Common Object Request Broker Architecture (CORBA) is an industry standard
Middleware. CORBA is used in the NOAAServer software from which this effort
will leverage. Based on performance indicators, Java Remote Method Invocation
(RMI), an alternative middleware, could easily be substituted for CORBA.
CORBA*
Gridded data support
34 Data
Data
How do we build a Data Portal?

Build on a proven prototype
– connects 5 geographically distributed data
servers in Silver Spring, Boulder, Seattle
– CORBA for network connections
– unified interactive Java graphics
– data from distributed servers are co-plotted
together on the same axis on the users
desktop
http://www.pmel.noaa.gov/~nns/noaaserver/nodc-coads-tao.html
http://www.pmel.noaa.gov/~nns/noaaserver/coads-tao-raster.html
35
Prototype Data Portal:
*
CDP
Silver Spring
MD
Seattle
WA
Boulder
CO
Honolulu
HI
*Climate
Data Portal
36
Climate Data Portal Sample Plots
37
Data Selection : Web Interface

Utilizes CORBA for
network connections.
 Utilizes EPIC Web
Technology:
– Java Applets
– JavaScript
– Java Servlets

Searches data by
keywords, location
and time ranges.
38
Web Interface screen Shots
39
Other Technologies for the
Collaboratory:

Networks (100 Megabits/sec today, 10 Gigabits/sec in
future)
– Next Generation Internet (NGI) and Internet 2

Visualization
– Interactive Java graphics
– 3D, Virtual reality
– Immersion technology

Collaboration tools
– high-speed telecommunications systems for advanced
collaboration applications
– tele-immersion systems allow individuals at different locations to
share a single virtual environment
40
– Use networks not airplanes for collaboration
Virtual Reality

Virtual Reality lets the scientist touch the data,
move into it, and see it from different viewpoints
–
–


The realism of virtual reality enables the scientist and the lay
person to understand complex ideas more easily
Scientists using virtual reality affirm this new technology
discloses features of their data and model outputs which were
undiscovered with standard visualization techniques
Virtual reality can be approachable and affordable
Widens audience for scientific data and
information
–
–
–
Government administrators and decision makers
Educators and students
General public
Courtesy of Nancy N. Soreides, PMEL
Some examples follow…
41
Why use Virtual Reality?
El Nino
La Nina
Virtual reality modeling language (VRML) rendering of temperatures and sea surface topography
along the equator in the tropical Pacific, viewed from South America, showing the dynamics of El
Nino and La Nina.
Using an inexpensive PC and a web browser with a free plug-in, the images can be rotated, animated,
and zoomed. Changes in the equatorial Pacific during El Nino and La Nina are clearly understood by
scientist and layman. http://www.pmel.noaa.gov/toga-tao/vis/vrml/ or http://www.pmel.noaa.gov/vrml
Courtesy of Nancy N. Soreides, PMEL
42
Stereographic Virtual Reality
Stereo
Fish larvae and velocity vectors in a submarine canyon, from a
circulation model of Pribolof Canyon in the Bering Sea. Use
red/green glasses to see images on the right in stereo.
Stereo
3D, interactive virtual reality
visualizations are not difficult for a
scientist to create or to view, from
the web or from the desktop, and the
effect can be enhanced dramatically
by including the capability of
stereographic viewing.
With a PC and a 99-cent pair of
red/green sci-fi glasses, the spheres
and vectors will pop out of the page
in stereo, revealing the true 3D
location of the fish, the steep slopes
of the bathymetry, and the vertical
motions near the submarine canyon.
The images can be rotated, animated
and zoomed.
http://www.pmel.noaa.gov/~herman
n/vrml/stereo.html
Courtesy of Nancy N. Soreides, PMEL
43
Immersive Virtual Reality

Immersive devices provide the graphical illusion of being in a threedimensional space by displaying visual output in 3D and stereo, and
by allowing navigation through the space.

Navigating through our virtual environments and viewing the data
from different vantage points greatly increases our ability to perform
analysis of scientific data.

The impact of such visualizations in person is stunning, and must be
experienced by the scientist to be fully comprehended .

Users of these advanced immersion technologies affirm that no
other techniques provide a similar sense of presence and insight
into their datasets.
Courtesy of Nancy N. Soreides, PMEL
44
View of the CAVE
The CAVE
The CAVE is a multi-person, high resolution, 3D
graphics video and audio virtual environment. The
size of a small room (10x10x10 foot), it consists of
rear-projected screen walls and a front-projected
floor.
Using special "stereoscopic" glasses inside a CAVE,
scientists are fully immersed in their data. Images
appear to float in space, with the user free to "walk"
around them, yet maintain a proper perspective.
Scientist inside the CAVE
CAVES have been deployed in academia,
government, and industry, including NASA, NCAR,
NCSA, Argon National Laboratory, Caterpillar
Corp., General Motors, among others.
http://www.pyramidsystems.com/CAVE.html
The CAVE was the first virtual reality technology to
allow multiple users to immerse themselves fully in
the same virtual environment at the same time.
45 PMEL
Courtesy of Nancy N. Soreides,
The ImmersaDesk
Courtesy of Nancy N. Soreides, PMEL
46
The Future
“The development of scientific data manipulation and visualization
capabilities requires an integrated systems approach … [including] the endto-end flow of data from generation to storage to interactive visualization,
and must support data retrieval, data mining, and sophisticated interactive
presentation and navigation capabilities.”
“Data Exploration of petabyte databases will required both technology
development and altered work patterns for research scientists and
engineers.”*
* Data and Visualization Corridors, Report on the 1998 DVC Workshop
Series, Edited by Paul H. Smith and John van Rosendale, Sponsored by the
Department of Energy and the National Science Foundation, 1998.
Courtesy of Nancy N. Soreides, PMEL
47
[email protected]
48