Transcript Grid Content Evolution & Management Institute for Remote Sensing Technology

Grid Content Evolution & Management
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management:
Enabling e-Science for the Rest of Us
Thilo Ernst
Fraunhofer Institute for Computer Architecture and
Software Technology ([email protected])
Jochen Wauer
DLR Institute for Remote Sensing Technology
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
Scientific Software Just A Mouse Click Away (1)
A 1968 vision:
Archivierungsangaben
... Back to our hypothetical user. He seats himself at his console [...]. He dials his
local computer and 'logs in' by presenting his name, problem number, and
password to the monitor program. He calls for either a public program, one
of his own programs, or a colleague's program that he has permission to
use. The monitor links him to it, and he then communicates with that
program...
- J.C.R. Licklider, "The Computer as a Communication Device"
(Director, ARPA-Information Processing Techniques Office)

Did he just mean telnet?
Seite 3
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
Archivierungsangaben
e-Science – a contemporary Definition
•
„e-Science is about global collaboration in key areas of science and the next
generation of infrastructure that will enable it.“
•
„large scale science that will increasingly be carried out through distributed
global collaborations enabled by the Internet. Typically, a feature of such
collaborative scientific enterprises is that they will require access to very
large data collections, very large scale computing resources and high
performance visualisation“
•
„The Grid is an architecture proposed to bring all these issues together and make
a reality of such a vision for e-Science“
•
„e-Science will change the dynamic of the way science is undertaken“ (*)
(Source: UK Research Council e-Science Core Programme)
Seite 4
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
The entire story?
• Grid computing / e-Science still focus on high-demand, large-scale
applications
• Evolution: high performance computing  Grid computing  e-Science
renders an emphasis on such applications understandable
Archivierungsangaben
• But when mature Grid & e-Science technologies are available they will likely
be attractive to, a wealth of smaller (and computationally less demanding)
research efforts - e-Science will become ubiquitous (*)
• But for this to happen, these technologies must develop additional
characteristics unrelated to the original HPC focus of the Grid movement
Seite 5
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
Scientific Software Just A Mouse Click Away (2)
2003: Are we there?
Archivierungsangaben
• Internet & WWW are ubiquitious and have revolutionized the exchange
of scientific information
•
But progress w.r.t. scientific software lacks behind when compared with
the ease of accessing scientific documents today
•
„Reuse by download“: requires installation or even porting – limiting
factor for broad exchange
• Distributed computing technologies exist (remote shells, CORBA, JavaRMI, Web Services) but there is no reliable, ubiquitous infrastructure to
which Licklider‘s request could be addressed.
Answer: No.
Seite 6
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
Scientific Software Just A Mouse Click Away (3)
... New question: Why?
• in much of real life science
Scientist  Computer Scientist
Archivierungsangaben
• (non-Computer) Scientists, beyound a reasonable limit of extra effort,
simply don‘t have time and/or motivation to do our (CS/IT) work:
- build, deploy, maintain e-Science-supporting platforms
- integrate scientific applications into these platforms on a regular basis
- and learn how to do all that in the first place
Seite 7
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
Archivierungsangaben
How did the Web succeed as a publishing medium?
Characteristics
Enabling factors, ~time
Evolutionary advantage
1. easy to use in read-mode
for almost everybody
Browsers ~ 1993
Search engines ~1994
more
„consumers“
2. easy to write without web
developer background
LaTex2HTML, MS-Office
web integration ~1996
more
„producers“
3. collaboration/community
support
(Cross-site links ~1990)
community portals ~1999
Mutual reinforcement
community / medium
4. support for traditional
editorial processes
content management
systems ~1999
organize content growth,
integrate with print media
5. infrastructure ubiquitous,
commoditized, reliable
ISPs/hosters become an
industry (~1998)
Specialization /
division of work
Seite 8
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
Towards „Broad e-Science“ – what is missing?
For e-Science/Grid technology to achieve the role of the primary collaboration medium in
„broad scientific computing“, it seems desirable to develop similar „platform survival
characteristics“:
1. Ease of use for passive users („consumers“) – Grid/Science portals / Desktops
2. Ease of use for „producers“ (authors of services) - ?
3. Collaboration support – at the core of the Grid / e-science movement
Archivierungsangaben
4. Support for editorial processes - ?
5. Infrastructure gets ubiquitous & commoditized – Industry interest
Seite 9
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
DLR VirtualLab: an e-Science precursor (1)
- technology transfer project
2000-02 („50%“)
- make scientific software accessible
from every web browser (like
scientific documents already are)
- tap unexploited application potential
Archivierungsangaben
- promote technology transfer as well as
scientific exchange (internally, too!)
- make better use of this part of DLR‘s
scientifíc output
Seite 10
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
DLR VirtualLab: an e-Science precursor (2)
- online at http://vl.nz.dlr.de
- Pre-Grid (dedicated cluster)
- Zope/Python,OpenPBS, mySQL,
OpenLDAP, ...
- a specialized ASP platform for an
„interesting“ class of scientific software
Archivierungsangaben
- IEEE CiSE, Feb. 2003
- project now continues as a
DLR/Fraunhofer collaboration
Seite 11
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
DLR VirtualLab: Concepts for broad adoption of e-Science(1)
Care about the process
- learn from publishing
(traditional & web-based)
I NTRANET
I NTERNET
Internal user
External users
Author
Repository
- use staging !
Archivierungsangaben
Integrator
Application Server
- keep the barrier to entry low
both for „consumers“ and for
„producers“
Scientific
Programs
Administrator
registered
external user
Anonymous
Seite 12
external
user
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
DLR VirtualLab: Concepts for broad adoption of e-Science(2)
Make life easy for consumers &
producers
- don‘t exclude important classes of users
- „black-box“ integration: no internal
modifications whatsoever
- Provide „cheap“ UIs through generation
Archivierungsangaben
- strive to re-use existing (G)UIs
- Integration tools, perhaps web-based
Seite 13
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
DLR VirtualLab: Concepts for broad adoption of e-Science(2)
Make life easy for consumers &
producers
Archivierungsangaben
- don‘t exclude important classes of users
ID(na='Lambda',
# Name
pe='_NF and _Lambda_use',# precondition
ty='FloatType',
# type
- „black-box“ integration: no internal
modifications whatsoever
de='0.5',
# default value
- Provide „cheap“ UIs through generation
ce='Lambda > 0.0',
# constraint
- strive to re-use existing (G)UIs
un='mu_m',
# physical unit
- Integration tools, perhaps web-based
an='Input wavelength')
# annotation
Seite 14
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
DLR VirtualLab: Concepts for broad adoption of e-Science(2)
Make life easy for consumers &
producers
- don‘t exclude important classes of users
- „black-box“ integration: no internal
modifications whatsoever
- Provide „cheap“ UIs through generation
Archivierungsangaben
- strive to re-use existing (G)UIs
- Integration tools, perhaps web-based
Seite 15
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
DLR VirtualLab: Concepts for broad adoption of e-Science(3)
Integrate human-readable
information with executable
(software) resources
- online help and background info
Archivierungsangaben
- (computational) „service search
engines“ for managing the new eScience „resource deluge“
-Structured metadata are good, but
documentation can serve as a good
fallback „fingerprint“
Seite 16
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management - e-Science For The Rest Of Us
T.Ernst, J.Wauer
Conclusions
• e-Science will likely evolve to broad/ubiquitous e-Science
• support not only HPC users but the broader/general scientifc computing community
Archivierungsangaben
• to help that process, make life easy for
„ Grid content
producers“
- strive for allowing black-box integration of apps & (G)UIs
- provide integration tools
„ Grid content
consumers“
- don‘t exclude important groups
- simple browser-based access for occasional users remains valuable
A widely deployed Grid-based e-science infrastructure is a publshing medium
- support the process by content management, leveraging suitable technologies
Seite 17
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing