Transcript Document

The German Astrophysical Virtual
Observatory (GAVO)
Knowledge Networking for Astronomy in
Germany and abroad
Gerard Lemson1,2, Wolfgang Voges1, Joachim Wambsganss2
GAVO team
1 Max-Planck-Institut
2 Astronomisches
GES 2007, 3.5.2007
für extraterrestrische Physik, Garching
Rechen Institut/Zentrum für Astronomy der Universität Heidelberg
Overview
• Knowledge networking for astronomy:
the Virtual Observatory
• Standardisation: the International Virtual
Observatory Alliance (IVOA)
• The Virtual Observatory in Germany:
GAVO
• Theory in the VObs
GES 2007, 3.5.2007
The status quo
• Astronomy produce a huge range of valuable data
products,
– Of sometimes astronomical sizes (soon Petabytes/yr).
– From ground-based observatories and satellites
– From all-sky surveys, and from large variety of individual
targeted observations.
– In all wavelengths.
– Don’t forget computer simulations
• Requires large variety of disciplines/specialisations to
produce and analyse.
• Data products are relatively quickly made public (~1 yr).
– Many already available as online archives, more or less
standardised, homogeneous.
• In combination promises interesting new science.
GES 2007, 3.5.2007
Some astronomical data products
Images
Spectra
Catalogues
Simulations
GES 2007, 3.5.2007
• John Hibbard http://www.cv.nrao.edu/~jhibbard/n4038/n4038.html
• NASA/CXC/SAO/G. Fabbiano et al.
• Di Matteo, Springel and Hernquist, 2005
GES 2007, 3.5.2007
But
•
•
•
•
•
•
Where is the data?
What do these archives contain?
How can they be accessed?
How do we analyse these (very) large datasets?
How can we combine them?
...
• The Virtual Observatory (VObs) is the proposed
answer to these questions.
GES 2007, 3.5.2007
The VObs ...
Promises
• improved communication
• reuse of results
• validation
• comparison
• combination
• federation
• collaboration
• new science
Requires
• archiving
• curation
• online availability
• description
• access tools
• filtering tools
• analysis tools
• standardisation
→ O(16) national VObs projects, organized in the IVOA
GES 2007, 3.5.2007
IVOA
GES 2007, 3.5.2007
Mission statement
Facilitate the international coordination and
collaboration necessary for the
development and deployment of the tools,
systems and organizational structures
necessary to enable the international
utilization of astronomical archives as an
integrated and interoperating virtual
observatory.
GES 2007, 3.5.2007
IVOA organisation
• Since 2002
• Activities divided in working - and interest
groups, “tiger teams”
• Meet twice a year in interoperability
meetings (two weeks from now, Beijing)
• Active mailing lists, wiki pages
• Standardised standardisation process
GES 2007, 3.5.2007
IVOA activities
• Working groups create standards for
–
–
–
–
–
–
–
publication and discovery (Resource Registry)
(meta-)data description (DM, Semantics)
selection and remote filtering (DAL, VOQL)
formats for transmitted data (VOTable)
(web) services, distributed workflows (GWS)
application interoperability (Applications)
event notification (VOEvent)
• interest groups represent special interests
– Data curation
– Grid
– Theory
GES 2007, 3.5.2007
Some results
• Resource registry data models + various implementations
• VOTable format (XML schema) + many support tools
• Data models for
– space-time coordinate systems
– characterisation of observations
– spectra
• Simple data access protocols for
– source catalogues
– 2D images
– 1D spectra
• In development:
–
–
–
–
Astronomical data query language (ADQL)
Astronomical web service standards (UWS)
Application messaging
Simulation data models + access protocols
GES 2007, 3.5.2007
The German Astrophysical Virtual
Observatory
http://www.g-vo.org
GES 2007, 3.5.2007
GAVO I: 2002-2005
• BMBF funded
• Partners
– Astrophysikalisches Institut Potsdam (AIP)
– Astronomisches Rechen Institut/Zentrum für Astronomy der Universität
Heidelberg (ARI)
– Max-Planck-Institut für extraterrestrische Physik (MPE), Garching
– Hamburger Sternwarte
– Asociated partner: Max-Planck-Institut fur Astrophysik (MPA), Garching
• Activities
– R&D
– Prototyping
– Special attention:
•
•
•
•
Archive publication: ROSAT, RAVE
Data mining: cross-matching, classification
Grid computing: simulations, distributed cluster finder
Theory: virtual telescopes, archiving simulations, IVOA
GES 2007, 3.5.2007
GAVO II: 2006-2008
• BMBF funded
• Partners
– AIP, MPE, MPA,
– Technische Universität Munchen-Informatik
– Universität Tübingen
• Focus: move to scientifically useful services
• Projects
•
•
•
•
•
Millennium database (see later)
IVOA representation (theory, VOQL)
Standard services (SIA, SSA,SCS)
Custom services
VObs expertise center at ARI ...
GES 2007, 3.5.2007
VObs expertise center @ ARI
• IVOA compatible metadata repository for
community
• Implementation data access query protocols
• Storage (smaller) data sets, especially science
ready data
• Tools
• Outreach/PR
• Help-desk
GES 2007, 3.5.2007
Theory in the VObs:
some observations
• Simulations not as simple as observations
–
–
–
–
less homogeneous
complex observables
no standardisation on data formats
archiving ad hoc, for local use
• Current IVOA standards somewhat irrelevant
– no common sky
– no common objects
– requires data models for content, physics, code
• Moore’s law for N-body) simulations
– Very large simulations possible
– NB: also makes useful lifetime relatively short
GES 2007, 3.5.2007
“Moore’s law” for N-body simulations
Courtesy Simon White
GES 2007, 3.5.2007
Virgo collaboration’s
Millennium database
• Largest cosmological
simulation to date
– 10 billion particles
evolving under gravity
– 500 Mpc (~2Gly) box
– 64 snapshots
– 350000 CPU hours
– O(30Tb) raw data
• Derived data
–
–
–
–
density fields
clusters, merger trees
galaxies, merger trees
realistic, “observed”
galaxy catalogues
GES 2007, 3.5.2007
Courtesy Volker Springel
Time evolution: merger trees
GES 2007, 3.5.2007
Real and Mock catalogues
Courtesy Volker Springel
GES 2007, 3.5.2007
Database + web server
• Derived data products only
• SQLServer database
• Apache web server
– portal: http://www.mpa-garching.mpg.de/millennium/
– public DB access: http://www.g-vo.org/Millennium
– private access: http://www.g-vo.org/MyMillennium
• Access methods
– browser producing various formats, plotting capabilities
– stream based wget + IDL, R, etc allows
– finite query time (30sec-7min)
• Features
– efficient tree storage+access
– spatial indexing
– MyDB
GES 2007, 3.5.2007
GES 2007, 3.5.2007
GES 2007, 3.5.2007
Usage statistics
• Up since Aug 2006
• Community notified via preprint server
http://xxx.lanl.gov/abs/astro-ph/0608019
• 130 registered scientific users
• >1.4 million individual SQL queries
• > 4 billion returned rows (since March 8
2007)
GES 2007, 3.5.2007
Summary
• VObs is natural extension of astronomy’s history
of data archiving, standardisation, online and
open access.
• IVOA active, questions are complex technically,
but politics are sometimes even harder.
• GAVO relatively small, but has found some
niches, particularly theory.
• To be successful requires use by non-VO
scientists, requires proper PR.
GES 2007, 3.5.2007
Thank you.
Further thanks to:
Volker Springel, Simon White, Gabriella DeLucia, Jeremy Blaizot, Manfred Kitzbichler (MPA, Garching),
Carlos Frenk, John Helly , Richard Bower (ICC, Durham, UK), Alex Szalay (JHU, Baltimore)
Opening picture courtesy of NASA Goddard Space Flight Center.
GES 2007, 3.5.2007