Astro Working Group

• • • •

Officers:

Nic Walton (Astrogrid, Cambridge) Guy Rixon (Astrogrid, Cambridge) Secretary: Reagan Moore (SDSC)

GGF paper in review on



Building IVOA Services for the Astrophysics Community Examines how to

  

Map IVOA requirements onto Global Grid Forum standards Assess relevance of standards from each working group

 

Needs to be updated Written 22 September 2005 Seeking feedback from each working group Issues

 Many of the IVOA services are being implemented in advance of the GGF standards   IVOA must deal with small data (2 MB files) and large data (2 GB files)  Latency is important

IVOA must deal with collections, catalog, image archives

www.gridforum.org

GGF-17 Astronomy Application Meeting

Intellectual Property Policy

• • • • • • • • •

I acknowledge that participation in GGF8 is subject to the GGF Intellectual Property Policy.

Intellectual Property Notices Note Well: All statements related to the activities of the GGF and addressed to the GGF are subject to all provisions of Section 17 of GFD-C.1 (.pdf), which grants to the GGF and its participants certain licenses and rights in such statements. Such statements include verbal statements in GGF meetings, as well as written and electronic communications made at any time or place, which are addressed to: the GGF plenary session, any GGF working group or portion thereof, the GFSG, or any member thereof on behalf of the GFSG, the GFAC, or any member thereof on behalf of the GFAC, any GGF mailing list, including any working group or research group list, or any other list functioning under GGF auspices, the GFD Editor or the GWD process Statements made outside of a GGF meeting, mailing list or other function, that are clearly not intended to be input to an GGF activity, group or function, are not subject to these provisions.

Excerpt from Section 17 of GFD-C.1 Where the GFSG knows of rights, or claimed rights, the GGF secretariat shall attempt to obtain from the claimant of such rights, a written assurance that upon approval by the GFSG of the relevant GGF document(s), any party will be able to obtain the right to implement, use and distribute the technology or works when implementing, using or distributing technology based upon the specific specification(s) under openly specified, reasonable, non discriminatory terms. The working group or research group proposing the use of the technology with respect to which the proprietary rights are claimed may assist the GGF secretariat in this effort. The results of this procedure shall not affect advancement of document, except that the GFSG may defer approval where a delay may facilitate the obtaining of such assurances. The results will, however, be recorded by the GGF Secretariat, and made available. The GFSG may also direct that a summary of the results be included in any GFD published containing the specification.

GGF Intellectual Property Policies are adapted from the IETF Intellectual Property Policies that support the Internet Standards Process.

www.gridforum.org

GGF-17 Astronomy Application Meeting

Infrasructure - NVO Summer School

•

Web technology



Tomcat – webserver

   

SQL structured query language XMLSchema / Xpath / Xquery / XML Stylesheet Language WS-Security / SAML environment SOAP messages

 

SOAP attachments / DIME, MTOM WSDL services



WSRF statefull and asynchronous web services



Languages



Java

 

C# programming language (Microsoft.Net and open source Mono version) IDL Interactive data language – libraries to support reduction, analysis, visualization of astronomical data

 

PHP scripting language and interface to MySQL database Python scripting language



Data management



MySQL – database



Teragrid grid technology and data grid technology

www.gridforum.org

GGF-17 Astronomy Application Meeting

Data and Metadata Standards

•

Data Encodings

  

VOTable – XML standard for the interchange of data represented as a set of tables. The standard separates data and metadata, and uses XML tags to allow validation and transformation of an input document.

FITS – Image encoding standard ADQL – Astronomy Data Query Language extensions to SQL for spatial queries

•

Virtual Observatory Metadata

  

VOResource description standard for registering catalogs. Components include the access URL for the information contained in the resource.

UCD – IVOA controlled formal vocabulary for astronomical data. Uniform Content Descriptors are built by combining words from the controlled vocabulary to define the type of physical quantities. The syntax specifies the general category and specific instance as “category;instance;type”. Examples are “phys.temperature” for temperature, “phot.flux;em.radio;arith.ratio’ for a ratio of radio fluxes.

STC – Space-Time Coordinate descriptors

www.gridforum.org

GGF-17 Astronomy Application Meeting

Virtual Observatory Access Mechanisms

•

SIAP - Simple Image Access protocol returning pointers to the requested information encapsulated in a VOTable format.

•

SSAP - Simple Spectral Access Protocol for one dimensional spectra, time series, and Spectral Energy Distributions. Returns pointers to the information encapsulated in a VOTable format.

•

Cone-Search – catalog access

•

VOEvent – event notification service

•

VOStore – access to image archives

•

VOSpace – management of shared data collections

www.gridforum.org

GGF-17 Astronomy Application Meeting

Virtual Observatory Services

• • • • •

Montage Mirage Topcat – dynamic creation of mosaics of images VOPlot - visualization Downloadmanager script to support queries on Cone/SIAP services and file download

• • • •

ReadVotable java program to list the rist line of a VOTable and all its fields Mosaic – creation of mosaics of images retrieved from image archives Voogle – Virtual Observatory interface WESIX – extract source catalogs from stacked Sloan Digital Sky Survey images

• • •

Object classifiers Outlier-detection FITS header validation

www.gridforum.org

GGF-17 Astronomy Application Meeting

Portals

• • •

OpenSkyQuery – correlation service for cross matching records from major catalogs in small areas of the sky. Parameters are selected using criteria based on comparisons between values.

SkyNode – standard interface to existing catalogs Virtual Observatory Registry – OAI-PMH compliant registry of available Skynodes

• •

SkyPortal – interface for composing ADQL queries IRAF Image Reduction and Analysis Facility – general purpose software system for the reduction and analysis of optical/infra-red astronomical data. Commands include task invocation, parameter specification, graphics/image display, interactive and batch execution, command language scripting

www.gridforum.org

GGF-17 Astronomy Application Meeting

Mapping of Services to Grid

• Utility  How the proposed GGF standard might be used to implement a Virtual Observatory Protocol.

 What advantages the GGF standard would provide over the current implementation.

• Virtual Observatory requirements  Support for interactions with existing catalogs and image archives.

 Interoperability with Virtual Observatory community standards.

• Viability concerns:  Ease of implementation of the GGF standards  Robustness of the standard.

 Scalability of the resulting service to support manipulation of 10 Terabyte collections and catalogs with 1 billion records.

 Timeliness.

www.gridforum.org

GGF-17 Astronomy Application Meeting

Evaluation

•

Paper examines each working group and makes a cursory attempt to evaluate

 Utility  Meeting Virtual Observatory Requirements  Viability •

At the moment, few of the standards efforts have viable technology

 We need input from GGF working groups to demonstrate why the Grid services will enable IVOA services www.gridforum.org

GGF-17 Astronomy Application Meeting

IVOA Services

 Virtual Observatory Protocols  SIAP - Simple Image Access protocol returning pointers to the requested information encapsulated in a VOTable format.

• Requires extraction of a subset of an image from a remote collection • SAGA / Grid File System / DFDL / GridFTP / Byte IO / OGSA naming • Grid Remote Procedure Call    SSAP - Simple Spectral Access Protocol for one-dimensional spectra, time series, and Spectral Energy Distributions. Returns pointers to the information encapsulated in a VOTable format.

• OGSA DAIS / DFDL / GridFTP / OGSA naming • Grid Remote Procedure Call Cone-Search – catalog access • OGSA DAIS / DFDL / GridFTP / OGSA naming / Grid Information Retrieval VOEvent – event notification service • Transaction management / Information Dissemination www.gridforum.org

GGF-17 Astronomy Application Meeting

IVOA Services

  VOStore – access to image archives • Grid Storage Management / GridFTP VOSpace – management of shared data collections • OGSA Naming / OGSA data replication / Grid File Systems / www.gridforum.org

GGF-17 Astronomy Application Meeting

GGF Services

• • •

Infrastructure Standards Groups

 Ipv6    Network Measurement Data Transport Grid High-Performance Networking  Network Measurement for Applications

Data Standards Groups

 Data Access and Integration Services          Grid File Systems Data Format Description Language GridFTP Grid Storage Management Information Dissemination OGSA Data Replication Services Transaction Management OGSA Data Byte IO

3Compute Standards Groups

 Grid Resource Allocation Agreement Protocol    Job Submission Description Language Grid Scheduling Architecture OGSA Basic Execution Services www.gridforum.org

GGF-17 Astronomy Application Meeting

GGF Services

•

Architecture Standards Groups

 Open Grid Services Architecture   Grid Protocol Architecture OGSA Naming •

Applications Standards Groups

     Grid Remote Procedure Call Grid Information Retrieval Distributed Resource management Application API Simple API for Grid Applications Grid Checkpoint Recovery •

Management Standards Groups

     Application Contents Service Configuration Description, Deployment, and Lifecycle Management Grid Economic Services Architecture OGSA Resource Usage Service Usage Record •

Security Standards Groups

   Open Grid Service Architecture Authorization OGSA-P2P-Security Firewall Issues  Trusted Computing www.gridforum.org

GGF-17 Astronomy Application Meeting