Ganga simplifying use of the Grid

F.Brochu

1 , U.Egede

2 , J.Elmsheuser

3 , B.Gaidioz

4 , K.Harrison

1 , B.Koblitz

4 , H.C.Lee

5 , D.Liko

4 , A.Maier

4 , J.T.Moscicki

4 , A.Muraru

4 , V.Romanovsky

6 ,A.Soroko

7 , C.L.Tan

8 1 University of Cambridge, 2 Imperial College London, 3 Ludwig-Maximilians-Universität München, 4 CERN, 5 ASGC Taipei, 6 IHEP Protvino, 7 University of Oxford, 8 University of Birmingham

http://cern.ch/ganga/ Uniform User Interface Worldwide Resources HEP Computing Scope

Ganga is an easy-to-use frontend for job definition and management, implemented in Python.

It is being developed to meet the needs of the ATLAS and LHCb experiments for a Grid user interface, but offers possibilities for extension and customisation that make it interesting for a wide range of user communities. ATLAS and LHCb will investigate various aspects of particle production and decay in high-energy proton-proton interactions at the Large Hadron Collider (LHC), due to start operation at the European Laboratory for Particle Physics (CERN), Geneva, towards the end of 2007. Both experiments will require processing of data volumes of the order of petabytes per year, and will rely on computing resources distributed across multiple locations. The experiments’ data-processing applications, from event modelling to physics analysis, are based on the Gaudi/Athena C++ framework.

Ganga (Gaudi/Athena and Grid Alliance), deals with configuring these applications, allows simple switching between testing on a local batch system and large-scale processing on the Grid, and helps keep track of results.

User view

Ganga classes and functions can be used interactively at a Python prompt, can be referenced in scripts, or can be used indirectly via a Graphical User Interface (GUI). The GUI consists of a main panel, which displays job information, and three dockable windows - windows that can be hidden, integrated in the main panel, or floated separately. The dockable windows provide for job building, for execution of arbitrary Ganga/Python commands, and for the logging of system messages.

Scriptor Job details

Job representation

A job in Ganga is constructed from a set of building blocks. All jobs must specify the software to be run (application) and the processing system (backend) to be used. Many jobs will specify an input dataset to be read and/or an output dataset to be produced. Optionally, a job may also define functions (splitters and mergers) for dividing a job into subjobs that can be processed in parallel, and for combining the resultant outputs. Ganga provides a framework for handling different types of application, backend, dataset, splitter and merger, implemented as plug in classes. Each of these has its own schema, which places in evidence the configurable properties.

Logical Folders Job Monitoring Job builder Log window

Ganga tutorials for ATLAS and LHCb have been held at locations including Bologna, Cambridge, CERN, Edinburgh, Lyon, Milan, Mumbai and Munich, and online guides to getting started with Ganga are also available.

Tutorials and guides have been key factors in stimulating Ganga take-up, with more than 300 Ganga users recorded.

Ganga Tutorials at National e-Science Centre, Edinburgh Tutorial for LHCb, January 2007 Tutorial for ATLAS, February 2007

Other projects using Ganga

In addition to its use in ATLAS and LHCb, Ganga is finding application in a growing number of other projects, both as a standalone tool and as submission engine and monitoring system for the DIANE job distribution framework.

The Ganga user groups include:

• other high-energy physics experiments:

Compass, Harp

• teams developing detector simulations:

Garfield, Geant4

• biomedical projects: med-austron,

projects researching Avian flue

• projects involving image processing:

Cambridge Ontology, Xmipp

• chemistry project: WIEN2k • telecomunications industry: International

Telecommunication Union (ITU)

5 May 2006 Job statistics from Ganga

Use of the Grid to search for drugs against avian flu was widely reported. About one eighth of the Grid jobs were submitted using Ganga/DIANE The ITU Regional Radio Conference, held in Geneva in May-June 2006, required real-time optimisation of evolving plans for sharing frequencies between 120 countries.

The aim was to maximise the number of satisfied requests, and minimise interference.

Ganga/DIANE was used to run the optimisation jobs on the Grid.

Regression tests Geant4 major releases require a few years of CPU, concentrated in a short time. These tests are run on the Grid using Ganga/DIANE.

for Cambridge Ontology is developing solutions for image retrieval based on image content.

Plug-in classes specialised for the required image processing tasks have been implemented, and are used with Ganga to submit jobs to the Grid.