Transcript Ganga - nw.ru

Enabling Grids for E-sciencE
Using of GANGA interface
for Athena applications
A. Zalite / PNPI
www.eu-egee.org
INFSO-RI-508833
Overview
Enabling Grids for E-sciencE
• General introduction to ATLAS software framework Athena: structure, applications, user interface.
• Introduction to GANGA.
• Running Athena using Ganga interface.
INFSO-RI-508833
Athena framework
Enabling Grids for E-sciencE
• In practice framework is a skeleton of the application
into which developers plug in their code and provides
most of the common functionality.
• Athena - Atlas software framework based on GAUDI.
Athena provides common services such as the
transient data store, (interactive) job configuration and
auditing, data(-base) access, message streams, etc. for
Atlas software. The idea is to improve the coherency of
the different software domains within Atlas, and
thereby the ease of use for end-users and developers,
by having them all use the same well-defined
interfaces.
INFSO-RI-508833
Athena framework
Enabling Grids for E-sciencE
• The overall principle of the Athena framework is
separation of algorithm and data.
• Each algorithm implements 3 methods for invocation
by framework: an initialize() function which is executed
at the start of the job, an execute() function which is
executed every event and a finalize() function which is
executed at the end of the job.
• An algorithm has a number of control and data
parameters which allow run-time configuration. They
can be set in the startup jobOptions files.
• The idea is you can write data to the Transient Store
(TS) from one algorithm. You can then read in this data
in another algorithm and do some processeing on that
data and then write resulting data to the TS and so
forth.
INFSO-RI-508833
CVS, CMT
Enabling Grids for E-sciencE
• All ATLAS offline code is stored in Concurrent Versions
System (CVS), which manages the evolution of the
source files.
• The build of the binaries (compilation option, include
files, libraries…) as well as the run-time environment
are managed by Configuration Management Tool
(CMT).
• The software is grouped into separate packages, where
some packages may be related or "depend" upon each
other.
INFSO-RI-508833
Running Athena
Enabling Grids for E-sciencE
• First step is setup of Atlas software environment
(controlled by CMT)
• > athena
By default, it looks for jobOptions.py
• > athena myJobOptions.py
Athena will use myJobOptions.py for input
configuration
INFSO-RI-508833
Ganga
Enabling Grids for E-sciencE
•
Ganga (Gaudi / Athena and Grid Alliance) is an
interface to the Grid that is being developed jointly by
Atlas and LHCb.
•
It is optimised for the configuration and management
of jobs that use the Gaudi / Athena Framework which
is common to the both of these experiments.
INFSO-RI-508833
Ganga
Enabling Grids for E-sciencE
•
Ganga aims to allow a user to run a job at any site
where the user has authorisation, and in particular at
remote sites to be accessed through the Grid. To this
end, Ganga creates a site-independent job
representation, based on information supplied by the
user, and takes care of any file transfers necessary
between the user's machine and the remote site.
•
In the current Ganga model, each job entails running a
single executable. This may correspond to a single
binary, for example to perform event reconstruction,
or may be a script that runs in sequence any number
of binaries, for example event generation followed by
detector simulation followed by event reconstruction.
INFSO-RI-508833
Ganga
Enabling Grids for E-sciencE
• To run from account on lxplus, or over afs, perform
setup with:
> source /afs/cern.ch/sw/ganga/install/etc/setup.sh
• After setup, the Ganga Graphical User Interface (GUI) is
started by giving the command:
> ganga
INFSO-RI-508833
Ganga GUI
Enabling Grids for E-sciencE
INFSO-RI-508833
Ganga GUI
Enabling Grids for E-sciencE
•
The Ganga Graphical User Interface (GUI), as shown
at startup, has the following features:
–
–
–
–
Panel for browsing job and directory trees
Drop-down menus and toolbar
Panel for job configuration and monitoring
Embedded python shell
INFSO-RI-508833
Ganga GUI (Job tree)
Enabling Grids for E-sciencE
INFSO-RI-508833
Ganga
Enabling Grids for E-sciencE
• To run from account on lxplus, or over afs, perform
setup with:
> source /afs/cern.ch/sw/ganga/install/etc/setup.sh
• After setup, the Ganga Graphical User Interface (GUI) is
started by giving the command:
> ganga
INFSO-RI-508833
Ganga GUI (File tree)
Enabling Grids for E-sciencE
INFSO-RI-508833
Ganga: examples
Enabling Grids for E-sciencE
Let’s try to perform practical examples
• Run /bin/hostname
• Run /bin/hostname with output to the file
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
• The job-creation window
Batch system to submit: Local, LSF, EDG/LCG, PBS,
Glite, DIRAC.
Applications are divided into two classes: Generic and
known applications, for example Athena.
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Submitting
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Submitting
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Submitting
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: outputs
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: outputs
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: outputs
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job Creation
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Submitting
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: output
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: output
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: NewJob3.output
Enabling Grids for E-sciencE
INFSO-RI-508833
Running Athena
Enabling Grids for E-sciencE
• Ganga uses Athena application handler to run Athena
within the Ganga framework.
• The handler benefits from Ganga's job submission and
job monitoring capabilities
• The handler has the following features:
– AthASK is used to hide away Atlas software configuration
complexities. The user simply provides AthASK commands
needed to run Athena job from within the GUI.
– User packages (i.e. binary builds, source code) can be specified
and these directory trees will be compressed and transported
(together with the rest of the job) to the worker node ready to be
used by the submitted job.
– Integrated with AMI browser. Job option files can be updated
with selected files from AMI using POOL.
– Integrated with JOE. Job option files can be edited using JOE
before submission.
INFSO-RI-508833
Athena Startup Kit (ASK)
Enabling Grids for E-sciencE
• The Athena Startup Kit (ASK) integrates the ATHENA
framework, and the software based on it, with the
official releases and the various supporting tools in
order to provide an easier, more consistent, and more
robust user interface to Atlas offline software.
• ASK operates completely in user-space: it performs
only those tasks that are otherwise left to the end-user
by the "standard" Atlas tools.
INFSO-RI-508833
New Job for Athena
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job for Athena
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job for Athena
Enabling Grids for E-sciencE
INFSO-RI-508833
New Job for Athena
Enabling Grids for E-sciencE
•In case of local instalation
INFSO-RI-508833
New Job for Athena
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Submitting
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Run
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: output
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: output
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: output
Enabling Grids for E-sciencE
INFSO-RI-508833
Job Finished: output
Enabling Grids for E-sciencE
INFSO-RI-508833
LHCb prototype: GANGA
Enabling Grids for E-sciencE
Gaudi/Athena aNd Grid Alliance
• Framework for job creatingsubmitting-monitoring
• Was also chosen as a
prototype component by
the Atlas experiment
GUI
• Can be used with different
physical applications and
different submission backends
Experiment
Book-keeping
DB
Submitting jobs
Monitoring
Retrieving results
GANGA
Collective
&
Resource
Grid
Services
JobOptions
Algorithms
GAUDI Program
GAUDI – LHCb analysis framework
INFSO-RI-508833