Transcript A GANGA tutorial - TR

A GANGA tutorial
Professor Roger W.L. Jones
Lancaster University
Introduction







This tutorial gives a short introduction to using the Grid with
GANGA
GANGA = user front end
GANGA is used by several communities, and by ATLAS and
LHCb - may be useful for your other local users
You will submit some simple tasks to the Grid with GANGA
You will see a demonstration of running ATLAS software on
the Grid using GANGA
Those with ATLAS softare set up can try this
This tutorial is based on the material here:
https://twiki.cern.ch/twiki/bin/view/Atlas/GangaTutorial44
I invite you to try all the tutorial material offline. The tutorial normally runs over 2
days, so there is a lot in it, but it is well documented
Ganga
3
Grid Infrastructure
WLCG based on
EGEE and OSG
Grid infrastructure.
NDGF
EGEE
OSG
GridPP
WLCG
4
Applications and Back ends

Running of a particular Application on a given Backend is enabled by
implementing an appropriate adapter component or Runtime Handler
 Can often use same Runtime Handler for several Backend: less coding
LHCb
Experiment neutral
Gauss/Boole/Brunel/DaVinci
(Simulation/Digitisation/
Reconstruction/Analysis)
Local
PBS
Executable
ATLAS
AthenaMC
Athena
(Production)
(Simulation/Digitisation/
Reconstruction/Analysis)
LSF
OSG
PANDA
LHCb WMS
US-ATLAS WMS
Implemented
Work in progress
In test/works
Ganga - User Interface
Ganga GUI
Scriptor
Job details
Logical
Folders
Job Monitoring
Job builder
Log window
There is also a scripting interface (like pAthena)
I will use the line mode……
Step 1 - Grid Certificate into
LXPLUS (I)









To submit Grid jobs you need a Grid certificate
We will be using the CERN LXPLUS User Interface so your certificate must be installed on
LXPLUS
Go to the browser of the machine into which you retrieved your Grid Certificate
Find the “Certificates” dialogue box - often in “Preferences”, then “Advanced”
Click on “Export Certificate” or “Export Key Pair”
Choose a location on the local machine and click “OK”
Open a terminal window
Go to the location of the file you just exported, which will end with .pfx or .p12
Copy the file to your LXPLUS account:

scp myfile.p12 [email protected]:/afs/cern.ch/user/<letter>/<username>/.

Log into your lxplus account……
Step 1 - Grid Certificate into
LXPLUS (II)







mkdir .globus
mv myfile.p12 .globus/.
cd .globus
(unless you have one already)
openssl pkcs12 -in myfile.p12 -clcerts -nokeys -out
usercert.pem
openssl pkcs12 -in myfile.p12 -nocerts -out
userkey.pem
chmod 400 userkey.pem
chmod 444 usercert.pem
Step 2 - Make an executable file
and test it



cd $HOME
Start up your favourite text editor
Copy in these lines and save the file (call it myscript.sh)
#!/bin/sh
echo 'myscript.sh running...’
echo "----------------------”

/bin/hostname
Test it: source
myscript.sh
echo "----------------------”
env
Step 3: Start up GANGA



source /afs/cern.ch/sw/ganga/etc/setup-atlas.sh
ganga
Then enter your Grid Certificate password
Step 4: Set up and run a LOCAL
job





j=Job()
j.application=Executable()
j.application.exe=File('$HOME/myscript.sh')
j.backend=Local()
j.submit()
Step 5: monitoring



To see all your jobs: jobs
To the status of a single job:
jobs(number).status()
To see the details of a single job:
jobs(number)
Step 6: collect your results

Exit ganga: CTRL+D


cd $HOME/gangadir/Local/number/output
Open the file stdout with your favourite editor. This
should show the output from your script.
You can look at the files from within GANGA as well:
!less $j.outputdir/stdout
(note that ‘!’ means ‘execute this command in the shell, not
GANGA)


Step 7: Copy the job and switch
to LCG mode




Start up GANGA again
jobs(number).copy()
jobs(number+1).backend=LCG()
jobs(number+1).submit()



Do your monitoring, etc again
When the job status is “completed”, check the output file again
In general this is how you should work; set up and test a LOCAL job,
then copy and switch to LCG only when it is working
You can see where in the world the job ran by typing jobs(number) and
looking in the back-end section of the report (under ‘CE’)

Step 8: Try the GANGA graphical
interface





The GANGA graphical interface (GUI) can do everything that the
command line can do, but allows you to point and click
You can only use it if x-forwarding is set up on your terminal; to test
this type xclock; you should see a clock on the screen.
To open the GUI, quit GANGA and re-start but with an additonal -gui option after ganga
ganga --gui
Can you see your two jobs?
Can you submit a new job through the GUI?
Help on GANGA

Several Ganga-related mailing lists have been set up, as detailed below. Most
of these are managed within the framework of the CERN Simba ListBox
Service, so that you can subscribe using the Simba web interface, and to
access the archives you may need to give your username and password for
the CERN Mail Server. The ATLAS HyperNews list is an exception, and to
subscribe to this, or access the forum, you need to have registered using the
ATLAS HyperNews New Membership page.
General information on Ganga project

[email protected]
List for Ganga developers

[email protected]
ATLAS HyperNews forum for Ganga user support

[email protected]
What’s next

A demonstration of how to run the ATLAS software
framework in GANGA
An Athena Demonstration

This assumes you are in the ATLAS VO, your cmt area set
up and have checked out the examples into a test area…..
source cmthome/setup.sh -tag=13.0.30,32
cd
$TestArea/PhysicsAnalysis/AnalysisCommon/UserAnalysis/
cmt
cmt config
source /afs/cern.ch/sw/ganga/install/etc/setup-atlas.csh
Ganga
#This is a temporary fix because the CERN srm was renamed
without warning!!!!!*
config['Athena']['LCGOutputLocation']='srm://srmatlas.cern.ch/castor/cern.ch/grid/atlas/scratch/Roger.Jones/
ganga'
#done - at least you see how to change config options - you can
#also edit your .gangarc file to do the same thing…..
j = Job()
j.name='your_name'
An Athena Demonstration
j.application=Athena()
j.application.prepare(athena_compile=False)
j.application.option_file='/afs/cern.ch/user/r/rjones/testarea/AtlasOffline13.0.30/PhysicsAnalysis/AnalysisCommon/UserAnalysis/run/Analy
sisSkeleton_topOptions.py'
j.application.max_events='100'
j.inputdata=DQ2Dataset()
j.inputdata.dataset="fdr08_run1.0003070.StreamMuon.merge.AOD.o1_r12
_t1"
j.outputdata=ATLASOutputDataset()
j.outputdata.outputdata=['AnalysisSkeleton.aan.root']
j.backend=LCG()
#Just in case you need to force a job to a specific site…..
#j.backend.requirements.sites= ['LANCS']
j.submit()
An Athena Demonstration
To monitor jobs:
jobs
To execute as a python script:
execfile('/afs/cern.ch/user/r/rjones/testarea/myscript.py')
When finished:
jobs(jobid).outputdata.retrieve()