Transcript Distributed Geant4 simulations on NorduGrid

Distributed Geant4 simulations
on NorduGrid
Andi Hektor
[email protected]
NICPB in Tallinn
(CERN, Estonian Grid)
2nd Geant4 Workshop at HIP
June 6-7, 2005
Outline





The problem: mobile gamma-ray spectrometry
The detector system and detection limits
Geant4 model and some examples
Distributed MC simulations on the Grid
Some results, conclusions and questions
In the frames of the project:
REAL TIME RADIATION SURVEILLANCE
EQUIPMENT FOR THE UNMANNED AERIAL
VEHICLE RANGER
Funded by: SCIENTIFIC
ADVISORY BOARD FOR DEFENCE
A. Hektor, K. Kurvinen, R. Pöllänen and P. Smolander
Radiation and Nuclear Safety Authority (STUK)
M. Kettunen
Finnish Defence Forces Technical Research Centre
J. Lyytinen
Helsinki University of Technology, Laboratory of
Lightweight Structures
Mobile gamma-ray detector
Tracking of a release plume
• Time critical, decision aid for initial response, radiation
safety of the air crew
Fallout mapping
• After the initial emergency, long term risk management
Locating of a point source
• Lost sources, smuggling and terrorism
Unmanned aerial vehicle (UAV) is
an excellent choice!
Unmanned aerial vehicle








Ranger Tactical UAV
Operated by FDF
Range: 150 km
Endurance: 5 hrs
Payload: 40 kg
Catapult take-off
Landing on skids
Fully autonomous or
remotely piloted
Geometry of the detector system
Light weight structure
• Airworthiness above all
• Vibration attenuation
• Temperature control
• Modularity
• Easy installation
Installation of the detector
The gamma-ray spectrometer
Physics in a gamma-ray
scintillator

Photoelectric absorption

Compton scattering

Pair production

Multiple scattering

Electron ionisation

Bremsstrahlung

Positron annihilation
Source
NaI(Tl)
Detection limits of the detector
3D
2D
1D
Different type of sources, long distances between
the source and detector, high speed of the
detector, energy resolution
Constructing of our detector model
in Geant4
Ideas for physics
and input data
CAD model
Physics List
Geometry
Input parameters
Kernel
Analysis
Visualization
Histograms
Analysis
Visualization files
Tracks
Kernel logs
Some examples of our MC
simulations in Geant4 (1/4)
Example 1
Energy of the gammas: 662 keV (Cs-137)
Number of the gammas: 15
Example 2
Some examples of the MC
simulations in Geant4 (2/4)
Example 2
Energy of the gammas: 662 keV (Cs-137)
Number of the gammas: 15
Some examples of the MC
simulations in Geant4 (3/4)
Example 4
Energy of the gammas: 662 keV (Cs-137)
Number of the gammas: 15
Some examples of the MC
simulations in Geant4 (4/4)
Example 4
Energy of the gammas: 662 keV (Cs-137)
Number of the gammas: 15
Distributed MC simulations


Parallel comutings on the
different comuters
Output
Input

The Grid
Cert. Authority (PKI)
Special Devices
Information System
Users

Data Storage Elements
Computing Resources
NorduGrid
Estonian Grid, Jan 2004
First “PC-cluster”
GIIS
Certification Authority
Phone of the local Registration Authority
Management of the “gridified”
Gean4 simulations
Generator of Grid jobs (simple Perl script)
Geant4 macro files
Resource Specification files
The Grid
Management Politics
Simple job manager
(Python script + MySQL)
ROOT data files
Data collector (simple Perl script)
Compiled ROOT data file
Data analysis tools (ROOT scripts)
Logs
Our Grid jobs

Compiled binary of the Geant4 code (3.7 MB):
Linux, glibc 2.6.9

Geant4 macro file with input parameters:
/gun/energy 662 keV
/run/beamOn 500000000, etc


In total 640 Grid jobs were submitted
Computation time in total: 417 CPU days
Statistics of the Grid jobs
 640
jobs and 2.6 GB of data
 Computation time in total:
417 CPU days (on 2.4 GHz
and 3.06 GHz Intel P4)
 17 failed jobs (2.6%) during
the computations and 16
post-processing errors
(2.5%)
3%
3%
0%
Correct jobs
During comp
Post-proc
Other
94%
NorduGrid middleware is
surprisingly stable!
Some conclusions and open
questions




It is easy to “gridify” Geant4 applications
NorduGrid middleware is stable enough
Is it possible to add “Grid functionality” to the Geant4
package?
Other middleware packages: LCG2, gLite, UNICORE
etc
Thank you and…







STUK and HIP in Helsinki
SCIENTIFIC ADVISORY BOARD FOR DEFENCE IN
FINLAND for funding the initial work
Other members of technical coordination group of the
Estonian Grid: Mario Kadastik (NICPB), Lauri Anton, Hardi
Teder (EENet), Konstantin Skaburskas (UT)
NorduGrid core developers for technical support
Estonian Science Foundation: the grants no. 5135, 5935,
5316
EU 6th FP: the grant EC MC MERG-CT-2003-503626
Ministry of Education and Research of Estonia: the support
of the technical meetings of the Estonian Grid