Transcript LCG Simulation Project

Simulation Project Overview
Gabriele Cosmo, CERN/PH-SFT
Simulation Project Leader
http://lcgapp.cern.ch/project/simu
LHCC Comprehensive Review of LCG
November 23, 2004
Simulation Project
Simulation framework
Witold
 Interface to multiple simulation engines (Geant4, Fluka) and
Pokorski
geometry models exchange

Geant4 team participating
John
 Aligned with and responding to needs from LHC experiments,
Apostolakis
physics validation, simulation framework

Fluka team participating
Alfredo
 Framework integration, physics validation
Ferrari

Simulation physics validation subproject
Alberto
 Assess adequacy of simulation and physics environment for LHC
Ribon
and provide the feedback to drive needed improvements

Generator services subproject
Paolo
 Generator librarian; common event files; validation/test suite;
Bartalini
development when needed (HEPMC, etc.)

LHCC Review , Nov 2004
Slide 2
Gabriele Cosmo, CERN/PH-SFT
Project Organization
Geant4
Project
Fluka
Project
Experiment
MC4LHC
Validation
Simulation Project Leader
Subprojects
Framework
Work packages
WP
WP
WP
Geant4
WP
WP
WP
LHCC Review , Nov 2004
Fluka
integration
WP
WP
Slide 3
Physics
Validation
Shower
Param
WP
WP
WP
Gabriele Cosmo, CERN/PH-SFT
Generator
Services
WP
WP
Simulation Framework – General goals




Support the physics validation activities and test beam simulations
Provide flexible infrastructure for efficient development and for usage of detector
simulation applications
 Provide means for interchanging geometry descriptions between different
applications
 Provide flexible, LCG-oriented interfaces to simulation toolkits
Agreed approach: set up a simulation infrastructure based on Geant4 and Fluka via
Flugg (which uses the Geant4 geometry)
 Existing Geant4 benchmarks/simulation-validation setups can be run with
Fluka with minimum effort
Idea of creating a generic framework for Geant4 and Fluka for full detector
simulation setups abandoned due to lack of strong interest from the experiments
(except for ALICE which has its own solution based on VMC)
 Interest in having a common tool to be applied for specific physics validation
studies
 Geant4/Fluka physics validation will be treated on a case by case basis
LHCC Review , Nov 2004
Slide 4
Gabriele Cosmo, CERN/PH-SFT
Simulation Framework – Status & plans


Complete Geant4+Fluka+Flugg setups to meet physics validation needs

First example of usage for test-beam validation studies provided (pixel detector
simulation)

New use-case examples to be provided, together with proper documentation and
guidelines. Expected removal of PEMF pre-processing stage in Fluka

Extensions to Flugg to meet the best achievable level of automation
Establish a persistent exchange format for geometry descriptions: GDML

Development motivated by several use-cases:


Status after recent developments:



Further extend the schema to support more complex geometries (ATLAS and CMS)
Short/Medium term:


Schema complete enough to describe realistic geometries like the LHCb detector
The Geant4 reader/writer has been developed and tested on realistic (LHCb) geometries
Immediate plan:


Debugging, visualization, XML-based, readability, low-overhead geometry exchange, …
Implement reader/writer for Root
Developed concrete milestones reflecting the program above

The program described makes maximum use of existing work

Manpower currently assigned to the project is just 0.5 FTE !
LHCC Review , Nov 2004
Slide 5
Gabriele Cosmo, CERN/PH-SFT
Simulation Framework – Longer Term




Increasing interest from different experiments in Python-based frameworks
(because of large flexibility)
LCG tools (LCGDict/PyLCGDict) provide excellent starting point for
development/study of ‘pythonized’ simulation applications
 First tests (creating LCGDict for Geant4 and running Geant4 via
PyLCGDict) look very promising
Geant4 expressed interest in the Python technology
 Require revision of public interfaces and classes exposure safety
 LCG tools (Pool/Root) could also be applied as persistency mechanism
Simulation environment
 Provide a working example of Python based (Geant4) simulation
application using LCGDict and PyLCGDict
 Investigate the possibility of using LCG persistency framework also for
geometry descriptions
LHCC Review , Nov 2004
Slide 6
Gabriele Cosmo, CERN/PH-SFT
Geant4 in production

Three LHC experiments (ATLAS, CMS, LHCb) now using it successfully in production

OSCAR (CMS), Gauss (LHCb) and ATLAS’s Geant4-based simulation programs are the
production tools


Substantial productions (numbers from Oct 20th Application Area Meeting presentations)




Rate decreasing from 1/10K events (5.2, CMS) to 1 per Million events (6.1, LHCb)
 G4 team addressed issues found in test productions
The Geant4 LCG/SI sub-project and the Geant4 Collaboration

LCG/SI/G4 responsible for CERN/LHC participation in Geant4

Work plan integrated with overall Geant4 plan


ATLAS DC2 (summer 2004) produced 12M events
Oscar (CMS) : 35 M pp interaction events, and first 100 Pb-Pb events
Gauss (LHCb): Over 200 M events simulated
In production use demonstrated low crash rate (and decreasing with new releases)


OSCAR and Gauss have replaced G3 based simulation
Geometry and tracking in field, Physics: hadronic and electromagnetic, testing and release,
coordination
Collaborating closely on validation, infrastructure and robustness

With Physics Valid. on validation, and with Framework on geometry exchange

Worked on shared infrastructure (testing, portal) with SPI: bonsai, Savannah

With experiment simulation and physics teams on robustness, integration and validation
LHCC Review , Nov 2004
Slide 7
Gabriele Cosmo, CERN/PH-SFT
Geant4: 2004 goals








Feb 2004 – Savannah/SPI prototype portal for problem reporting system for Geant4
 Prototype and assessment delivered; under evaluation in the Geant4 Collaboration
Mar 2004 - Release 6.1 (Contributions in several areas)
 focused on improving production usage in LCG experiments
Jun 2004 – Scheduled release 6.2 (Contributions)
 focused on better use of computing resources, including performance and memory use,
and refinements to specific physics models, persistency and windows support
Sep 2004 - Development release
 included additional geometry volume registration, refinements to physics models
 supported CLHEP 1.9, and still compatible with CLHEP 1.8
Oct 2004 - First consolidated acceptance suite for LHC applications

Suite of simplified test-beam setups created, and being deployed
Dec 2004 - Release 7.0 (Contributions)

Release 7.0 contributions focus on improvement of physics models and additional
geometry functionality
Dec 2004 - Prototypes & Process Improvements

prototype 3D string fragmentation; ensure maintenance and improve examples, system
tests and physics lists
Dec 2004 - Geant4 validation in LHC production (added May 2004)

documenting results, response to feedback, status of production use of Geant4
LHCC Review , Nov 2004
Slide 8
Gabriele Cosmo, CERN/PH-SFT
Geant4: some highlights


Geant4 reliability in production: crash rate low and decreasing

CMS (Geant4 5.2, 1 crash per 10,000 events)

ATLAS (Geant4 6.0 patch1, ~1 crash per 1 Million events)

LHCb (Geant4 6.1, ~1-2 crashes per 1 Million events)
Support and maintenance

Addressing issues found in LHC experiment production





Code improvements to help identify problem conditions

In hadronics and geometry (Geant4, releases 6.0 & 6.1)
Creation of ‘statistical testing’ suite

Automated physics comparisons in simple test-beam-like setups

Being deployed for validation of release 7.0 (December 2004)
 Requires significant computer resources
New and improved physics models

And improvements in EM & hadronics (Geant4, releases 6.1 & 6.2)
Refinements & more functionality in kernel

E.g. enabling experiments to easily construct detectors


providing high job ‘robustness’ (less than ~1 per mille job failures in 6.1, 6.2)
Reflections, divisions, …
New fast shower capability (a-la GFLASH)

Integrating efforts in LHC experiments into Geant4 toolkit

New addition, just scheduled for Geant4, release 7.0
LHCC Review , Nov 2004
Slide 9
Gabriele Cosmo, CERN/PH-SFT
Geant4: potential future goals

Provide continued support and maintenance

Address queries on physics, geometry, tracking issues

Solve problems and respond to other issues / reports
Sustain existing physics use cases and enable emerging LHC-related uses
Enhance physics

Address requirements for sub-1% EM stability and precision

Address precision needs for combined calorimetry in full detector simul. / LHC exp.

Address issues of pre-calibration using Monte Carlo (ATLAS)

Further enable use of radiation studies, addressing needs for data and physics lists
Improve and extend ‘automated’ release validation

Extend and refine comparisons







Within available computing resources
Address requirements for geometry and improved persistency

Improvements to Boolean solids, to address robustness issues

Enable exchange format (GDML) and ‘direct’ persistency (POOL / ROOT) for Geant4
Address new requirements for radiation applications

Improve ‘standard’ tallying and extend with additional observables
Continue to improve CPU performance

Address hot spots identified in 2004 and bring new tools into use
LHCC Review , Nov 2004
Slide 10
Gabriele Cosmo, CERN/PH-SFT
Geant4: manpower in 2004




Effort currently
Work Area
Staff/Assoc/Fell
 9.0 FTE from Staff/ Associates
/Fellows
Coordination
1.00
 ~1.0 visitors
Hadronics
3.00
Expertise and effort are required to
address ‘expert’ inquiries
Geometry / Biasing
2.75
 Need to maintain expertise
EM Physics
0.50
Balance between support/maintenance
Testing and Soft Man
1.75
and development
 Increased effort on support and
Total
9.00
maintenance
 New requirements necessitate
refinements & developments.
Continuity of effort depends
Move of 0.25 FTEs to simulation project leadership
 On the expected continued
pushed some geometry deliverables to 2005
availability of associates &
fellows.
LHCC Review , Nov 2004
Slide 11
Gabriele Cosmo, CERN/PH-SFT
Fluka Integration


Fluka is an official joint CERN-INFN project since January 2004
 Fluka development is not an LCG activity
Participation involves
 Integration of Fluka as a simulation engine in the simulation
framework with FLUGG
 Physics validation



Working with the physics validation subproject – simple
benchmarks, test beam (manpower issues !)
Activity is led by Alfredo Ferrari
Fluka source code public at CERN by end of 2004
LHCC Review , Nov 2004
Slide 12
Gabriele Cosmo, CERN/PH-SFT
Fluka Integration
Current developments

Physics

Heavy ions transport and interactions




DPMJET-III interface
Electromagnetic Dissociation
Special effects in heavy ion Coulomb interactions
Development of QMD models
done
done
in progress
in progress
Fragmentation

Code structure and user interface (pre-condition for the full release)
in progress

Integration of PEMF functionalities in the run time code

Name-oriented advanced input interface

General clean-up

Publication of code documentation as CERN yellow report
Man power issues

Physics developments: covered by A.Ferrari and non-CERN collaborators OK

“Complex benchmarks” (test-beams): expertise support from FLUKA team, manpower
must come from LCG ?

Code structure and user interface (release oriented) : CERN-AB manpower absorbed by
urgent LHC tasks. CERN-PH ending soon
critical

Struggling to keep the December-January
deadline for the public β release of the full code
LHCC Review , Nov 2004
Slide 13
Gabriele Cosmo, CERN/PH-SFT
Physics Validation



Validation based mainly on
 Comparisons with LHC detector test beam data
 “Simple benchmarks”: thin targets, simple geometries
Coordinates a lot of work being done in the experiments, Geant4, Fluka
 Foster cooperation, coherence, completeness
Output of the project
 Certification that simulation packages are OK for LHC physics





Understanding strengths/weaknesses/uncertainties of Geant4, Fluka
Contributions to systematic errors of measurements
Recommended optimized physics lists
Simulation benchmark suite for acceptance and performance
monitoring
Final report summarizing the work
LHCC Review , Nov 2004
Slide 14
Gabriele Cosmo, CERN/PH-SFT
Physics Validation – Status






Geant4 electromagnetic physics validated at percent level
Simulation physics requirements revisited
First round of hadronic physics validation has been completed, with good results
 For the observables, in the case of the simple benchmarks (pixels, neutron
double differential, pion absorption) there is a reasonable agreement
between data and both Geant4 and Fluka
 For the calorimeter test-beams, Geant4 describes well the pion energy
resolution, σ/E, and the ratio e/
 The shape of hadronic showers needs some improvement
LCG notes: 1. F.Gianotti et al., CERN-LCG-APP-2004-02
2. A.Ribon, CERN-LCG-APP-2004-09
3. F.Gianotti et al., CERN-LCG-APP-2004-10
4. W.Pokorski, to be released very soon
Monthly meetings presenting and coordinating experiment and project work
Information, results gathering on web page
LHCC Review , Nov 2004
Slide 15
Gabriele Cosmo, CERN/PH-SFT
Physics Validation – Ongoing and Future Work






Validate precision of Geant4 electromagnetic physics at the permil level
More detailed studies of hadronic shower profiles, both at the simulation and
experimental (test-beam data) level.
Evaluate the possibility to study another simple benchmark, relevant to LHC,
to validate both Geant4 and Fluka.
 Man power could be a problem here!
Complete validation for Fluka in the calorimeter test-beam validations (as it
has been done for Geant4)
 The adoption of FLUGG and strong interaction with the Simulation
Framework project is required here
Geant4 studies of background radiation in the LHC caverns in progress
 Will be soon compared with Fluka.
Man-power:
 People from the experiments are busy with the new test-beam data
 From LCG: M. Gallas, W. Pokorski, A. Ribon



all of them involved in other activities!
First version of simulation test and benchmark suite delayed to end 2004
Physics validation document delayed to June 2005
LHCC Review , Nov 2004
Slide 16
Gabriele Cosmo, CERN/PH-SFT
Physics Validation Manpower





Witek Pokorski ~ 0.4 FTE
 simple benchmarks and generic framework (general infrastructure
supporting test beam physics validation)
Alberto Ribon ~ 0.5 FTE

project leader, test beam calorimeters with Fluka, simple benchmarks
Giuseppe Daquino (Geant4 team) ~ 0.75 FTE
 radiation background simulation with Geant4 (biasing), background
studies in the LHCb detector environment
Manuel Gallas Terreira ~ 0.75 FTE
 ATLAS combined test-beam simulation
Total ~ 2.4 FTEs
 ~ 1.5 FTEs currently shifted to experiments-specific validation activities
 ~ 0.9 FTEs dedicated to simple benchmarks, test beam calorimeters and
coordination !
LHCC Review , Nov 2004
Slide 17
Gabriele Cosmo, CERN/PH-SFT
Hadronic interactions in ATLAS pixel test-beam
A.Ribon LCG-APP-2004-09
LHCC Review , Nov 2004
Slide 18
Gabriele Cosmo, CERN/PH-SFT
Energy resolution of pions
 

E simulation
LHCC Review , Nov 2004
Slide 19
 

E
test beam
Gabriele Cosmo, CERN/PH-SFT
e/π ratio
 simulation  testbeam
e
LHCC Review , Nov 2004
e
Slide 20
Gabriele Cosmo, CERN/PH-SFT
CMS longitudinal shower profile in HCAL for 100 GeV pions
LHCC Review , Nov 2004
Slide 21
Gabriele Cosmo, CERN/PH-SFT
Generator Services



Goal: guarantee the generator support for LHC

WP1 - Generator services library (GENSER)

WP2 - Tuning and validation of event generators

WP3 - Common generator event files, event database (MCDB)

WP4 - Event format, interfaces and particle services
Oversight of MC4LHC

Project Leader: P.Bartalini (Florida). Most of the resources from LCG Russia.
Generator library – GENSER – progressing well

First “production-quality” version 1.0.0 expected for December

ATLAS uses GENSER in production, LHCb and CMS currently validating

Addresses LHC experimentalists and theorists both at CERN and in external laboratories

Supports many MC4LHC-mandated generators, including all first-priority ones



List as of mid-September - GENSER 0_2_0 release: HERWIG (P.Richardson), PYTHIA
(T.Sjöstrand), HIJING (X.N. Wang), ISAJET (F.E. Paige), LHAPDF (M.Whalley), ALPGEN
(M.Mangano), COMPHEP (A.Sherstnev), EVTGENLHC (P. Robbe), GLAUBER
(V.Uzhinsky), FROZEN (PHOTOS, PDFLIB), 10 generators, different versions
In future, to come: CASCADE (H.Jung), DPMJET, MC@NLO, GRACE, MADGRAPH,
NEXUS, PHOJET (Z.Was), SFM, TAUOLA, HERWIG++, PYTHIA 7, SHERPA (F.Krauss)
Internal MC4LHC review in March 2004


Definition of a release policy and roles for the GENSER librarian (A. Pfeiffer, PH/SFT)
Involvement of A.Ribon (LCG) to help the project leader as ‘liaison’ person between MC
authors and experiments.
LHCC Review , Nov 2004
Slide 22
Gabriele Cosmo, CERN/PH-SFT
Generator Services - 2

Tuning and validation of event generators

Advanced proposal for an event generator validation framework


Basic sanity check in standalone way in GENSER sub packages
Physics validation through JetWeb (UK collaboration)
Validation of ALPGEN and HIJING in collaboration with groups in Perugia and
Dubna
Common event file production / event database

Event file database MCDB α-version deployed

Core software (MySQL, POOL, CASTOR (RFIO), …) supported by LCG
Software Project Infrastructure

Web Interface with dedicated web server: http://mcdb.cern.ch





Configuration and book-keeping tested at Fermilab and CMS
Identified production centers (Santander, Oviedo)


α version deployed, being extended to support most common browsers
Leveraging existing CERN production infrastructure
Event formats and interfaces

HEPML as proposed meta-data format, based on XML for interfacing matrix
element generators and showering/hadronization generators

HEPMC as interface between generators and MC simulators

Support the new OO MCs (ThePEG, PYTHIA 7, HERWIG++, SHERPA)
LHCC Review , Nov 2004
Slide 23
Gabriele Cosmo, CERN/PH-SFT
Generator Services Oversight/Review


Review of the subproject conducted in March (M. Mangano chair and
MC4LHC chair) with participation from experiment experts and many
leading generator authors
Review report end of April with clear elements
 Project is delivering what the experiments need


Manpower issues to be addressed



GENSER in production in ATLAS, under validation in CMS and LHCb
Stability in project leader: P.Bartalini (now University of Florida)
Librarian positions: A.Pfeiffer LCG librarian for GENSER and defined
release policy
Communication with the generator providers


A.Ribon as ‘liaison’ person between MC authors and experiments
Primary role of Russian collaborators in the “assembly” process of
features to be included in sub-packages
LHCC Review , Nov 2004
Slide 24
Gabriele Cosmo, CERN/PH-SFT
Generator Services 2004 (and beyond) Milestones










Jan 2004 – Proposal for MCDB deployment in the LCG environment
Feb 2004 – LHAPDF generator included in generator library
Mar 2004 – Agreed format for event-level generator files
Apr 2004 – COMPHEP, ALPGEN and EVTGEN in GENSER
Jun 2004 – Proposal for generator event production environment
Jul 2004 – Beta version of MCDB in production
Jul 2004 – Proposal for an event generator validation framework
Sep 2004 – Agreement on parton-level event generator file format
Dec 2004 – Generator production framework final release
Plan extends through 2005-2006 to reach complete, production versions of GENSER,
MCDB, validation framework, etc. (in blue those not yet verified)

Mar 2005 - First C++ Monte Carlo fully integrated in GENSER

Jun 2005 - Generator level production framework "beta version 0_1_0"

Jun 2005 - First test of ThePEG and Evtgenlhc integration in Herwig++

Sep 2005 - Production centre integrated in the grid-middleware

Sep 2005 - Integration of GENSER in JetWeb

Dec 2005 - Generator level production framework "release version 1_0_0"

Mar 2006 - MCDB Integration, experiment specific APIs and management of
large files

Jun 2006 - Generator level validation framework beta version
LHCC Review , Nov 2004
Slide 25
Gabriele Cosmo, CERN/PH-SFT
Generator Services – perspectives on required resources




Critical situation of resources for future developments in GENSER
 Particularly concerning the new MC generators
Little resources available for the validation activity
 Requiring MC experts !
Long-term resources for user-support required
 User-support expected to grow !
 IT-expert for maintenance, MC-expert for development
 E.g. - FAQ for GENSER did not start for lack of resources …
See manpower tables for 2004 and estimation for 2005 in appendix
LHCC Review , Nov 2004
Slide 26
Gabriele Cosmo, CERN/PH-SFT
Simulation Project Milestones



























2003/12: Geant4 release 6.0
2003/12: Simulation physics requirements revisited
2003/12: Simulation framework prototype supporting G4 and FLUKA via FLUGG
2003/12: Proposal for MCDB deployment in the LCG environment
2004/2: LHAPDF generator included in generator library
2004/2: First cycle of hadronic physics validation complete
2004/2: SPI-G4 collaborative infrastructure pilot
2004/3: Agreement on formats for event generator common samples
2004/3: Detector description (GDML) proposal to PEB
2004/03: Geant4 6.1 release - production improvements
2004/4: COMPHEP, ALPGEN and EVTGEN generators included in GENSER
2004/5: Review/prioritization of simple benchmarks for simulation physics validation
2004/6: Proposal for generator event production environment
2004/6: Geant4 6.2 release - resource usage refinements
2004/7: Beta version of MCDB in production in the LCG environment
2004/7: Proposal for an event generator validation framework
2004/9: Agreement on parton-level event generator file format
2004/9: Comparison of LHC calorimeters for EM shower development
2004/10: Geant4 geometry volume registration customization
2004/10:First consolidated G4 acceptance suite for LHC applications
2004/12: Generator production framework, final release
2004/12: Geant4 physics model prototype concluded
2004/12: Second iteration of hadronic physics validation complete
2004/12: Simulation test and benchmark suite available
2004/12: Geant4 7.0 release - physics models and geometry
2004/12: Geant4 validation in LHC production
2005/06: Final physics validation document complete
LHCC Review , Nov 2004
Slide 27
Gabriele Cosmo, CERN/PH-SFT
Simulation Project Major Milestones

















2003/12: Simulation physics requirements revisited
2003/12: Simulation framework prototype supporting G4 and FLUKA via FLUGG
2004/2: First cycle of hadronic physics validation complete
2004/2: SPI-G4 collaborative infrastructure pilot
2004/3: Agreement on formats for event generator common samples
2004/5: Review/prioritization of simple benchmarks for simulation physics validation
2004/6: Geant4 6.2 release - resource usage refinements
2004/7: Beta version of MCDB in production in the LCG environment
2004/9: Agreement on parton-level event generator file format
2004/9: Comparison of LHC calorimeters for EM shower development
2004/10:First consolidated G4 acceptance suite for LHC applications
2004/12: Generator production framework, final release
2004/12: Second iteration of hadronic physics validation complete
2004/12: Simulation test and benchmark suite available
2004/12: Geant4 7.0 release - physics models and geometry
2004/12: Geant4 validation in LHC production
2005/06: Final physics validation document complete
LHCC Review , Nov 2004
Slide 28
Gabriele Cosmo, CERN/PH-SFT
Current simulation manpower
Simulation framework
0.50
Geant4
8.65
Infrastructure, performance, management (approx)
2.75
EM / Hadronic physics effort (approx)
3.55
Tracking, biasing, geometry (approx)
2.35
CERN staff (4.15), CERN fell./assoc. (4.25), LCG (0.25)
Physics validation
2.40
Validation specific to experiments
1.50
Generator services
2.35
mgmt. (Florida Univ.), documentation/release (LCG/CERN)
0.60
GENSER dev., Validation, MCDB (LCG Russia)
1.50
Production Framework (LCG Spain)
0.25
FLUKA integration
0.00
Management
0.25
Total
14.05
LHCC Review , Nov 2004
Slide 29
Gabriele Cosmo, CERN/PH-SFT
As of Oct 1
Concluding Remarks

Striking success of Geant4 in the LHC experiments

ATLAS, CMS and LHCb now using it in full production

Crash-rate close to zero !
Strict collaboration with Geant4 team for support and new requirements
Very active program in physics validation delivering results and conclusions

Shift of manpower to experiment-specific validation activities (ATLAS, LHCb)

First cycle of hadronic physics validation completed

New program of work for test-beam validation and simple benchmarks to be
defined, also according to the available manpower in the project
Outstanding progress for generator services

First production release of GENSER in December





Already used in production in ATLAS, under validation in CMS and LHCb
Impressive collection of validated generator packages with common event DB
Manpower issues partially addressed
Simulation framework program scoped to make maximal use of existing software and
meet the reality of minimal available manpower

Prototype setup based on Flugg for physics validation

Extension of GDML and implementation of interfaces for Geant4/Root to import
and export geometry descriptions

Investigations and prototyping on Python-based interfaces
Fluka integration/validation feeling effects of slow progress due to low manpower
(physics validation, Flugg framework)

Expected public source-code release of Fluka soon



LHCC Review , Nov 2004
Slide 30
Gabriele Cosmo, CERN/PH-SFT

Simulation in the LHC experiments

ALICE

LHCC Review , Nov 2004
Courtesy of A.Morsch
Slide 32
Gabriele Cosmo, CERN/PH-SFT
ALICE – AliRoot Framework

Using VMC (Virtual Monte-Carlo)
VMC
User Code
G3
G3 transport
G4
G4 transport
FLUKA
transport
FLUKA




Reconstruction
TGeant3

Currently used in production
Visualisation
TFluka

Implementation completed and interfaced to TGeo

Testing ongoing
TGeant4
 Plans for 2005

Implementation completed
 Validation of

Interfacing to TGeo planned
External Generators integrated into AliRoot

HIJING for underlying event simulation

PYTHIA: jets, heavy flavor




LHCC Review , Nov 2004
Slide 33
Virtual
Geometrical
Modeller
Geometrical
Modeller
(TGeo)
the Geometrical Modeller
(GM) with Geant3 and Fluka
Extension of the interface to Geant4
Validation of the Geant4 interface
Validation of new AliRoot simulation
with GM, including physics
Phasing out Geant3.
Gabriele Cosmo, CERN/PH-SFT
ALICE – Physics Validation

Emphasis on simple (thin target) benchmark tests
 Thin target benchmark tests represent the only direct way of
physics validation
 Calorimeter tests lead in the best case to a multi-parameter tuning
of MC parameters, but not to a better understanding of the
underlying physics
 Benchmark tests started by ALICE and continued within the LCG
simulation project

Test-beam simulation activities ongoing for ITS, TPC, HMPID
 Problem of manpower
 Support from LCG requested, but not obtained
LHCC Review , Nov 2004
Slide 34
Gabriele Cosmo, CERN/PH-SFT
ALICE - Status of Simulation Production
Physics Data Challenge ‘04


Test and validate the ALICE Offline computing model:
 Produce and analyse ~10% of the data sample collected in a
standard data-taking year
 Use the entire (complicated) system: AliEn, AliROOT, LCG,
Proof…
 Dual purpose: test of the software and physics analysis of the
produced data for the Alice PPR
Structure:
 Logically divided in three phases:



Phase 1 - Production of underlying Pb+Pb events with different
centralities (impact parameters) + production of p+p events
Phase 2 - Mixing of signal events with different physics content
into the underlying Pb+Pb events (underlying events are reused
several times)
Phase 3 – Distributed analysis
LHCC Review , Nov 2004
Slide 35
Gabriele Cosmo, CERN/PH-SFT


ATLAS
Courtesy of A.Rimoldi
LHCC Review , Nov 2004
Slide 36
Gabriele Cosmo, CERN/PH-SFT
ATLAS - Status

A Geant4-based simulation suite for the ATLAS experiment is in place (and available to developers) since mid-’03

Data Challenge phase-2 (summer 2004)

Geant4 -based, ~12M events (leap of faith)

Tests of the ATLAS computing model, distributed production

Running right now
Characteristics

Based on home-grown simulation infrastructure (FADS)

Interfaced to the ATLAS common framework (ATHENA)

Persistency based on POOL

Unified detector description scheme as in ATLAS (GeoModel)

Hand-coded geometries being phased out

~5 millions of positioned volumes (~300K volume types)

All detectors described to a very high level of detail

Detector configuration chosen at run time

Detailed field map covering the whole detector

Primary numbers from NOVA/MySQL (now from Oracle)

UI currently provided by Geant4 (moving to python)

Several Physics Lists available at any time (QGSP_GN by default)

Very low tracking cuts for precise physics (20-30m in the calorimeters)

About 400Mbytes at run time

~660secs for Ze+e-, ~770secs for SUSY events (PIV, 2.4Ghz) over the whole rapidity
range ||6

Heavy ions trial successfully done (few full events generated in |eta|<6 or 3.2

LHCC Review , Nov 2004
Slide 37
Gabriele Cosmo, CERN/PH-SFT
ATLAS - Production setup





Physics validation programme helped debugging the sub-detectors in an
independent way
 200 talks in a time span of 3 years
Sub-detector integration took several months (2003)
 Make sure there are no overlaps and check detector layout
 Move to new detector description scheme
 Implement missing bits &pieces (services, dead material etc.)
 Optimize performance
Running long tests since September 03
 Continuous monitoring helps maintaining the program functional
Set up production environment in the meantime
Currently running Geant4 6.2 for development work and CTB
simulation
LHCC Review , Nov 2004
Slide 38
Gabriele Cosmo, CERN/PH-SFT
ATLAS - DC2 production



Slowed down by instability of the production tools
 GRID middleware?
Simulation part completed
 12M events
Exceptional Geant4 performance and robustness
 Only two jobs crashed b/c of Geant4 problems !


NorduGrid sample (3.5M events) completed with no job failure !



log file examination pending …
35K jobs !
1M Ze+e- events without any problem !
Continuing now with further tests (Tier-0 production…) which are not
relevant to Geant4
LHCC Review , Nov 2004
Slide 39
Gabriele Cosmo, CERN/PH-SFT
ATLAS - Future plans




Production for the ATLAS CTB (ongoing)
Continuous production (starting in December) for the physics
community
 Physics Workshop in Rome, May ’05
Commissioning
DC-3
 ATLAS “initial” layout
 “GeoModel-ization” of the LAr calorimeters
 Implementation of missing bits (shielding, support structures…)
 Python
 General refurbishing
LHCC Review , Nov 2004
Slide 40
Gabriele Cosmo, CERN/PH-SFT

CMS
Courtesy of A. De Roeck
LHCC Review , Nov 2004
Slide 41
Gabriele Cosmo, CERN/PH-SFT
CMS - OSCAR




In CMS, OSCAR, the OO simulation program based on the Geant4 toolkit, has
successfully replaced its Fortran/Geant3 predecessor. It has been validated and adopted by
all CMS detector and physics groups. It has proven robust and performant, easily
extensible and configurable
 Complete CMS geometry: 1M geometrical volumes
 35M pp interaction events produced
 Time ~ 200 sec/event at 2.8 GHz CPU (High ET QCD events)
 First 100 PbPb heavy ion events
CMS has now entered sustained-mode production:
 10M physics events/month through the full chain (simulation, digitization, …, DSTs)
OSCAR 3.6.0 released now with G4.6.2 p1
 new validation phase (new field map, new forward detectors, new G4 version, many
improvements…) Deploy for mass production next month
Continuing validation of G4 physics (EM+HAD) and performance optimization
important. CMS participates in this common effort
LHCC Review , Nov 2004
Slide 42
Gabriele Cosmo, CERN/PH-SFT
OSCAR/G4: Performance



CPU time of OSCAR 2.4.5 (with Geant4 5.2 p2) is 2xGEANT3 time, but
 much more sophisticated cut/region scheme
 more conservative cuts than for CMSIM
 more detailed physics in G4
 improvements implemented by G4 team, and by CMS through more
optimized access to/use of the (new) magnetic field.
 we have now a 20-30% better performance of OSCAR 3.6.0 (with G4.6.2
p1) compared to the version 2.4.5. So now we are at 1.5 x GEANT3; effort
is continuing
Memory usage in OSCAR 2.4.5: 220 MB OSCAR vs 100 MB GEANT3
simulation.
 further optimization led to ~110 Mb/event for pp (>500 Mb/event for HI)
Crashes occurred with 2.4.5: about 1/10000 for pp events. Mostly hadronic
physics (baryon decays, -nuclear intertactions)
 latest stress test (800K single particles, 300K full QCD events) showed no
single crash !
Many thanks to G4 for help!
LHCC Review , Nov 2004
Slide 43
Gabriele Cosmo, CERN/PH-SFT
CMS – LCG projects

Other LCG projects
 GENSER being validated, plan to deploy early 2005
 Use FLUKA via FLUGG if PRS groups in CMS
require it
 Validation: participate with new 2004 HCAL and
ECAL testbeam data (beam energies down to 2 GeV,
longitudinal readout)

Future
 Production of additional O(100K) events with OSCAR
for physics TDR
LHCC Review , Nov 2004
Slide 44
Gabriele Cosmo, CERN/PH-SFT
Validation of the Physics in OSCAR
Validation of G4 physics in the context of the LCG study group
So far: Comparisons of hadronic test beam data with models in G4
Also: Comparison of EM physics with test beam data
CERN-LCG-2004-10
Generally
QGSP model
adequate
Studies of energy resolutions, e/ ratios, and shower profiles
LHCC Review , Nov 2004
Slide 45
Gabriele Cosmo, CERN/PH-SFT
Physics Events with OSCAR
SUSY events (LM4 point: leptons,
missing ET)
View of 180 Higgs ZZ event
simulated in CMS Tracker detector
Samples of “standard sets” of events now
automatically produced for each new release
LHCC Review , Nov 2004
Slide 46
Gabriele Cosmo, CERN/PH-SFT
Ultimate test: Heavy-Ion Collisions


CMSIM: chop event in slices of 100 tracks, run them separately
 Needed due to limitations in CMSIM
OSCAR/Geant4 can run full events.
 Timing is good/Memory > 500 Mbyte (2GB memory machines used)
 Have now run 100 events without problems
~ Timing for the
first event with
55K generator tracks
Program
The first CMS PbPb event with OSCAR/G4
CPU
(2.8GHz)
CMSIM
OSCAR
2_4_5
OSCAR
3_4_0
(min)
230
320
180
LHCC Review , Nov 2004
Slide 47
Gabriele Cosmo, CERN/PH-SFT

LHCb
Courtesy of G.Corti
LHCC Review , Nov 2004
Slide 48
Gabriele Cosmo, CERN/PH-SFT
LHCb - Simulation status

Gauss, the “new” Gaudi/Geant4 based simulation application has been put in
production this year

main simulation engine

has replaced Geant3 based simulation for overall simulation to mimic what will
happen in the spectrometer and understand experimental conditions and
performance

integrates two independent phases:
1. generation of proton-proton collisions and decays of b hadrons
2. tracking of particles in the detector and interactions with the material
production of “hits" when particles cross sensitive detectors

makes use of simulation external engines via dedicated interfaces and services

Pythia, EvtGen, HepMC, Geant4
based on LHCb Gaudi framework, EventModel and Detector Description
Digitization performed by Boole a different application
Dedicated background studies also performed with FLUKA



LHCC Review , Nov 2004
Slide 49
Gabriele Cosmo, CERN/PH-SFT
LHCb - Gauss status

Gauss is fully operational
 complete



stable




all detectors simulated
all information needed in later processing provided
low crash rate, reasonable CPU time
good collaboration with Geant4 team to achieve this ( various iteration before
frozen version for DC04 )
validated as replacement of Geant3 simulation with detailed comparisons
under continuous validation with test beam data to tune physics settings


work in different sub-detectors at different paces based on their needs and data
(RICH, ECAL,…)
collaboration with LCG physics validation project
 also to use Geant4 for radiation studies
LHCC Review , Nov 2004
Slide 50
Gabriele Cosmo, CERN/PH-SFT
LHCb - Gauss in production (May – August)
186 M Produced
Events
LCG
paused
LCG in
action
Phase 1
Completed
3-5 106/day
LCG
restarted
Still producing data
Geant4 version 6.1
In each job 3 or 4
sets of 500 events
produced with
Gauss
1.8 106/day
Data produced in ~
60 different sites
LHCC Review , Nov 2004
Slide 51
Gabriele Cosmo, CERN/PH-SFT
LHCb - Simulation developments & plans - 1


Data produced with Gauss in DC04 will be intensively scrutinized in the
coming months by many physicists
 allow to understand, improve the simulation, fix bugs, identify new needs
Ongoing developments
 Validating GENSER as provider of generator libraries



Adopting and validating new versions of Geant4 as they are available



need to be “stable” for use in production
collaborating for LHC version of EvtGen
provide feed back and new requirements to Geant4
try out new relevant features
Physics validation (tuning) with test beam analysis

RICH, ECAL test beams just completed, analysis in progress
LHCC Review , Nov 2004
Slide 52
Gabriele Cosmo, CERN/PH-SFT
LHCb - Simulation developments & plans - 2

Future developments ( in random order )
 Introduce more realism and details in Gauss





Investigate alternative, new paths to those adopted currently in Gauss





understand better both the simulation and test beam data
physics tuning (… or validation)
Further investigation of use of Geant4 for radiation studies
Introduce accelerator contributions to background (halo, beam gas, etc.)
delta rays production
production/tracking cuts per region
investigation of tracking in magnetic field (parameters, regions)
Investigate use of other generators for proton-proton collisions
Follow LCG simulation projects developments
LHCC Review , Nov 2004
Slide 53
Gabriele Cosmo, CERN/PH-SFT

Appendix
Geant4 in LCG/SI, and collaborations

The Geant4 LCG/SI sub-project and the Geant4 Collaboration
 LCG/SI/G4 responsible for CERN/LHC participation in Geant4



Work plan integrated with overall Geant4 plan




Maintaining the focus of CERN/LHC effort on LHC priorities
Leading role in management, development and infrastructure within the
Geant4 Collaboration
Physics: hadronic and electromagnetic
Geometry and tracking in field
Infrastructure, management, coordination
Collaborating with sub-projects & other LCG project, with experiments teams
on validation, robustness and infrastructure
 Collaboration with Physics Validation, experiment detector groups


Collaborated with experiment simulation teams, e.g. on robustness



Some improvements, updates are direct consequence of this feedback
Substantial decrease in job crash rate in past year
Work with the framework team on geometry exchange
Worked on shared infrastructure (testing, portal) with SPI
LHCC Review , Nov 2004
Slide 55
Gabriele Cosmo, CERN/PH-SFT
Geant4-based Simulations in ATLAS, CMS, LHCb

Three LHC experiments (ATLAS, CMS, LHCb) are now using
Geant4 successfully in production
 ATLAS G4 simulation is now the ‘standard’ in ATLAS


OSCAR (G4) has successfully replaced CMSIM (G3) in CMS


35 M pp interaction events, and first 100 Pb-Pb events
GAUSS (G4) has replaced G3 based simulation for 2004
production


DC2 (summer 2004) produced 12M events
Over 200 M events simulated
Reliable in production, with low crash rate (and decreasing with
each release)

Rate decreasing from 1/10K events (5.2, CMS) to 1 per Million
events (6.1, LHCb)
G4 team addressed issues found in test productions

LHCC Review , Nov 2004
Slide 56
Gabriele Cosmo, CERN/PH-SFT
Geant4: sample details of highlights

Establishment of ‘statistical testing’ suite
 Automating comparisons of physics quantities

Simple setups for ‘regression testing’
Simplified, typical HEP detectors without digitization
Complementing INFN and SLAC efforts (comparing to NIST
data) and full LHC ‘test-beam’ comparisons



Identifying problem conditions
 In hadronics by recording initial reaction conditions



and printing them out in case of ‘soft’ error or program crash
problems are reproduced easily, identified and fixed quickly
In geometry by providing a new ‘check’ mode

Giving users information on difficulties with their setup
LHCC Review , Nov 2004
Slide 57
Gabriele Cosmo, CERN/PH-SFT
Geant4: future goals (more detail)
Additional geometry/biasing goals:

Enable use of dual geometries for radiation studies

Improve detection of problems in ‘user’ geometry models

Better identifying issues in large scale productions

Improve field tracking, by taking account (inside a ‘step’) of effect of energy loss on
momentum
Additional potential EM goals:

Enable the refining of multiple scattering for pions and electrons

Improve modeling of transition radiation detectors

and provide specific customization for ATLAS and ALICE XTR detector setups

Enable the use of shower parameterization in sampling calorimeters
Additional potential hadronic goals:

Establish a 'data service' for radiation studies

Establish the use of binary cascade in conjunction with QGS model

Bring CHIPS ‘string’ prototype to fruition

Validate ion reactions for use in HEP
Contribute and enable the Geant4 physics teams to:

Establish service for physics lists in emerging/upcoming LHC use-cases

Establish physics for exotics (ex.: R-hadrons) in collaboration with experiments

Provide additional modeling options for most-sensitive use cases
LHCC Review , Nov 2004
Slide 58
Gabriele Cosmo, CERN/PH-SFT
In-flight Pion Absorption
W.Pokorski - LCGAPP note almost ready
pi+/-
LHCC Review , Nov 2004
Slide 59
Gabriele Cosmo, CERN/PH-SFT
LHC hadronic calorimeter test-beams
F. Gianotti et. al LCG-APP-2004-10
ATLAS HEC test beam
CMS HCAL & ECAL test beam setup
extended
barrel
module
extended barrel
module
η=0.25
LHCC Review , Nov 2004
Crystal
25
Slide 60
Gabriele Cosmo,
CERN/PH-SFT
ATLAS
Tilecal
test beam setup
η=0.65
Generator Services – manpower 2004





Coordination: 0.25 (0.50)
WP1. GENSER – 0.70 (1.00)

Development: 0.50 (0.50) LCG Russia

Maintenance: 0.10 (0.25) LCG Russia

User Support: 0.10 (0.25) LCG Russia
WP2. ThePEG – 0.10 (0.25)

Development: 0.10 (0.25) * Alberto * (documentation)

Maintenance: 0.00 (0.00) Not yet an issue
WP3. MCDB & Simple Production Framework – 1.25 (1.25)

Development: 0.75 (0.75) LCG Russia

Maintenance: 0.00 (0.00) Not yet an issue

User Support: 0.00 (0.00) Not yet an issue

Framework Development: 0.50 (0.50) LCG Spain / CMS

Framework Maintenance: 0.00 (0.00) Not yet an issue

Framework User Support: 0.00 (0.00) Not yet an issue

Production 0.00 (0.00) Spanish CENTRE (to be started 2005)
WP4. Validation – 0.30 (1.00)

Validation (Standalone) 0.30 (0.50) LCG Russia

Validation (Framework) 0.00 (0.50) Collab with JetWeb (NOT STARTED)
LHCC Review , Nov 2004
Slide 61
Gabriele Cosmo, CERN/PH-SFT
Generator Services – manpower 2005





Coordination: 0.25 (0.50)
WP1. GENSER – 1.00 (1.50)

Development: 0.50 (0.50) LCG Russia

Maintenance: 0.25 (0.50) * Andreas *

User Support: 0.25 (0.50) LCG Russia
WP2. ThePEG – 0.25 (0.25)

Development: 0.00 (0.00) Frozen

Maintenance: 0.25 (0.25) * Alberto *
WP3. MCDB & Simple Production Framework – 1.75 (2.25)

Development: 0.50 (0.50) LCG Russia

Maintenance: 0.25 (0.25) LCG Russia

User Support: 0.25 (0.25) LCG Russia

Framework Development: 0.50 (0.50) LCG Spain

Framework Maintenance: 0.25 (0.25) LCG Spain

Framework User Support: 0.00 (0.25) Not sure LCG Spain can guarantee

Production: 0.00 (0.25) LCG Spain (first tests)
WP4. Validation – 0.25 (1.00)

Standalone: 0.25 (0.50) LCG Russia

Framework: 0.00 (0.50) Cannot Start if no resources...
LHCC Review , Nov 2004
Slide 62
Gabriele Cosmo, CERN/PH-SFT

LHCC Review , Nov 2004
Slide 63
ALICE
Gabriele Cosmo, CERN/PH-SFT
Example TFluka Testing:
RICH 5 GeV Pions
Geant3
LHCC Review , Nov 2004
FLUKA
Slide 64
Gabriele Cosmo, CERN/PH-SFT
PDC’04 Phase I
Centrality name
Impact parameter value [fm]
Produced events
Cent1
0-5
20K
Per1
5 - 8.6
“
Per2
8.6 - 11.2
“
Per3
11.2 - 13.2
“
Per4
13.2 - 15
“
Per5
> 15
“
LHCC Review , Nov 2004
Slide 65
Gabriele Cosmo, CERN/PH-SFT
PDC’04 Phase II
Ongoing
Completed
LHCC Review , Nov 2004
Slide 66
Gabriele Cosmo, CERN/PH-SFT
ALICE Physics Data Challenges
Period
(milestone)
Fraction of the
final capacity (%)
06/01-12/01
1%
Physics Objective
pp studies, reconstruction of TPC and ITS
•
•
06/02-12/02
5%
•
•
•
•
01/04-06/04
10%
•
•
•
Test of the final system for reconstruction and analysis
•
01/06-06/06
TBD
Complete chain used for trigger studies
Prototype of the analysis tools
Comparison with parameterised MonteCarlo
Simulated raw data
Refinement of jet studies
Test of new infrastructure and MW
TBD
•
05/05-07/05
First test of the complete chain from simulation to reconstruction
for the PPR
Simple analysis tools
Digits in ROOT format
20%
LHCC Review , Nov 2004
Slide 67
Gabriele Cosmo, CERN/PH-SFT

LHCC Review , Nov 2004
Slide 68
CMS
Gabriele Cosmo, CERN/PH-SFT
OSCAR
Sliced view of CMS barrel detectors
View of the CMS detector
muon detectors
Complete CMS detector geometry included
More than 1M geometrical volumes
LHCC Review , Nov 2004
Slide 69
Gabriele Cosmo, CERN/PH-SFT

LHCC Review , Nov 2004
Slide 70
LHCb
Gabriele Cosmo, CERN/PH-SFT
Crash analysis

A production job is made of several steps:

3 (signal + 2 minbias) Gauss (detector simulation) – 500 events
or


4 (b-incl + 3 minbias) Gauss – 500 and 3*300 events
Two periods analyzed:
th
th
 62635 jobs run between 5 May and 28 of June




97718 jobs run between 16th August and 19th October




1111 jobs crashed in one of the Gauss runs for any reasons
29 stopped in the generator step
667 stopped in the Geant4 step which includes GiGa interface
529 jobs crashed in one of the Gauss runs for any reasons
85 stopped in the generator step
278 stopped in the Geant4 step which includes GiGa interface
Bug fixes and protections introduced between the 2 periods

for ex. against some looping particles
LHCC Review , Nov 2004
Slide 71
Gabriele Cosmo, CERN/PH-SFT
Gauss in production - DC04

Gauss used in LHCb data challenge (DC04) started 3rd of May 2004
 in production environment for the first time (after tests)
 check its robustness
 Pythia 6.205 for pp-collision generation
 EvtGen v5r0 ( LHCb modification from a – 00 – 11 – 07 ) for B
decays

Geant4 6.1 for detector simulation

three versions of Gauss - fix bugs identified in production


in conversion of MCtruth
particle getting stuck during tracking
LHCC Review , Nov 2004
Slide 72
Gabriele Cosmo, CERN/PH-SFT
Debugging Gauss in production


More than 50% of the Gauss crashes are independent of Geant4
For the ~ 50% occuring in the Geant4 step
 Detailed information available when a problem occurs for foreseen
reason is very useful


As it is done in Hadronic processes
Detailed printout for normal situation
 very important when developing or adopting a new Geant4 version
 should be under the control of the user to not clutter log files during
production
LHCC Review , Nov 2004
Slide 73
Gabriele Cosmo, CERN/PH-SFT
Gauss benchmarking
Memory usage ~220 MB
and stable (no significant
memory leaks)
CPU time performance
( 2.4GHz PIV, gcc 3.2 –O2 )
• minimum bias ~ 22 s/event
• BBbar inclusive ~ 65 s/event
Virtual Memory for a job
kB
Event #
LHCC Review , Nov 2004
Slide 74
Gabriele Cosmo, CERN/PH-SFT