ATLAS Computing
Download
Report
Transcript ATLAS Computing
ATLAS Overview Week - 7 October 2004
Computing plans and outlook
Dario Barberis
CERN & Genoa University
Dario Barberis: Computing plans and outlook
1
ATLAS Overview Week - 7 October 2004
ATLAS Computing Timeline
2003
• POOL/SEAL release (done)
• ATLAS release 7 (with POOL persistency) (done)
• LCG-1 deployment (done)
2004
• ATLAS complete Geant4 validation (done)
• ATLAS release 8 (done)
NOW
2005
• DC2 Phase 1: simulation production (in progress)
• DC2 Phase 2: intensive reconstruction (the real challenge!) LATE!
• Combined test beams (barrel wedge) (in progress)
• Computing Model paper
2006
• Computing Memorandum of Understanding
• ATLAS Computing TDR and LCG TDR
• DC3: produce data for PRR and test LCG-n
• Physics Readiness Report
2007
• Start commissioning run
• GO!Dario Barberis: Computing plans and outlook
2
ATLAS Overview Week - 7 October 2004
Miscellaneous News
Computing Model:
convergence to be reached soon (including data sizes)
document to be submitted by end 2004 for LHCC review in Jan’05
Computing TDR:
still due in June 2005 (we have to get going with it!)
LCG MoU and ATLAS Computing Addendum to M&O MoU:
documents being worked on
will be finalized by the end of this year
submitted to the C-RRB and ATLAS RRB in April 2005
to be signed by Funding Bodies before the end of 2005
hopefully effective in 2006
ATLAS Core Computing Project:
will be defined in the C-MoU – draft being discussed with management
Dario Barberis: Computing plans and outlook
3
ATLAS Overview Week - 7 October 2004
Plans for 2005
Consolidation
most required functionalities are now in place or being actively
worked on
some of the developments in 2004 were targeted at specific
clients: DC2, CTB, HLT...
Robustness
both of the software base and the computing environment
(production and generally job submission tools)
Optimization
all have to be brought together coherently
speed of data access and algorithm execution, as well as memory
size, has to become a priority for next year
Usability
easing the threshold for newcomers has to be a priority
Dario Barberis: Computing plans and outlook
4
ATLAS Overview Week - 7 October 2004
Reviews in 2005
We plan a complete set of reviews of all software development
and deployment activities:
1) Grid production tools
2) Software Infrastructure
3) Framework and EDM
4) Detector Description and Graphics
5) Simulation
6) Calibration & Alignment
7) Tracking (InDet & Muons)
8) Calorimetry (LAr & Tile)
9) ESRAT (including Physics Analysis Tools)
10) Database & Data Management
11) Distributed Analysis
Reviews will take
between
and July 2005
Darioplace
Barberis:
ComputingJanuary
plans and outlook
5
ATLAS Overview Week - 7 October 2004
The “final” prototype: DC3
We should consider DC3 as the “final” prototype, for both software
and computing infrastructure
Current tentative schedule is Q1-2006 to end Q2-2006
Schedule is highly constrained
DC2 Phase III (tests of Distributed Analysis) will probably run during Q1-2005
We agreed to have a “season” of reviews during the first half of 2005
Developers will need some time between DC2 and DC3 for necessary work:
completion of the system with the inclusion of missing functionality
integration of components developed in the context of the CTB
implementation of review recommendations
Cosmic run will be later in 2006
This will be the last large scale exercise before real data
After DC3 surely we will be forced to sort out problems day-by-day, as
the need arises, for real, imperfect data coming from the DAQ: no time
for more big developments
Dario Barberis: Computing plans and outlook
6
ATLAS Overview Week - 7 October 2004
DC3 aims
We should agree unambiguously on the aims of this operation
We need from time to time large scale tests (increasing scale
tests) of both software and computing infrastructure
We should build on the lessons of DC2 and not repeat blindly
the same exercise
DC1 and DC2 started with 1-2 main objectives but many other
clients showed up with sometimes negative interferences
example: running ATLAS Geant4 simulation is easy (everything
works well), using the current set of Grid tools is another story
Main aim of DC3 should be to test the software and computing
infrastructure that we will need at the beginning of 2007:
calibration and alignment procedures and conditions DB
Tier-0 reconstruction and data distribution
distributedDario
access
to the data for analysis
Barberis: Computing plans and outlook
7
ATLAS Overview Week - 7 October 2004
DC3 Software (1)
Geometry:
propose initial geometry
updated version with representation of really built detectors
some time-varying misalignments and miscalibrations must be
introduced (this part was descoped from DC2)
Simulation:
good performance in DC2 Geant4 production
continuous production with DC2/initial geometry will start soon
should use “as-built” geometry as soon as available
all available simulated data at the end of 2005 can be used as input
for DC3
Event Data Model:
reviews in early 2005 and feedback from the Rome Physics Workshop
will assess the current EDM for reconstruction and analysis
Dario Barberis: Computing plans and outlook
8
ATLAS Overview Week - 7 October 2004
DC3 Software (2)
Trigger:
we should run L1 trigger as part of digitization routinely
for L2 and EF, we could think of running them as a “pre-challenge”
together with the HLT: let’s open the discussion
Calibration & Alignment:
was descoped from DC2 but developed with the CTB
this winter we have to merge these developments back into the main
stream
we could envisage two Cal/Al passes:
fast, run in a few hours, to provide parameters for Tier-0 reconstruction
slow, with more precise results, to provide parameters for Tier-1
reprocessing within a few months of data taking
we would like to exercise both in DC3
Dario Barberis: Computing plans and outlook
9
ATLAS Overview Week - 7 October 2004
DC3 Software (3)
Reconstruction:
Physics Analysis Tools:
reviews in early 2005 will assess the current algorithms for
reconstruction and analysis
the Rome Physics Workshop will assess the current functionality
Data Management:
we’ll build on the experience of DC2 (intensive part is still to come!)
DM review is foreseen towards the middle of 2005
Production tools:
the review of the production system used for DC2 will be at the beginning
of 2005
this leaves ~1 year to set up the system to be used for DC3
there was also 1 year to develop the DC2 ProdSys but clearly it was not enough
one the other hand, this time we are not starting from zero again
Dario Barberis: Computing plans and outlook
10
ATLAS Overview Week - 7 October 2004
Preliminary plan for DC3 (1)
Aim is a sustained 50% data flow test: we need at least 50M
triggered events if we want to run 10 days intensively
remember: the “nominal” raw data rate is 200 Hz and each day has
50k “active” seconds: 10M events/day
Simulation:
use all production done in any case in the 2nd half of 2005
supplement by missing channels to reach a reasonably
representative mixture of event types
we have to decide on pile-up levels
Pre-DC3:
run trigger (if appropriate), bring events to CERN, mix, put on
Castor disk pool as if they were coming from online
Dario Barberis: Computing plans and outlook
11
ATLAS Overview Week - 7 October 2004
Preliminary plan for DC3 (2)
Real DC3:
run “fast” Cal/Al procedures
reconstruct on Tier-0
distribute output according to the Computing Model
run analysis jobs in many sites
run “slow” Cal/Al procedures
reprocess at Tier-1’s and redistribute the data
Manpower:
need to plan shifts in advance
DC3 should be treated as a test beam operation
timescales to be followed both for developments and productions
We need a “DC3 Coordinator” soon!
Dario Barberis: Computing plans and outlook
12
ATLAS Overview Week - 7 October 2004
After DC3
We should keep the DC3 infrastructure working “forever”
Any modification to software, infrastructure or data
distribution must be planned at that point well in advance
after DC3 we can use both simulated and cosmic ray data as event
source
no “here is the solution!” untested bug fix in the production
environment
We must run the production in a more professional way
effectively from early 2006 for many many years
production tool developer team
production operation team
adequate monitoring tools
people on shift (also remotely) to follow and monitor the production
running offline shifts will be a task for the whole Collaboration
Dario Barberis: Computing plans and outlook
13