Transcript ATLAS Computing

ATLAS Overview Week - 7 October 2004
Computing plans and outlook
Dario Barberis
CERN & Genoa University
Dario Barberis: Computing plans and outlook
1
ATLAS Overview Week - 7 October 2004
ATLAS Computing Timeline
2003
• POOL/SEAL release (done)
• ATLAS release 7 (with POOL persistency) (done)
• LCG-1 deployment (done)
2004
• ATLAS complete Geant4 validation (done)
• ATLAS release 8 (done)
NOW
2005
• DC2 Phase 1: simulation production (in progress)
• DC2 Phase 2: intensive reconstruction (the real challenge!) LATE!
• Combined test beams (barrel wedge) (in progress)
• Computing Model paper
2006
• Computing Memorandum of Understanding
• ATLAS Computing TDR and LCG TDR
• DC3: produce data for PRR and test LCG-n
• Physics Readiness Report
2007
• Start commissioning run
• GO!Dario Barberis: Computing plans and outlook
2
ATLAS Overview Week - 7 October 2004
Miscellaneous News


Computing Model:

convergence to be reached soon (including data sizes)

document to be submitted by end 2004 for LHCC review in Jan’05
Computing TDR:



still due in June 2005 (we have to get going with it!)
LCG MoU and ATLAS Computing Addendum to M&O MoU:

documents being worked on

will be finalized by the end of this year

submitted to the C-RRB and ATLAS RRB in April 2005

to be signed by Funding Bodies before the end of 2005

hopefully effective in 2006
ATLAS Core Computing Project:

will be defined in the C-MoU – draft being discussed with management
Dario Barberis: Computing plans and outlook
3
ATLAS Overview Week - 7 October 2004
Plans for 2005

Consolidation

most required functionalities are now in place or being actively
worked on

some of the developments in 2004 were targeted at specific
clients: DC2, CTB, HLT...


Robustness


both of the software base and the computing environment
(production and generally job submission tools)
Optimization


all have to be brought together coherently
speed of data access and algorithm execution, as well as memory
size, has to become a priority for next year
Usability

easing the threshold for newcomers has to be a priority
Dario Barberis: Computing plans and outlook
4
ATLAS Overview Week - 7 October 2004
Reviews in 2005

We plan a complete set of reviews of all software development
and deployment activities:
1) Grid production tools
2) Software Infrastructure
3) Framework and EDM
4) Detector Description and Graphics
5) Simulation
6) Calibration & Alignment
7) Tracking (InDet & Muons)
8) Calorimetry (LAr & Tile)
9) ESRAT (including Physics Analysis Tools)
10) Database & Data Management
11) Distributed Analysis

Reviews will take
between
and July 2005
Darioplace
Barberis:
ComputingJanuary
plans and outlook
5
ATLAS Overview Week - 7 October 2004
The “final” prototype: DC3

We should consider DC3 as the “final” prototype, for both software
and computing infrastructure

Current tentative schedule is Q1-2006 to end Q2-2006

Schedule is highly constrained

DC2 Phase III (tests of Distributed Analysis) will probably run during Q1-2005

We agreed to have a “season” of reviews during the first half of 2005

Developers will need some time between DC2 and DC3 for necessary work:




completion of the system with the inclusion of missing functionality
integration of components developed in the context of the CTB
implementation of review recommendations
Cosmic run will be later in 2006

This will be the last large scale exercise before real data

After DC3 surely we will be forced to sort out problems day-by-day, as
the need arises, for real, imperfect data coming from the DAQ: no time
for more big developments
Dario Barberis: Computing plans and outlook
6
ATLAS Overview Week - 7 October 2004
DC3 aims

We should agree unambiguously on the aims of this operation



We need from time to time large scale tests (increasing scale
tests) of both software and computing infrastructure
We should build on the lessons of DC2 and not repeat blindly
the same exercise


DC1 and DC2 started with 1-2 main objectives but many other
clients showed up with sometimes negative interferences
example: running ATLAS Geant4 simulation is easy (everything
works well), using the current set of Grid tools is another story
Main aim of DC3 should be to test the software and computing
infrastructure that we will need at the beginning of 2007:

calibration and alignment procedures and conditions DB

Tier-0 reconstruction and data distribution

distributedDario
access
to the data for analysis
Barberis: Computing plans and outlook
7
ATLAS Overview Week - 7 October 2004
DC3 Software (1)



Geometry:

propose initial geometry

updated version with representation of really built detectors

some time-varying misalignments and miscalibrations must be
introduced (this part was descoped from DC2)
Simulation:

good performance in DC2 Geant4 production

continuous production with DC2/initial geometry will start soon

should use “as-built” geometry as soon as available

all available simulated data at the end of 2005 can be used as input
for DC3
Event Data Model:

reviews in early 2005 and feedback from the Rome Physics Workshop
will assess the current EDM for reconstruction and analysis
Dario Barberis: Computing plans and outlook
8
ATLAS Overview Week - 7 October 2004
DC3 Software (2)


Trigger:

we should run L1 trigger as part of digitization routinely

for L2 and EF, we could think of running them as a “pre-challenge”
together with the HLT: let’s open the discussion
Calibration & Alignment:

was descoped from DC2 but developed with the CTB

this winter we have to merge these developments back into the main
stream

we could envisage two Cal/Al passes:


fast, run in a few hours, to provide parameters for Tier-0 reconstruction

slow, with more precise results, to provide parameters for Tier-1
reprocessing within a few months of data taking
we would like to exercise both in DC3
Dario Barberis: Computing plans and outlook
9
ATLAS Overview Week - 7 October 2004
DC3 Software (3)

Reconstruction:


Physics Analysis Tools:



reviews in early 2005 will assess the current algorithms for
reconstruction and analysis
the Rome Physics Workshop will assess the current functionality
Data Management:

we’ll build on the experience of DC2 (intensive part is still to come!)

DM review is foreseen towards the middle of 2005
Production tools:

the review of the production system used for DC2 will be at the beginning
of 2005

this leaves ~1 year to set up the system to be used for DC3

there was also 1 year to develop the DC2 ProdSys but clearly it was not enough

one the other hand, this time we are not starting from zero again
Dario Barberis: Computing plans and outlook
10
ATLAS Overview Week - 7 October 2004
Preliminary plan for DC3 (1)

Aim is a sustained 50% data flow test: we need at least 50M
triggered events if we want to run 10 days intensively



remember: the “nominal” raw data rate is 200 Hz and each day has
50k “active” seconds: 10M events/day
Simulation:

use all production done in any case in the 2nd half of 2005

supplement by missing channels to reach a reasonably
representative mixture of event types

we have to decide on pile-up levels
Pre-DC3:

run trigger (if appropriate), bring events to CERN, mix, put on
Castor disk pool as if they were coming from online
Dario Barberis: Computing plans and outlook
11
ATLAS Overview Week - 7 October 2004
Preliminary plan for DC3 (2)


Real DC3:

run “fast” Cal/Al procedures

reconstruct on Tier-0

distribute output according to the Computing Model

run analysis jobs in many sites

run “slow” Cal/Al procedures

reprocess at Tier-1’s and redistribute the data
Manpower:

need to plan shifts in advance

DC3 should be treated as a test beam operation


timescales to be followed both for developments and productions
We need a “DC3 Coordinator” soon!
Dario Barberis: Computing plans and outlook
12
ATLAS Overview Week - 7 October 2004
After DC3

We should keep the DC3 infrastructure working “forever”


Any modification to software, infrastructure or data
distribution must be planned at that point well in advance


after DC3 we can use both simulated and cosmic ray data as event
source
no “here is the solution!” untested bug fix in the production
environment
We must run the production in a more professional way
effectively from early 2006 for many many years

production tool developer team

production operation team

adequate monitoring tools

people on shift (also remotely) to follow and monitor the production

running offline shifts will be a task for the whole Collaboration
Dario Barberis: Computing plans and outlook
13