Transcript Document

High Level Triggering
Fred Wickens
High Level Triggering (HLT)
• Introduction to triggering and HLT systems
– Why do we Trigger
– Why do we use Multi-Level Triggering
– Brief description of “typical” 3 level trigger
• Case study of ATLAS HLT (+ some
comparisons with other experiments)
• Summary
2
Why do we Trigger and why multi-level
• Over the years experiments have focussed on rarer processes
– Need large statistics of these rare events
– DAQ system (and off-line analysis capability) under increasing
strain
• limiting useful event statistics
• Aim of the trigger is to record just the events of interest
• i.e. Trigger = system which selects the events we wish to study
• Originally - only read-out the detector if Trigger satisfied
– Larger detectors and slow serial read-out => large dead-time
– Also increasingly difficult to select the interesting events
• Introduced: Multi-level triggers and parallel read-out
– At each level apply increasingly complex algorithms to obtain better
event selection/background rejection
• These have:
– Led to major reduction in Dead-time – which was the major issue
– Managed growth in data rates – this remains the major issue
3
Summary of ATLAS Data Flow Rates
• From detectors
> 1014 Bytes/sec
• After Level-1 accept
~ 1011 Bytes/sec
• Into event builder
~ 109 Bytes/sec
• Onto permanent storage
~ 108 Bytes/sec

~ 1015 Bytes/year
4
The evolution of DAQ systems
5
TDAQ Comparisons
6
Level 1
• Time:
few microseconds
• Hardware based
– Using fast detectors + fast algorithms
– Reduced granularity and precision
• calorimeter energy sums
• tracking by masks
• During Level-1 decision time store event data in frontend electronics
– at LHC use pipeline - as collision rate shorter than Level-1
decision time
• For details of Level-1 see Dave Newbold lecture
– 2 weeks ago
7
Level 2
• Previously - few milliseconds (1-100)
– Dedicated microprocessors, adjustable algorithm
• 3-D, fine grain calorimetry
• tracking, matching
• Topology
– Different sub-detectors handled in parallel
• Primitives from each detector may be combined in a global
trigger processor or passed to next level
• 2009 - few milliseconds (10-100)
–
–
–
–
Processor farm with Linux PC’s
Partial events received via high-speed network
Specialised algorithms
Each event allocated to a single processor, large farm of
processors to handle rate
8
Level 3
• millisecs to seconds
• processor farm
– Previously microprocessors/emulators
– Now standard server PC’s
• full or partial event reconstruction
– after event building (collection of all data from all
detectors)
• Each event allocated to a single processor,
large farm of processors to handle rate
9
Summary of Introduction
• For many physics analyses, aim is to obtain as high
statistics as possible for a given process
– We cannot afford to handle or store all of the data a detector
can produce!
• The Trigger
– selects the most interesting events from the myriad of events
seen
• I.e. Obtain better use of limited output band-width
• Throw away less interesting events
• Keep all of the good events(or as many as possible)
– must get it right
• any good events thrown away are lost for ever!
• High level Trigger allows:
– More complex selection algorithms
– Use of all detectors and full granularity full precision data
10
Case study of the ATLAS HLT system
Concentrate on issues relevant for
ATLAS (CMS very similar issues), but
try to address some more general
points
Starting points for any HLT system
• physics programme for the experiment
– what are you trying to measure
• accelerator parameters
– what rates and structures
• detector and trigger performance
– what data is available
– what trigger resources do we have to use it
12
Physics at the LHC
Interesting events are buried in a sea
of soft interactions
B physics
High energy QCD jet
production
top physics
Higgs production
13
The LHC and ATLAS/CMS
• LHC has
– design luminosity 1034 cm-2s-1 (In 2009 from 1030 - 1032 ?)
– bunch separation 25 ns (bunch length ~1 ns)
• This results in
– ~ 23 interactions / bunch crossing
• ~ 80 charged particles (mainly soft pions) / interaction
• ~2000 charged particles / bunch crossing
• Total interaction rate
– b-physics
– t-physics
– Higgs
fraction ~ 10-3
fraction ~ 10-8
fraction ~ 10-11
109 sec-1
106 sec-1
10 sec-1
10-2 sec-1
14
Physics programme
• Higgs signal extraction important - but very difficult
• There is lots of other interesting physics
–
–
–
–
–
B physics and CP violation
quarks, gluons and QCD
top quarks
SUSY
‘new’ physics
• Programme will evolve with: luminosity, HLT capacity and
understanding of the detector
– low luminosity (2009 - 2010)
• high PT programme (Higgs etc.)
• b-physics programme (CP measurements)
– high luminosity (2011?)
• high PT programme (Higgs etc.)
• searches for new physics
15
Trigger strategy at LHC
• To avoid being overwhelmed use signatures with
small backgrounds
– Leptons
– High mass resonances
– Heavy quarks
• The trigger selection looks for events with:
–
–
–
–
Isolated leptons and photons,
-, central- and forward-jets
Events with high ET
Events with missing ET
16
Example Physics signatures
Objects
Physics signatures
Electron 1e>25, 2e>15 GeV
Higgs (SM, MSSM), new gauge bosons,
extra dimensions, SUSY, W, top
Photon 1γ>60, 2γ>20 GeV
Higgs (SM, MSSM), extra dimensions,
SUSY
Muon 1μ>20, 2μ>10 GeV
Higgs (SM, MSSM), new gauge bosons,
extra dimensions, SUSY, W, top
Jet 1j>360, 3j>150, 4j>100 GeV
SUSY, compositeness, resonances
Jet >60 + ETmiss >60 GeV
SUSY, exotics
Tau >30 + ETmiss >40 GeV
Extended Higgs models, SUSY
17
Trigger
40 MHz
ARCHITECTURE
DAQ
Three logical levels
Hierarchical data-flow
LVL1 - Fastest:
Only Calo and Mu
Hardwired
On-detector
electronics:
Pipelines
~40 ms
LVL2 - Local:
LVL1 refinement +
track association
Event fragments
buffered in
parallel
~4 sec.
LVL3 - Full event:
“Offline” analysis
Full event in
processor farm
~2.5 ms
~ 200 Hz
Physics
~1 PB/s
(equivalent)
~ 300 MB/s
18
Selected (inclusive) signatures
Process
Level-1
2 em, ET>20 GeV
H0
H0Z Z* + – + – 2 em, ET>20 GeV
2 µ, pT>6 GeV
1 em, ET>30 GeV
1 µ, pT>20 GeV
2 em, ET>20 GeV
Z+–+X
2 µ, pT>6 GeV
1 em, ET>30 GeV
1 µ, pT>20 GeV
1 em, ET>30 GeV
t t  leptons+jets
1 µ, pT>20 GeV
W', Z'jets
1 jet, ET>150 GeV
SUSYjets
1 jet, ET>150 GeV
E miss
T
Level-2
2 , ET>20 GeV
2 e, ET>20 GeV
2 µ, ET>6 GeV, I
1 e, ET>30 GeV
1 µ, ET>20 GeV, I
2 e, ET>20 GeV
2 µ, ET>6 GeV, I
1 e, ET>30 GeV
1 µ, ET>20 GeV, I
1 e, ET>30 GeV
1 µ, ET>20 GeV, I
1 jet, ET>300 GeV
3 jet, ET>150 GeV
E miss
T
19
Trigger design – Level-1
• Level-1
– sets the context for the HLT
– reduces triggers to ~75 kHz
• Uses limited detector data
– Fast detectors (Calo + Muon)
– Reduced granularity
• Trigger on inclusive
signatures
• muons;
• em/tau/jet calo clusters;
missing and sum ET
• Hardware trigger
– Programmable thresholds
– CTP selection based on
multiplicities and thresholds
20
Level-1 Selection
• The Level-1 trigger
– an “or” of a large number of inclusive signals
– set to match the current physics priorities and beam
conditions
• Precision of cuts at Level-1 is generally limited
• Adjust the overall Level-1 accept rate (and the
relative frequency of different triggers) by
– Adjusting thresholds
– Pre-scaling (e.g. only accept every 10th trigger of a
particular type) higher rate triggers
• Can be used to include a low rate of calibration events
• Menu can be changed at the start of run
– Pre-scale factors may change during the course of a run
21
Example Level-1 Menu for 2x10^33
Level-1 signature
Output Rate (Hz)
EM25i
12000
2EM15i
4000
MU20
800
2MU6
200
J200
200
3J90
200
4J65
200
J60 + XE60
400
TAU25i + XE30
2000
MU10 + EM15i
100
Others (pre-scaled, exclusive, monitor, calibration)
Total
5000
~25000
22
Trigger design - Level-2
• Level-2 reduce triggers to ~2 kHz
– Note CMS does not have a physically separate Level-2 trigger, but
the HLT processors include a first stage of Level-2 algorithms
• Level-2 trigger has a short time budget
– ATLAS ~40 milli-sec average
• Note for Level-1 the time budget is a hard limit for every event, for the
High Level Trigger it is the average that matters, so OK for a small
fraction of events to take times much longer than this average
• Full detector data is available, but to minimise resources
needed:
–
–
–
–
Limit the data accessed
Only unpack detector data when it is needed
Use information from Level-1 to guide the process
Analysis proceeds in steps with possibility to reject event after each
step
– Use custom algorithms
23
Regions of Interest
• The Level-1 selection is dominated by local
signatures (I.e. within Region of Interest RoI)
– Based on coarse granularity
data from calo and mu only
• Typically, there are
1-2 RoI/event
• ATLAS uses RoI’s to reduce
network b/w and processing
power required
24
Trigger design - Level-2 - cont’d
• Processing scheme
– extract features from sub-detector data in each
RoI
– combine features from one RoI into object
– combine objects to test event topology
• Precision of Level-2 cuts
– Emphasis is on very fast algorithms with
reasonable accuracy
• Do not include many corrections which may be applied
off-line
– Calibrations and alignment available for trigger not
as precise as ones available for off-line
25
ARCHITECTURE
Trigger
Calo
MuTrCh
40 MHz
40 MHz
LVL1
Muon
Trigger
ROD
ROIB
L
ROD
120
RoI’s
LVL2
~ 10 ms
RoI
requests
L2SV
ROB
ROD
GB/s
ROB
ROB
ROS
RoI data = 1-2%
L2P
L2P
L2P
T Event Filter
EFP
EFP
EFP
L2N
~2 GB/s
LVL2 accept
~ 1 sec
~ 1 PB/s
FE Pipelines
2.5 ms
LVL1 accept
75 kHz
~2 kHz
Other detectors
2.5 ms
Calorimeter
Trigger
H
DAQ
Read-Out Drivers
Read-Out Links
Read-Out Buffers
Read-Out Sub-systems
~3 GB/s
Event Builder
EB
~3 GB/s
EFN
~ 300 MB/s
~ 200 Hz
~ 300 MB/s
26
CMS Event Building
• CMS perform Event Building after Level-1
• This simplifies the architecture, but places
much higher demand on technology:
– Network traffic ~100 GB/s
• Use Myrinet instead of GbE for the EB network
• Plan a number of independent slices with barrel shifter to
switch to a new slice at each event
– Time will tell which
philosophy is better
27
Example for Two electron trigger
Level1 seed 
EM20i
+
EM20i
HLT Strategy:



Validate step-by-step
Check intermediate signatures
Reject as early as possible
Sequential/modular
approach facilitates
early rejection
STEP 2
Signature 
STEP 3
Signature 
STEP 4
Signature 
track
finding
e
track
finding
+
pt>
30GeV
e30
pt>
30GeV
+
Iso–
lation
e30i
e
time
LVL1 triggers on two isolated
Cluster
Cluster
STEP 1
shape
shape
e/m clusters with pT>20GeV
(possible signature: Z–>ee)
Signature  ecand + ecand
e30
Iso–
lation
+
e30i
28
Trigger design - Event Filter / Level-3
• Event Filter reduce triggers to ~200 Hz
• Event Filter budget ~ 4 sec average
• Full event detector data is available, but to
minimise resources needed:
– Only unpack detector data when it is needed
– Use information from Level-2 to guide the process
– Analysis proceeds in steps with possibility to reject
event after each step
– Use optimised off-line algorithms
29
Electron slice at the EF
Wrapper of CaloRec
TrigCaloRec
EFCaloHypo
Wrapper of newTracking
matches electromagnetic
clusters with tracks and builds
egamma objects
EF tracking
EFTrackHypo
Wrapper of EgammaRec TrigEgammaRec
EFEgammaHypo
30
Trigger design - HLT strategy
• Level 2
– confirm Level 1, some inclusive, some semiinclusive,
some simple topology triggers, vertex
reconstruction
(e.g. two particle mass cuts to select Zs)
• Level 3
– confirm Level 2, more refined topology selection,
near off-line code
31
Example HLT Menu for 2x10^33
HLT signature
Output Rate (Hz)
e25i
40
2e15i
<1
gamma60i
25
2gamma20i
2
mu20i
40
2mu10
10
j400
10
3j165
10
4j110
10
j70 + xE70
20
tau35i + xE45
5
2mu6 with vertex, decay-length and mass cuts (J/psi, psi’, B)
10
Others (pre-scaled, exclusive, monitor, calibration)
20
Total
~200
32
Example B-physics Menu for 10^33
LVL1 :
•
•
•
MU6 rate 24kHz (note there are large uncertainties in cross-section)
In case of larger rates use MU8 => 1/2xRate
2MU6
LVL2:
•
•
•
•
•
Run muFast in LVL1 RoI ~ 9kHz
Run ID recon. in muFast RoI mu6 (combined muon & ID) ~ 5kHz
Run TrigDiMuon seeded by mu6 RoI (or MU6)
Make exclusive and semi-inclusive selections using loose cuts
– B(mumu), B(mumu)X, J/psi(mumu)
Run IDSCAN in Jet RoI, make selection for Ds(PhiPi)
EF:
•
•
•
Redo muon reconstruction in LVL2 (LVL1) RoI
Redo track reconstruction in Jet RoI
Selections for B(mumu) B(mumuK*) B(mumuPhi), BsDsPhiPi etc.
33
Matching problem
Background
Off-line
Physics
channel
On-line
34
Matching problem (cont.)
• ideally
– off-line algorithms select phase space which
shrink-wraps the physics channel
– trigger algorithms shrink-wrap the off-line selection
• in practice, this doesn’t happen
– need to match the off-line algorithm selection
• For this reason many trigger studies quote trigger
efficiency wrt events which pass off-line selection
– BUT off-line can change algorithm, re-process and
recalibrate at a later stage
• So, make sure on-line algorithm selection is
well known, controlled and monitored
35
Selection and rejection
• as selection criteria are tightened
– background rejection improves
– BUT event selection efficiency decreases
1
select / reject fraction
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
cut value
select
reject
36
Selection and rejection
• Example of a ATLAS Event Filter (I.e. Level-3) study
of the effectiveness of various discriminants used to
select 25 GeV electrons from a background of dijets
37
Other issues for the Trigger
• Efficiency and Monitoring
– In general need high trigger efficiency
– Also for many analyses need a well known efficiency
• Monitor efficiency by various means
– Overlapping triggers
– Pre-scaled samples of triggers in tagging mode (pass-through)
– To assist with overall normalisation ATLAS divides each run into
periods of a few minutes called a luminosity block.
• During each block the beam luminosity should be constant and can
also exclude any blocks where there is a known problem
• Final detector calibration and alignment constants not available
immediately - keep as up-to-date as possible and allow for the
lower precision in the trigger cuts when defining trigger menus
and in subsequent analyses
• Code used in trigger needs to be very robust - low memory
leaks, low crash rate, fast
38
Other issues for the Trigger – cont’d
• Beam conditions and HLT resources will evolve over several
years (for both ATLAS and CMS)
– In 2009 luminosity low, but also HLT capacity will be < 50% of full
system
• For details of the current ideas on ATLAS Menu evolution see
– https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerPhysicsMenu
• Gives details of menu for Startup (including single beam running),
10^31, 10^32, 10^33
• Description of current menu is very long !!
– Trigger Workshop next week in Beatenberg
• One aim to reduce menus to something more manageable for early
running
• Corresponding information for CMS is at
– https://twiki.cern.ch/twiki/bin/view/CMS/TriggerMenuDevelopment
39
Summary
• High-level triggers allow complex selection procedures to
be applied as the data is taken
– Thus allow large samples of rare events to be recorded
• The trigger stages - in the ATLAS example
– Level 1 uses inclusive signatures (mu’s; em/tau/jet; missing and
sum ET)
– Level 2 refines Level 1 selection, adds simple topology triggers,
vertex reconstruction, etc
– Level 3 refines Level 2 adds more refined topology selection
• Trigger menus need to be defined, taking into account:
– Physics priorities, beam conditions, HLT resources
• Include items for monitoring trigger efficiency and calibration
• Try to match trigger cuts to off-line selection
• Trigger efficiency should be as high as possible and well
monitored
• Must get it right - events thrown away are lost for ever!
• Triggering closely linked to physics analyses – so enjoy!
40
Additional Foils
41
ATLAS HLT Hardware
Each rack of HLT (XPU) processors contains
- ~30 HLT PC’s (PC’s very similar to Tier-0/1 compute nodes)
- 2 Gigabit Ethernet Switches
- a dedicated Local File Server
Final system will contain ~2300 PC’s
42
SDX1|2nd floor|Rows 3 & 2
43
Naming Convention
First Level Trigger (LVL1) Signatures in
capitals e.g.
LVL1
HLT
type
e
electron
g
photon
MU
mu
muon
HA
tau
tau
fj
forward
jet
JE
je
jet energy
JT
jt
jet
TM
xe
missing
energy
threshold
EM
MU 20 I
name isolated
HLT in lower case:
threshold
EF in tagging
mode
mu 20 i _ passEF
name isolated
New in 13.0.30:
•
Threshold is cut value applied
•
previously was ~95% effic. point.
•
FJ
More details : see :https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerPhysicsMenu
44
What is a minimum bias event ?
- event accepted with the only requirement being activity in the detector
with minimal pT threshold [100 MeV] (zero bias events have no requirements)
- e.g. Scintillators at L1 + (> 40 SCT S.P. or > 900 Pixel clusters) at L2
- a miminum bias event is most likely to be either:
- a low pT (soft) non-diffractive event
- a soft single-diffractive event
- a soft double diffractive event
(some people do not include the diffractive events in the definition !)
- it is characterised by:
- having no high pT objects : jets; leptons; photons
- being isotropic
- see low pT tracks at all phi in a tracking detector
- see uniform energy deposits in calorimeter as function of rapidity
- these events occur in 99.999% of collisions. So if any given crossing
has two interactions and one of them has been triggered due to a high pT
component then the likelihood is that the accompanying event will
be a dull minimum bias event.
45
L1 Rates 1031 14.4.0
L1 output rate 1031
(Total Rate 20 kHz)
Multi Muon
0%
Trigger Group
Multi Tau
2%
Single EM
28%
Multi EM
32%
Single Muon
9%
XE
0%
Jets
0%
Multi Object
28%
Single Tau
1%
Multi EM
Multi Object
Single EM
Single Muon
Multi Tau
Single Tau
Jets
Multi Muon
XE
TOTAL
Rate (Hz)
6400
5500
5500
1700
470
150
80
70
50
20000
Removing overlaps between single+multi EM gives 18 kHz
Total estimated L1 rate with all overlaps removed is ~ 12 kHz
46
L2 Rates 1031 14.4.0
L2 output rate 1031
(Total Rate 900 Hz)
XE+
10%
Jets
3%
Trigger Group
B Phys
5%
Electrons
34%
Photons
5%
Tau+X
20%
Muons
23%
Electrons
Muons
Taus+X
XE+
Photons
B Phys
Jets
TOTAL
Rate (Hz)
310
210*
180
82
46
43
22
900
X=anything;
+ includesJE,
TE, anything
with MET
except taus;
Bphys includes
Bjet
* Manually prescaled off pass-through triggers mu4_tile, mu4_mu6
Total estimated L2rate with all overlaps removed is 840 Hz
47
EF Rates 1031 14.4.0
EF output rate 1031
(Total Rate 310 Hz)
Trigger Group
B Phys
12%
XE+
4%
Misc
4%
Electrons
22%
Jets
8%
Photons
6%
Muons
26%
Tau+X
18%
Rate (Hz)
Muons
Electrons
Tau+X
B Phys
Jets
Photons
XE+
Misc
TOTAL
80
67
56
37
25
18
13
13
310
91 Hz total is in prescaled triggers;
51 Hz of prescaled triggers is unique rate
Total estimated EF Rate with overlaps removed is 250 Hz
48
L1 Rates 1032 14.4.0
L1 output rate 1032
(Total Rate 73 kHz)
Trigger Group
Multi Muon
1%
Multi Tau
6%
XE
0%
Single EM
11%
Multi EM
15%
Jets
0%
Single Muon
24%
Multi Object
42%
Single Tau
1%
Rate (Hz)
Multi Object
30000
Single Muon
17000
Multi EM
11000
Single EM
8100
Multi Tau
4300
Single Tau
870
Multi Muon
690
Jets
300
XE
300
TOTAL
73000
Total estimated L1 rate with all overlaps removed is 46 kHz
49
L2 Rates 1032 14.4.0
L2 output rate 1032
(Total Rate 2600 Hz)
Trigger Group
Other
1%
3 Objects
B Phys 10%
Electrons
15%
4%
Muons
11%
XE+
22%
Jets
1%
Photons
5%
Tau+X
31%
Rate (Hz)
Tau+X
820
XE+
590
Electrons
390
Muons
280
3 Objects
270
Photons
120
B Phys
110
Jets
33
Misc
28
TOTAL
2600
Total estimated L2 with all overlaps removed is 1700 (too high!)
50
EF Rates 1032 14.4.0
EF output rate 1032
(Total Rate 510 Hz)
Trigger Group
Misc
2%
Tau+X
3 Objects
9%
BPhys
9%
Electrons
15%
Muons
9%
XE+
8%
Jets
2%
Photons
9%
Tau+X
37%
Rate (Hz)
187
Electrons
77
Muons
46
Photons
46
BPhys
45
3 Objects
45
XE+
42
Jets
11
Misc
11
TOTAL
510
Total estimated EF rates with all overlaps removed is 390 Hz
(Fixing L2 will likely come close to fixing EF as well)
51