Transcript CBM at FAIR - New Challenges for FEE, DAQ and Trigger Systems

CBM at FAIR
New challenges for Front-End Electronics,
Data Acquisition and Trigger Systems
Walter F.J. Müller, GSI, Darmstadt
XXXVI. Treffen "Kernphysik",
Schleching/Obb., 17-24 February 2005
Vortrag zum Thema:
Hochleistungsdatenverarbeitung in der
Kern- und Teilchenphysik
Outline

CBM (very briefly)



FEE/DAQ/Trigger



17-24 February 2005
observables
setup
requirements
challenges
strategies
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
2
CBM at FAIR
SIS 100 Tm
SIS 300 Tm
U: 35 AGeV
p: 90 GeV
Compressed Baryonic Matter
Experiment
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
3
Mapping the QCD Phase Diagram

RICH/LHC:



CBM






17-24 February 2005
explore μB → 0
travel back to the early
universe
explore μB → max
travel into a neutron star
10 – 45 AGeV
2nd generation experiment
penetrating probes
rare probes
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
4
CBM Physics Topics and Observables

In-medium modifications of hadrons
 onset of chiral symmetry restoration at high ρB
 measure: , ,   e+e- (μ+ μ-)
Good e/π separation
open charm: D0, D±

Strangeness in matter
Vertex detector
 enhanced strangeness production
 measure: K, , , , 

Indications for deconfinement at high ρB
 anomalous charmonium suppression ?
 measure: D0, D±
J/  e+e- (μ+ μ-)

Low cross sections
→ High interaction rates
→ Selective Triggers
Critical point
 event-by-event fluctuations
 measure: π, K
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
Hadron identification
5
CBM Setup
 Radiation hard Silicon pixel/strip detectors in a magnetic dipole field
 Electron detectors: RICH & TRD & ECAL: pion suppression up to 105
 Hadron identification: RPC, RICH
 Measurement of photons, π0, η, and muons: ECAL
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
6
All you want to know about CBM:
Technical Status Report (400 p)
now publicly available
CBM and HADES
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
7
Meson Production in central Au+Au
W. Cassing, E. Bratkovskaya, A. Sibirtsev, Nucl. Phys. A 691 (2001) 745
SIS300
17-24 February 2005
10 MHz interaction rate
needed for 10-15 A GeV
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
8
Open Charm Detection





Example: D0  K-+ (3.9%; c = 124.4 m)
reconstruct tracks
find primary vertex
find displaced tracks
find secondary vertex
target
few 100 μm
5 cm

high selectivity because
combinatorics is reduced
17-24 February 2005
first two planes
of vertex detector
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
9
A Typical Au+Au Collision
Central Au+Au collision
at 25 AGeV:
URQMD + GEANT
160 p
360 - 330 +
41 K+ 13 K-
170 n
360 0
42 K0
 107 Au+Au interactions/sec
 109 tracks/sec to reconstruct
for first level event selection
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
10
CBM Trigger Requirements

In-medium modifications of hadrons
 onset of chiral symmetry restoration at high ρB
offline
 measure: , ,   e+e±
open charm (D0, D ) trigger

Strangeness in matter
 enhanced strangeness production
 measure: K, , , , 
offline

Indications for deconfinement at high ρB
 anomalous charmonium suppression ?
trigger
 measure: D0, D± trigger
J/  e+e

Critical point
 event-by-event fluctuations
 measure: π, K
17-24 February 2005
assume archive rate:
few GB/sec
20 kevents/sec
trigger on
displaced vertex
drives FEE/DAQ
architecture
trigger on high pt
e+ - e- pair
offline
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
11
CBM DAQ Requirements Profile


D and J/Ψ signal drives the rate capability requirements
D signal drives FEE and DAQ/Trigger requirements




Problem similar to B detection, like in BTeV, LHCb
Adopted approach:
displaced vertex 'trigger' in first level, like in BTeV
Additional Problem:
DC beam → interactions at random times
→ time stamps with ns precision needed
→ explicit event association needed
Current design for FEE and DAQ/Trigger:


Self-triggered FEE
Data-push architecture
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
12
Conventional FEE-DAQ-Trigger Layout
Especially
instrumented
detectors
Detector
L0 Trigger
fbunch
Trigger
Primitives
FEE
Dedicated
connections
Buffer
Limited
capacity
Modest
bandwidth
L2 Trigger
Archive
17-24 February 2005
Standard
hardware
Limited
L1 trigger
latency
L1 Accept
DAQ
Cave
Shack
L1 Trigger
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
Specialized
trigger
hardware
13
Limits of Conventional Architecture
Decision time for first
level trigger limited.
typ. max. latency 4 μs for LHC
Not suitable for complex
global triggers like secondary
vertex search
Only especially instrumented
detectors can contribute to
first level trigger
Limits future trigger
development
Large variety of very
specific trigger hardware
High development cost
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
14
The way out .. use Data Push Architecture
Especially
instrumented
detectors
Detector
L0 Trigger
ffbunch
clock
Trigger
Primitives
FEE
Dedicated
connections
Time
distribution
Buffer
Limited
capacity
Modest
High
bandwidth
L2
L1 Trigger
L2Archive
Trigger
17-24 February 2005
Standard
Special
hardware
Limited
L1 trigger
latency
L1 Accept
DAQ
Cave
Shack
L1 Trigger
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
Specialized
trigger
hardware
15
The way out ... use Data Push Architecture
Detector
fclock
FEE
Cave
Shack
DAQ
High
bandwidth
L1 Trigger
L2Archive
Trigger
17-24 February 2005
Special
hardware
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
16
The way out ... use Data Push Architecture
Detector
Self-triggered front-end
Autonomous hit detection
fclock
FEE
No dedicated trigger connectivity
All detectors can contribute to L1
Cave
Shack
Large buffer depth available
System is throughput-limited
and not latency-limited
Use term: Event Selection
17-24 February 2005
DAQ
High
bandwidth
L1 Select
L2
Select
Archive
Special
hardware
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
Modular design:
Few multi-purpose rather
many special-purpose
modules
17
CBM DAQ and Online Event Selection



needed
for D
needed
for J/μ
More than 50% of total data
volume relevant for first level
event selection
Aim for simplicity
Simple two layer approach:
1. event building
2. event processing
usefull
for J/μ
STS, TRD, and ECAL data used
in first level event selection
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
18
Logical Data Flow
Concentrators:
multiplex channels
to high-speed links
Time distribution
Buffers
Build Network
Processing resources for
first level event selection
structured in small farms
Connection to
'high level'
selection processing
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
19
Bandwidth Requirements
Data flow:
~ 1 TB/sec
Gilder helps
Moore helps
1st level selection:
~ 1014-15 operation/sec
Data flow:
few 10 GB/sec
to archive: few 1 GB/sec
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
20
L1 Event Selection Farm Layout





Current working hypothesis: CPU + FPGA hybrid system (proviso follows)
Use programmable logic for cores of algorithms
Use CPU for the non-parallelizable parts
Use serial connection fabric (links and switches)
Modular design (only few board types)
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
21
FPGA – Basic Building Block
CLB = Configurable Logic Block
CLB
X
F0
F1
D
Q
XQ
LUT
F2
C
F3
CLK
Universal
logic gate
17-24 February 2005
Look-up Table
just a 4x1 RAM
D Flip-Flop
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
Elementary
storage unit
22
FPGA – Putting it together
CLB
CLB
PSM
CLB
PSM
CLB
Configurable
Logic Block
PSM
Wiring
CLB
CLB
PSM
CLB
PSM
CLB
PSM
Programmable
switch matrix
I/O blocks
CLB
CLB
PSM
CLB
17-24 February 2005
CLB
PSM
CLB
CLB
PSM
CLB
CLB
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
Modern FPGA's:
>100.000 LUT
500 MHz
23
Algorithms

Performance of L1 feature extraction algorithms is essential


Look for algorithms which allow massive parallel implementation



critical in CBM: STS tracking + vertex reconstruction
TRD tracking and Pid
e.g. Hough Transform Tracker
needs lots of bit level operations, well suited for FPGA
Caveat: simulation on normal CPU quite time consuming....
Co-develop tracking detectors and analysis algorithms


L1 tracking is necessarily speed optimized
→ more detector granularity and redundancy needed
Aim for CBM:
Validate final hardware design with at least 2 trackers suitable for L1
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
24
Interim Summary

Event definition has changed:


Role of DAQ has changed:



filter events delivered by DAQ
'Online Event Selection' is better term
System aspects:





DAQ is simply responsible to transport data from producers to consumers
Role of 'Trigger' has changed:


now based on time stamps and time correlation
'online' – 'offline' boundary blurs
more COTS (commercial off the shelf) components
much more modular system
much more adaptable system
This is emerging technology in HEP, though baseline for ILC
However: being used since many years in nuclear structure
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
25
Moore – quo vadis ?

Will price/performance of computing continue to improve ?



Technology



What are limits of CMOS technology ?
Where are the markets ? What are market forces ?
most of the gain comes from architecture anyway
conventional designs, especially x86, reach their limits
Markets


end of the metal-box PC age
→ Laptops + PDA + all kind of dedicated boxes (Video, Games)
end of the binary compatibility age
→ intermediate code + 'Just in Time' Compilers (JIT)
There is life after Intel x86
A lot of architectural innovation ahead
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
26
BlueGene vs Cell Processor
CPU
CPU
CPU
Cache
Cache
Cache
Mem
IO
IO
IO
IO
BlueGene:
121 mm2; 130 nm
2.8/5.6 DP GFlop
Finally presented
on ISSCC 2005
International Solid-State Circuit Conf.
17-24 February 2005
Mem
IO
Mem
IO
STI Cell:
221 mm2; 90 nm
256 SP GFlop
30 DP GFlop
25 GB/sec mem
78 GB/sec IO
SPE
Mem
Mem
SPE
SPE
Mem
Mem
SPE
SPE
Mem
Mem
SPE
SPE
Mem
Mem
SPE
SPE = Synergistic Processing Element
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
27
BlueGene vs Cell Processor
Developed by IBM
Market: national security
science
Budget: ~100 M$
Developed by
Sony, Toshiba and IBM
Market: VIDEOGAMES
Budget: 500 M$
High performance computing is driven now by embedded systems
(games, video, ....)
→ Science is a spin-off, at best ...
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
28
Game Processors as Supercomputers ?
Slide from CHEP'04
Dave McQueeney
IBM CTO US Federal
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
29
CPU and FPGA paradigms merge
SIMD (single instruction – multiple data) CPU
Register
Control
Control
Conventional CPU
ALU
Wide Register
ALU
ALU
ALU
ALU
Configurable Instruction Set CPU
Control
Wide Register
ALU
ALU
PSM
ALU
PSM
ALU
PSM
ALU
17-24 February 2005
ALU
PSM
ALU
PSM
ALU
ALU
PSM
ALU
PSM
ALU
ALU
PSM
ALU
PSM
ALU
ALU
ALU
PSM
ALU
ALU
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
arithmetic
resources
configurable
connection
fabric
30
Configurable Instruction Set Processor

Example Stretch S5xxx

Hybrid design:



C/C++ compiler analyses the
kernel of algorithms



conventional fixed
instruction set part
plus configurable
instruction set part
generates custom
instruction set
generates code to use it
The promise


easy of use of C/C++
performance of an FPGA
Stretch S5 engine
Fabric is the keyword
interconnected resources
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
31
CPU and FPGA paradigms merge
CPU
configurable
logic
Processor
industry
world view
Unclear what the most suitable
architecture will be
The general trend however
will produce a lot of
innovation in the years to come
Essential will be availability of
efficient development tools
configurable
logic
CPU
17-24 February 2005
CPU
FPGA
industry
world view
Moore will go on !
There are the technologies
There are the markets
Architectural changes ahead
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
32
Summary

Self-triggered FEE:



High bandwidth event building network



to cope with few 100 MHz interaction rate in p-p, p-A
likely be done in time slices or event slices
L1 processor farm



autonomous hit detection, time-stamping with ns presision
sparsification, hit buffering, high output bandwidth
Substantial
R&D
needed
Quite
different
from the
current
LHC style
electronics
feasible with PC + FPGA + Moore (needed 2014)
but look beyond todays PC's and FPGA's
Efficient algorithms (109 tracks/sec)

co-design of critical detectors and tracking software
RII3-CT-2004-506078
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
33
The End
Thanks for
your attention
17-24 February 2005
XXXVI. Treffen Kernphysik, Schleching/Obb., Walter F.J. Müller, GSI
34
CBM Collaboration : 39+ institutions, 14+ countries
China:
Hua-Zhong Univ., Wuhan
Croatia:
RBI, Zagreb
Cyprus:
Nikosia Univ.
Czech Republic:
Czech Acad. Science, Rez
Techn. Univ. Prague
France:
IReS Strasbourg
Hungaria:
Russia:
KFKI Budapest
CKBM, St. Petersburg
Eötvös Univ. Budapest IHEP Protvino
INR Troitzk
Korea:
ITEP Moscow
Korea Univ. Seoul
KRI, St. Petersburg
Pusan National Univ.
Kurchatov Inst., Moscow
LHE, JINR Dubna
Norway:
LPP, JINR Dubna
Univ. Bergen
LIT, JINR Dubna
LTP, JINR Dubna
MEPhi, Moskau
Poland:
Krakow Univ.
Obninsk State Univ.
Warsaw Univ.
Germany:
PNPI Gatchina
Univ. Heidelberg, Phys. Inst. Silesia Univ. Katowice SINP, Moscow State Univ.
Univ. HD, Kirchhoff Inst.
St. Petersburg Polytec. U.
Portugal:
Univ. Frankfurt
LIP Coimbra
Univ. Kaiserslautern
Spain:
Univ. Mannheim
Santiago de Compostela Uni.
Romania:
Univ. Marburg
NIPNE Bucharest
Univ. Münster
Ukraine:
FZ Rossendorf
Shevshenko Univ. , Kiev
membership applications in italic
GSI Darmstadt