Transcript Timepix(3) - Indico
Integrating Timepix(3)
Szymon Kulis, Mathieu Benoit, Bas van der Heijden, Frans Schreuder, Henk Boterenbrood, MvB and the Timepix3 designers Xavi Llopart, Tuomas Poikela, Vladimir Gromov, Francesco Zappon Massimiliano de Gaspari and others 10 December 2013
Timepix versus Timepix3
Timepix
55x55 um 2 pixels 256x256 matrix = 1.4x1.4 cm 2 ToA or ToT measurement frame based readout max. 120 frames/sec, with 100 tracks/frame = ~10k tracks/s (more is possible) dead time due to read out of single frame: 8+ ms not radiation hard FitPix or Relaxd readout
Timepix3
55x55 um 2 pixels 256x256 matrix = 1.4x1.4 cm 2 ToA and ToT simultaneously data driven readout zero-suppressed frame readout possible (tens of) millions of tracks/s theoretical max of 80 Mhits/s dead time less radiation hard; several hundred Mrad but not SEU robust SPIDR or FitPix readout Martin van Beuzekom Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013 2
Timepix integration in common DAQ
Detailed talk given by Mathieu Benoit in WP9.3 meeting on Nov. 21 Lots of info in that talk, will not repeat it here. Please look at https://indico.desy.de/conferenceDisplay.py?confId=8849 Main bottle-neck: Timepix is either in acquisition mode, or in readout mode Readout time of 8 (or more) ms per frame -> long dead time Operation of FitPix and Relaxd is similar
Martin van Beuzekom Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013 3
Our (Nikhef/LHCb) focus is on Timepix3
In view of the LHCb VeloPix developments (ASIC submission aimed for Q3 2014) VeloPix which has to cope with 600 Mhits/s But we also have a complete and working Timepix/Relaxd telescope
Martin van Beuzekom Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013 4
SPIDR
S
peedy
PI
xel
D
etector
R
eadout Readout system for Timepix3 and MPX3.1/RX over 10 Gb or 1 Gb Ethernet
Martin van Beuzekom Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013 5
SPIDR
Firmware properties:
Vendor independent and highly configurable 1 Timepix3 at full (80 Mhits/s) speed, or multiple Timepix3 chips at lower speed LFSR lookup tables in FPGA Pixel data over UDP/IP, slow control over TCP/IP Currently running on a development system (Xilinx VC707)
Multiple setups running at Nikhef and CERN Development of Compact SPIDR ongoing (but at a slower pace)
+/- 400V Bias supply I2C FMC FGPA PHY SFP+ cage 1G/10G DC/DC 1V2 DC/DC 1V5 DC/DC 2V5 USB 12V Trigger/busy Ext bias Expansion header Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013 6 Martin van Beuzekom
Multiple TPX3 hardware configurations
Nikhef Chipboard Nikhef Chipboard Nikhef Chipboard Cern Chipboard (Thanks to Jerome Alozy) Cern Chipboard FMC extender cable 50cm FMC extender cable 50cm FMC extender cable 50cm Xilinx VC707 development board Nikhef Compact readout FMC 2 VHDCI Nikhef Compact readout FitPix Martin van Beuzekom Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013 7
Data acquisition
To run at high speed, each SPIDR needs its own DAQ PC
A single timepix3 at max. speed produces 5.12 Gbits/s Including the extended time-stamp and overhead this fills ~70% of a 10 GbE link Data is streamed to disk, without looking at the data
Not possible to evaluate each packet at maximum speed
Separate monitoring stream which samples (copies) the main data stream
DAQ Format: header of several kBytes (settings etc. ) followed by a stream of pixel packets, 8 bytes each
Header format not yet frozen No trigger/event number or alike Pixels packets are unique by their time-stamp Synchronisation and checking of different SPIDR data streams is important
Martin van Beuzekom Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013 8
Data acquisition II
63
hdr 4b
60 59 PixAddr 16b 44 43 ToA 14b 30 29 ToT 10b 20 19 FTOA 16 4b 15 SPIDR time 16b 0 33 SPIDR time 16b 18 17 ToA 14b 4 3 FTOA 4b 0 time stamp 30 bits, 25ns resolution + 4 bits, 1.56 ns resolution range: 26.844 sec
However 26 seconds is too short for a normal run Timepix3 has an internal 48 bit counter, which is reset with the T0-sync
resolution 25 ns, range 81.44 days This timer can be read on request, and has unique packet header -> timer data will end up in the data-stream Hence to cover a larger time range, the (Leon) processor in the SPIDR FPGA will request readout of this timer every second.
Martin van Beuzekom Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013 9
Running with multiple SPIDRs
repeater repeater
SPIDR SPIDR SPIDR
busy
TLU
clock, T0-sync, shutter
Required TLU functionality can be very basic Provide clock, shutter, T0-sync and combine busy signals (OR) Somewhat more advanced functionality required for high speed monitoring (next slides)
Martin van Beuzekom Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013 10
Interface to TLU, sync time-stamp
Synchronising time-stamps
Disable shutter Send T0-sync, min. 25 ns Enable shutter shutter (ext) tpx3-reset (int, optional) T0-sync (ext) time-stamp data-out w.o reset data-out with reset 25 ns (or more) > 1.6 us > 25 ns n-2 n-1 n 0 1 2 3 4 5 6 7 8 no data packets no data packets
Martin van Beuzekom Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013 11
TLU interface, Busy
Data flow is controlled/halted via shutter/busy mechanism Busy is tied to ‘almost full flag’ of SPIDR ethernet buffer DAQ PC signals busy/overflow by sending pause-frames to SPIDR
-> SPIDR buffers fill up, leading to the SPIDR pulling the busy In addition DAQ PC can send a ‘halt’ command to Leon processor
This will pull also pull the busy line Will be implemented soon shutter (ext-in) busy-0 (out) busy-1 (out) data-out0 data-out1 no data packets no data packets
Martin van Beuzekom Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013 12
Monitoring of data streams
Monitoring of data streams by copying a fraction of the data Send data to a dedicated monitoring/slow control PC Two sampling methods Software sampling
Each DAQ PC takes a snapshot at a given time (CPU timer triggered, or via run control?)
However DAQ streams are not synchronised -> need a large snapshot to guarantee overlap between streams Hardware sampling
Dedicated ethernet packets (different destination IP address, and/or port number) are generated by SPIDR
Copies data from a range of timestamps to this dedicated monitoring packet Start of monitoring sample could be controlled by TLU Length of monitoring sample defined by SPIDR setting This is foreseen, but not yet implemented
Martin van Beuzekom Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013 13
Summary & Plans
Development of Timepix3 readout is actively ongoing (Nikhef + CERN) But building a high speed readout is not trivial (so we have to be patient) We (LHCb) are building a high speed Timepix3 telescope
Timepix3 is good exercise for VeloPix which produces 8x more hits/tracks VeloPix has to be submitted Q3 next year
Integrating Timepix3 into an AIDA telescope seems simpler than integrating Timepix Matching of data from different SPIDR streams using time-stamps
Checking of synchronicity is key Separate monitoring data stream
14 Martin van Beuzekom Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013