Transcript New DAQ for precise experiments with CMD
NPD09
CMD-3 DAQ hardware : features and status.
A.Ruban*, A.Aulchenco, A.Kozyrev, A.Selivanov, L.Smolina, V.Titov
BINP, Novosibirsk.
Moscow, ITEP, November, 23-27,2009
CMD CMD-3 DAQ hardware: features and status.
Moscow, ITEP, November, 23-27,2009 – 3 Subsystem Layout
1 – Vacuum Pipe 2 – Drift Chamber 3 – BGO Endcap Calorimeter 4 – Z – Chamber 5 – Superconducting Solenoid CMD-3 6 – LXe Calorimeter 7 – CsI Calorimeter 8 – Yoke 9 – Superconducting magnet lenses Mu-System and TOF are not showed
CMD-3 DAQ hardware: features and status.
Moscow, ITEP, November, 23-27,2009 General requirements for DAQ Total number of channel is about 10k Average speed of good events – 1k Evtps Low Cost, low EMI, low Power Step-by-step inferring capability, support for legacy system Specific requirements for DAQ Precise time resolution – less then 20ps Low, well-known level of error rate Precise leave and dead time control Precise analog channel efficiency control Precise Trigger efficiency control CMD-3 and VEPP2000 specific environment Low energy deposition per cell – low primary signals Relatively small size of cell – preamplifiers and shielding is space constrained Tight, high density layout, cross interference aware High pick-up from power supply of accelerator facility
CMD-3 DAQ hardware: features and status.
Moscow, ITEP, November, 23-27,2009 When CMD-3 DAQ project starts the CERN S-Link and Belle Copper system exists. But it was very “heavy” to start, power angry and expensive. To satisfy the CMD-3 requirements the new Time Oriented Measurement and Acquire (TOMA) DAQ is designed. Typically modern HEP DAQ hardware has very complicated hierarchical design. The main feature of TOMA DAQ is exchange of hardware hierarchical complexity to synchronization modes hierarchical complexity. It means different modes of synchronization are organized in hierarchy levels and each specific mode is assigned to specific task. Thus it is possible to make hardware in single range level, or Flat Model Approach. In conjunction with modern FPGA’s “natural parallelism” it allows to use modular design of hardware with a little cross dependence of modules. Obviously, it is the way to built reusable IP which will be common for all kind of boards in DAQ.
To support system unification and man power saving two “building block” was designed. It are “Skeletal Design” and “C-Link”.
CMD-3 DAQ hardware: features and status.
Moscow, ITEP, November, 23-27,2009 This picture demonstrates so called Skeletal Design which is created for and used in all boards of TOMA DAQ. It is parameterized HDL description suitable for low cost FPGA. The design is built as a Flat Model approach. Adding or removing of board specific modules does not affect functionality of another modules in the design.
TOMA DAQ Skeletal Design
CMD-3 DAQ hardware: features and status.
Moscow, ITEP, November, 23-27,2009 All synchronization and data transferring in TOMA DAQ is provided by single specially developed tool. While well-known S-link is designed to hide all synchronization detail, our tool is intended to obtain all chronopher functions, thus it is called C-Link. Note, please, this is NOT a mezzanine at physical layer, this is part of FPGA design. C Link modules and it’s interface is designed in flat model approach.
C-Link specificaton region Physical Interface Transmission media Processing Board Module Down-Link Module Up-Link Frontend Board Electrical Interface Logical Interface
Reference Clock speed Data speed Electrical Levels Media Connector Type F bunch_Crossing*2 = 25 МHz 25 Мbps (50 or 100 optional) LVDS, 4mA same as IEEE-1394 Double UTP Cat5 130Ohm, double side termination USB, type А, both ends
CMD-3 DAQ hardware: features and status.
Moscow, ITEP, November, 23-27,2009 Simple
Synchronous mode
is used to generate Time Stamps (
Common Stop
) in Time measuring Boards. Start bit of each transaction is sampled at reference Phase. Generally this sampling circuit is implemented as specific fast channel, bypassing FPGA and any complex logic IC. While LVDS level is recommended, it is compatible to use a LVPECL electric layer. It is possible to reach good enough jitter performance.
The drawback of this “simple” method is tight requirements for relative phase shifts of all signals for all boards involved. It is real problem for HEP’s DAQ.
PLL Phase Line RF Cavity Freq.
Link's_bitstream CLK D Q Bunch Crossing Enable MChS Signaling Layer Data/ Comm line CLK D Q Start bit detect Time Stamp Physical Layer Time Measuring Board Signaling Layer
CMD-3 DAQ hardware: features and status.
Moscow, ITEP, November, 23-27,2009
Mesochronous mode
of synchronization is used within
triggering
boards to carefully align data sampling and frame generating time. This method allows to fine compensate differences in detector’s subsystem latency. Calibrating Generator boards will utilize this mode too.
Mesochronouse mode does not interfere with synchronous mode because It is not necessary to exact control for PLL phase shift. Trigger time resolution is close to Local Clock period and a programmable Offset value brings enough grade of control. PLL Phase Line RF Cavity Freq.
Link's_bitstream CLK D Q Bunch Crossing Enable MChS Signaling Layer Data/ Comm line Physical Layer PLL CLK Q D Start bit detect Counter CLK sLoad Q Triggering Board Signaling Layer Local Clock Sample Enable or Frame Align
CMD-3 DAQ hardware: features and status.
Moscow, ITEP, November, 23-27,2009 Well-known
Plesiochronous mode
the processing of a data.
of synchronization is used for Command and Data movement during
transaction
. This method is not sensitive to any phase shifts thus it does not interfere with other synchronization modes used in DAQ. Any local clock which is satisfying Nyquist oversampling criteria can be used to synchronize Com mand Event Num ber Data Word1 Data Word8 Com mand Event Num ber Data Word1 Data Wor d62 CRC DAQ Synchro sends Message Dead Time Analog signals setup time, up to 10us C-Link data/command stack Digitizer sends Data A/D conversion in progress, up to 40us Dead Time
CMD-3 DAQ hardware: features and status.
Moscow, ITEP, November, 23-27,2009 Asynchronous mode is used to collaborate with universal tools, such as an oscilloscope, pulse generator, etc. This mode signals is NIM levels. Dedicated time-to-digital converters channel allows to check asynchronous signals arrival time. It is the way to preserve good time resolution during resynchronization process. Isochronous mode of synchronization is used to change status of DAQ. The broadcast-like commands are distributed at each case DAQ need to change the status. Each board in DAQ accepts this command and execute associated Command List. It rewrites configuration registers of a Board causing a status to change. Exact moment of change depends of instruction number. It is guaranteed all Boards will change status within current transaction (i.e. within current event). This is looks like ordinary event with standard parameters, thus it can be written to tape and is suitable for off-line run processing.
Isochronous mode is also used to collaborate with legacy DAQ. It allows cross check of event number, dead time and other parameters. The Command List technology allow Boards with different internal structure to be adopted for the same kind action in DAQ.
CMD-3 DAQ hardware: features and status.
Moscow, ITEP, November, 23-27,2009 Table of synchronization modes used in CMD-3 TOMA DAQ.
Layer of C-Link specification Mode of synchronization Signaling Synchronous Signaling Mesochronous Target process Time stamp for precision time measure and DSP fast A/D conversion First Level Trigger data sampling and frame aligning Time resolution, s 10 -11 10 -9 Transaction Plesiochronous Command/Data Transmitting Event Event Asynchronous Isochronous 10 -8 Collaboration with common universal tools 10 -8 Changing of DAQ status, Data buffering, collaboration with legacy DAQ 10 -7
CMD-3 DAQ hardware: features and status.
VEPP2000 Storage Ring SND RF Cavity Freq.
Bunch Crossing Particle Injection CMD-3 Track Finder Clbr Pulser Tracker Frontend Claster Finder Clbr Pulser Calorimeter Frontend Extended Decision Clbr Pulser TOF&Mu Frontend DAQ Synchro Command, Clock to Event Builder Switch Digitizer T2Q Digitizer SAD Digitizer TQ Data Delivery Groupe 1 Data Delivery Groupe 2 CMD-3 TOMA DAQ layout Trigger Data Pipeline Synchronization Event Queue and Time Control Data Collection Status Control and Check Calibration DAQ Synchro Down Link Interface Up Link Interface CMD-3 DAQ Link Data Delivery Digitizer
CMD-3 DAQ hardware: features and status.
Moscow, ITEP, November, 23-27,2009 End-to-end testability On-line data transfer check On-line signal condition and error rate check Random simulating trigger can be added to ordinary event flow to check DAQ efficiency Calibrating events can be added to ordinary event flow to check analog channels efficiency Calibrating pattern can be added to ordinary event flow to check Trigger channels efficiency Extensive using of scalers to check signal rates All status changes is organized as standard event with standard parameters Unique ID for each Board in DAQ to support automatic topology check and data path control
CMD-3 DAQ hardware: features and status.
Moscow, ITEP, November, 23-27,2009
CMD-3 DAQ Status
All modes of synchronization are tested with full-scale DC, TOF and Trigger, no interference present Targeted resolutions are reached Calorimeters turn-on is in progress now Software for on-line control is under construction DAQ is ready for Beam Run at December 09
CMD-3 DAQ hardware: features and status.
Moscow, ITEP, November, 23-27,2009