Modelisation of SuperB Front-End Electronics Christophe Beigbeder, Dominique Breton, Jihane Maalmi

Moving from BABAR to SuperB We want the global system architecture to be synchronous The idea of directly addressing data in the ring buffer (asynchronous model) could have looked appealing but: it makes the system more complex and harder to commission it implies counters running synchronously at numerous different locations (images of the FEE have to run in the FCTS, potentially at frequencies below 56MHz) it doesn’t permit detecting easily a loss of synchronization Due to machine and detector constraints, trigger latency is rather long and has a certain jitter, this leading to a rather large (1µs) time window for data readout. Latency should be reduced thanks to modern FPGA’s Window could be reduced and detector dependent to optimize the dataflow It should be fixed but programmable in the FEE With the radiation, the simpler = the better for the FEE. No unnecessary complexity has to sit there A shorter L1 trigger latency would decrease the latency buffer size Dominique Breton, Jihane Maalmi - Orsay SuperB Workshop – February 16th 2009

Constraints for the FEE model There is no reason to fix a minimum distance between consecutive triggers at the architecture level (except very small values). Neither to specifically limit their number in a burst (this will come from dataflow). => those constraints should only depend on the trigger system itself. Due to the time precision of trigger primitives, a minimum distance of about 100ns between triggers is highly probable. This leads to two different types of problems : Two consecutive physics events may reside within the trigger time window (overlapping). For detectors with slow signals (like EMC barrel), physics events may sit on the queue of large background events (pile-up). FEE should thus be able to deal with close triggers, and send data in consequence overlapping events will be treated in such a way as to simplify the FEE and provide full and clean events to HLT and DAQ. For this purpose, useful event data will only be sent once between FEE and ROM, and truncated event data will be restored in the ROM before events be further processed. this limits the bandwidth necessary, as well as the event_buffer size in the FEE Dominique Breton, Jihane Maalmi - Orsay SuperB Workshop – February 16th 2009

Possible SuperB electronics implementation FCTS Max jitter: ~ 10-15ps rms => links ~ 50-100 ps rms => physics & no phase spread 56MHz clock L1 accept Cal_strobe Multi-Gbits/s Optical links FEE Command decoder 56MHz clock Cal_strobe FE_reset FE_reset S E R D ROM Setup and control L1 accept DAQ L1 buffer Detector data Setup and control One 32-bit command word every clock cycle Subsystems control Dominique Breton, Jihane Maalmi - Orsay SuperB Workshop – February 16th 2009

Simulation of synchronous model for L1 buffer The FCTS sends a L1 trigger command optionally associated with a value corresponding to a time window. The FEE sends to the DAQ the data contained inside a readout window, embedded in a frame including status, trigger tag or time, and length of data field. Trigger is defined by three parameters: - The latency: L (fixed in the FEE) - The readout window: W (fixed in the FEE or event dependent) - The time distance between triggers: D (measured in the FEE) Constraints : - No dead time - Triggers with overlapping windows - Dealing with going back in time (explained farther) - Triggers with optional variable width windows

Parameter Definition for Synchronous Model L1 Trigger #0 Data to keep Data to dump L W Time M Baseline: latency pipeline always provides the oldest relevant data L: fixed latency W: window containing the relevant data for trigger #0 M: data sent to ROM Dominique Breton, Jihane Maalmi - Orsay SuperB Workshop – February 16th 2009

Synchronous Model: fixed readout window (1) L : Latency W : Window D : Distance between triggers M : data sent to ROM Trigger #0 Trigger #1 Case 1 : D ≥ W D L D ≥ L W W M0 M1 Non overlapping latencies with 2 different windows (green): no problem M1 = W Trigger #0 Trigger #1 Or D Overlapping latency with non overlapping windows. Still straightforward. M1 = W D < L W M0 W M1 Trigger #0 Trigger #1 Case 2 : D < W D Overlapping latency trigger with overlapping windows: trickier … The window W1 is then shortened! M1 = W – (W – D)= D W M0 W M1 Dominique Breton, Jihane Maalmi - Orsay SuperB Workshop – February 16th 2009

Synchronous Model: Fixed readout window (2) Case 1 : Dn ≥ W : Mn = W Case 2 : Dn < W : Mn = Dn Mn : amount of data to send to ROM for trigger #n Trigger input Counter Dn Dn ≥ W? Clock 56 MHz W Fifo “M” !empty M U X Mn FSM W end enable W Mn Counter Registers L All 56 MHz synchronous pipelined operations Start_flag, Mn to serializer Wr_en Data input Latency Pipeline EVT_BUFFER L Dominique Breton, Jihane Maalmi - Orsay SuperB Workshop – February 16th 2009

Synchronous Model : Problem with Physics on a Bhabha’s tail (not triggered) physics Trigger W’ L W M W’: extra window to go back in time Dominique Breton, Jihane Maalmi - Orsay SuperB Workshop – February 16th 2009

All 56 MHz synchronous pipelined operations Synchronous Model : Fixed readout window with back jump Case 1 : Dn ≥ W : Mn = W ( + W’) Case 2 : Dn < W : Mn = Dn ( + W’) Mn : amount of data to send to ROM for trigger #n Go_back_in_time Go_back_in_time Trigger input Latency W’ Dn Counter Dn ≥ W? W Fifo “M” !empty M U X M U X Mn FSM W end A D enable W’ Mn Counter All 56 MHz synchronous pipelined operations registers L Start_flag, Mn, go_back W to serializer Wr_en Data input Latency Pipeline EVT_BUFFER L + W’ Dominique Breton, Jihane Maalmi - Orsay SuperB Workshop – February 16th 2009

Synchronous Model : Physics on a Bhabha’s tail itself on physics tail … (not triggered) They saw me!! and me too! physics physics Trigger x Trigger W’ L W W M D D = W + W ’- x M = (W’-x) +W= D – W + W = D M = D … still works! Dominique Breton, Jihane Maalmi - Orsay SuperB Workshop – February 16th 2009

Synchronous Model : variable readout window (1) L : Latency W : Window D : Distance between triggers M : data sent to ROM Trigger #0 Trigger #1 Case 1 : D ≥ W0 D L D ≥ L W0 W1 M0 M1 Non overlapping latencies with 2 different windows (green): no problem M1 = W1 Trigger #0 Trigger #1 Or D D < L Overlapping latency with non overlapping windows. Still straightforward. M1 = W1 W0 M0 W1 M1 Trigger #0 Trigger #1 Case 2 : D < W0 D Overlapping latency trigger with overlapping windows: trickier … The window W1 is then shortened! M1 = W1 – (W0 – D)= W1- W0 + D W0 M0 W1 M1 Dominique Breton, Jihane Maalmi - Orsay SuperB Workshop – February 16th 2009

All 56 MHz synchronous pipelined operations Case 1 : Dn ≥ Wn : Mn = Wn (+ W’) Case 2 : Dn < Wn : Mn = Dn + Wn – Wn-1 (+ W’) Synchronous Model : Variable readout window (2) Go_back_in_time Problem ! Mn could be negative … Trigger input Go_back_in_time Latency W’ Dn Wn Counter Dn ≥ Wn? Fifo “M” Wn !empty M U X M U X Wn Mn FSM S U B A D Dn+Wn-Wn-1 end Wn-Wn-1 A D D Wn-1 enable W’ Mn Counter All 56 MHz synchronous pipelined operations registers L Start_flag, Mn,Wn,go_back to serializer Data input Latency Pipeline Wr_en EVT_BUFFER L + W’ Dominique Breton, Jihane Maalmi - Orsay SuperB Workshop – February 16th 2009

Finite State Machine details rd_fifo = 0 en_counter = 0 St0 fifo_empty = 0 St1 rd_fifo = 1 en_counter = 0 end_counter = 1 & fifo_empty = 0 end_counter = 1 & fifo_empty = 1 St2 Outputs : rd_fifo en_counter + start_flag … Inputs : fifo_empty end_counter rd_fifo = 0 en_counter = 0 St3 rd_fifo = 0 en_counter = 1 end_counter = 0 & fifo_empty = 1 end_counter = 0 & fifo_empty = 0 rd_fifo = 1 en_counter = 1 St4 end_counter = 1 end_counter = 0 rd_fifo = 0 en_counter = 0 St7 end_counter = 0 St5 end_counter = 1 St6 rd_fifo = 0 en_counter = 1 rd_fifo = 0 en_counter = 1

FSM case 1: D ≥ W + 3 (standard case) rd_fifo = 0 en_counter = 0 St0 fifo_empty = 0 St1 rd_fifo = 1 en_counter = 0 end_counter = 1 & fifo_empty = 1 St2 rd_fifo = 0 en_counter = 0 St3 end_counter = 0 & fifo_empty = 1

FSM Case 2 : D = W + 2 St0 St1 St2 St3 fifo_empty = 0 rd_fifo = 1 en_counter = 0 end_counter = 1 & fifo_empty = 0 St2 rd_fifo = 0 en_counter = 0 St3 rd_fifo = 0 en_counter = 1 end_counter = 0 & fifo_empty = 1

FSM Case 3 : D = W + 1 St0 St1 St2 St3 St4 St7 St6 rd_fifo = 0 en_counter = 0 St0 fifo_empty = 0 St1 rd_fifo = 1 en_counter = 0 end_counter = 1 & fifo_empty = 1 St2 rd_fifo = 0 en_counter = 0 St3 rd_fifo = 0 en_counter = 1 end_counter = 0 & fifo_empty = 1 end_counter = 0 & fifo_empty = 0 rd_fifo = 1 en_counter = 1 St4 end_counter = 1 rd_fifo = 0 en_counter = 0 St7 St6 rd_fifo = 0 en_counter = 1

FSM Case 4 : D ≤ W ( overlapping) rd_fifo = 0 en_counter = 0 St0 The model works down to D = 2 clock cycles (36 ns) fifo_empty = 0 St1 rd_fifo = 1 en_counter = 0 end_counter = 1 & fifo_empty = 1 St2 rd_fifo = 0 en_counter = 0 St3 rd_fifo = 0 en_counter = 1 end_counter = 0 & fifo_empty = 1 end_counter = 0 & fifo_empty = 0 rd_fifo = 1 en_counter = 1 St4 end_counter = 0 rd_fifo = 0 en_counter = 0 end_counter = 0 St5 end_counter = 1 St6 rd_fifo = 0 en_counter = 1 rd_fifo = 0 en_counter = 1

Verilog behavioral simulation results (1) 1- General view 2- single trigger

Verilog behavioral simulation results (2) 3- Overlapping case 4- Go back in time case 5- Go back in time case + overlapping windows

Verilog behavioral simulation results (3) 6- Overlapping burst 7- Overlapping triggers both with go back in time!

Conclusion about synchronous model This model seems to cover all the requirements. Assuming that the L1 central trigger processor is able to easily produce triggers with a “fixed” latency: - Like in Babar, the only information to send to the FEE is the trigger tag (a few bits). - Even with a fixed readout window, this model is able to deal with event overlapping and backing up in time. - It also permits working with a variable window, but it makes it more complex and difficult to understand and debug. Moreover, it will make the trigger system itself much more complex. So if the variable readout window is not necessary for physics, there is no reason to implement it.

System level implementation of synchronous model L1 central processor Only constraint : trigger latency is almost fixed (BABAR like) Inter-trigger minimum spacing linked to detector precision (142ns, 71ns ? ) All links : multi-Gbits/s optical links FCTS Production of trigger tag (56 bits) Synchronous Model runs autonomously at 56MHz Short command à la BABAR 104 bits still OK (4 clock periods => 72ns) ROM FEC DAQ Optimized bandwidth (with trigger tag and event length) Restores the event from the FEE information Dominique Breton, Jihane Maalmi - Orsay SuperB Workshop – February 16th 2009

Conclusion DAQ and trigger are based on BABAR experience but must evolve to cope with the new level of the requirements We want the system to be synchronous Experience of LHC experiments and others will help building the new design We started studying the synchronous front-end model for the L1 buffer A behavioral Verilog model has been built. It looks like it copes with all the requirements. Should be farther studied thanks to the software presented by S. Cavaliere in the next talk. Could the control and L1 buffer block be delivered as a mezzanine on the FEE ? Reminder: Clock and control distribution have to be rethought => we need to develop our own solution if possible (see talk of A. Aloisio) ROM and L1 trigger processors have to be redesigned => there is a real need for new collaborators to work on DAQ and L1 trigger hardware (there seems to be people interested therein … ) Dominique Breton, Jihane Maalmi - Orsay SuperB Workshop – February 16th 2009