1. CMS ECAL A new readout system architecture for the CMS ECAL Magnus Hansen 20030930

2. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch Agenda l Short history l rEvolution l A new readout system architecture l A New ASIC: FENIX l Front End Card l System Test l Conclusion

3. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch Agenda l Short history l rEvolution l A new readout system architecture l A New ASIC: FENIX l Front End Card l System Test l Conclusion New System Architecture Chips / ASICs Electronics Modules System

4. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch Old Design Architectural Overview ROSE FE DCC Partition Regional Trigger Data DAQ Data TTS TTC Local Triggers In Stand alone Mode

5. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch rEvolution l Simple Front End s Minimal hardware l Large number of optical link s 1 link per channel l Large upper level readout system s Maximal flexibility (FPGA tech.) l Estimated not to be affordable l Developed Front End s Trigger Primitive generation s Primary event storage l Modest number of optical link s 2 data links per tower (25 ch) l Modest upper level readout system l Estimated to be affordable

6. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch New architecture l Implemented in Front End s MGPA + Multi-ADC for dynamic range compression and digitization (Change from analogue gain switching to digital gain selection) s TPG (Trigger Primitive Generator) s Pipeline storing digitized data waiting for level 1 trigger decision s Primary event buffer l Implemented in Counting room s CCS (Clock and Control System card, Collaboration with CMS Tracker, Pixel) s DCC (Data Concentrator Card) s TCC (Trigger Concentrator Card) s SRP (Selective Readout Processor)

7. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch Architectural Overview TCCCCS FE DCC Partition Trigger Data DAQ Data TTS TTC Local Triggers In Stand alone Mode Regional Trigger Data DAQ Data SRP

8. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch Front End System Functional Requirements l Trigger Primitive Generation s Absolute calibration of each channel s Implement existing well defined algorithm è Verification needed before production è No future basic changes possible s Latency budget imposed l Readout of data corresponding to positive trigger decision s Dead time free è Three clocks / trigger imposed by TTC system s 100 kHz level 1 trigger rate è Some trigger rules apply è Overflow protection s Programmable level 1 trigger delay è Pipeline of programmable length l Support for monitoring s Laser monitoring, temperature measurements, etc.

9. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch Other Requirements and Constraints l Debugging and testability features s Pattern injection è Possibility to inject known pattern in the beginning of the trigger primitive generation and the readout chain s Boundary scan / scan chains s BIST è Built In Self Test for production test and in situ l Radiation environment l Size s Have to fit behind served crystals l Short development time s Start June 2002 s Full production January 2004 s One advantage – knowledge of the requirements

10. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch The Front End - A Readout Cube l Motherboard s Creating flat surface for electronics installation s Kapton cable to APD connector l VFE card s Analogue Signal Processing s Digitization l LVR card s Voltage regulation for FE system l FE card s Digital Signal Processing è Trigger Primitive generation s Temporary storage è Pipeline, event buffer l GOH s Complete Optical transmitter module including a GOL and a laser diode

11. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch A technical choice i) l Single Front End ASIC l O(500) IOs s Huge chip s ~20 by 20 mm l O(4000) chips in CMS ECAL l Quickly considered as a non-optimal choice

12. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch A technical choice ii) l Seven Front End ASICs l Three types l O(150) IOs each s ~7 by 7 mm l O(20000) + O(4000) + O(4000) chips in CMS ECAL l Soon considered as a non- optimal choice

13. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch The technical choice l Seven Front End ASICs l Single type l Three operation modes l O(150) IOs s ~7 by 7 mm l O(30000) chips in CMS ECAL l Considered as an optimal choice l The new ASIC is called FENIX s Front End New Intermediate data eXtractor

14. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch FENIX Description 1: Four Operation Modes l Strip s Creating filtered Strip / Pseudo-strip sums for TCP inputs s Pipelines and primary event buffers l TCP s Trigger Cell Processor s Finalising the trigger primitive for one trigger tower in the Barrel l DAQ s Tower (Super Crystal) readout state machine s Event encapsulation l MEM s Reading out the Laser monitoring monitoring system s Pipelines and primary event buffers

15. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch FENIX Description 2: Control l Fast control s T1 signal as defined in the Tracker slow control system è “100” => Level 1 Trigger accept è “101” => BC0 øLocal Bunch Crossing counter reset è “110” => Re-synch øReset of all counters and state machines è “111” => Force VFE mode øFrom programmable default mode to programmable calibration mode è “110110” => Power-up reset øReset of all counters and state machines and load default values into all registers è “1100110” => Power-up reset øAs above l Slow Control s I2C interface è Extended 10 bit addressing øStandard protocol øDirect addressing of all set-up addresses è Compatible with CCU I2C master ports è Fully synchronous design øSynthesizable øAuto P&R s 150 set-up addresses è 132 Set-up registers è 18 RAM access

16. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch Development Acceleration l Intermediate device s Xilinx FPGA s Cadence for simulation s Synplify for synthesis l Modern ASIC design tools s Synopsis for synthesis s Silicon Ensemble for Place & Route s Very short design turn- around time è 2 weeks claimed l Generic HDL description s No component instantiation

17. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch ASIC Emulation in FPGA l Features s Observable functionality identical s Identical footprint s Identical pin-out l Not implemented to save resources s Triple-redundancy in registers s Error Correcting Code in RAMs s BIST in RAMs

18. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch ASIC Emulation in FPGA - Applied HDL design rules l Generic Source Code s No process dependent component instantiation l Exception: RAM s Technology specific, recommended not to infer l Adopted strategy: s All functional simulation done with generic RAM è “Superset” of Xilinx RAM and ASIC RAM used s For the FPGA, the Xilinx RAM block is wrapped and instantiated è Routed design simulated and verified for conformity s For the ASIC, the modular static RAM cell developed at CERN is wrapped and instantiated è Routed design simulated and verified for conformity

19. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch FENIX ASIC Radiation Tolerance: Strategy l Observation s ASIC technology is radiation tolerant s Registers and RAM cells subject to SEU l Strategy s Protect against SEU s Not protect against hardware failure l Testability s Always a challenge s Improved by insertion of a “testability flag”

20. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch FENIX ASIC SEU Tolerance 1: Set-up Registers l Set-up registers s Triple-redundant flip-flops è SEU resistant øVoting logic øthree (two) clocks long write pulse needed s Features è Synthesizable s Test è Any discrepancy flagged è Discrepancy when written

21. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch Triple-redundant Set-up register VHDL code setup_register : process (clock40) begin if rising_edge(clock40)then if pwup_reset = '0' then register1_8b <= pwup_value; seu_flag <= '0'; elsif address = register_address and write_enable = '1' then register1_8b <= write_data; else register1_8b <= voted_register_value_8b; end if; register3_8b <= register2_8b; register2_8b <= register1_8b; if register1_8b = register2_8b and register1_8b = register3_8b then seu_flag <= '0'; else seu_flag <= '1'; end if; else null; end if; end process setup_register;

22. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch FENIX ASIC SEU Tolerance 2: State Machine l State Machine s Triple-redundant è SEU resistant øVoting logic øExcept state changes s Features è Synthesizable s Test è Any discrepancy flagged è Discrepancy when state change

23. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch FENIX ASIC SEU Tolerance 3: RAM l RAM s Hamming code è Encode at write è Decode & Correct at read øone bit error correction s Features è Synthesizable è Single bit SEU safe s Test è ECC Decoder øDuring BIST execution è ECC Encoder øDuring normal operation øThrough slow control (I2C)

24. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch FENIX ASIC Testability l Test time budget: 1 second s Without chip handling l Triple-redundant Registers s Testability flag can be used as a signature of operation l RAM BIST s Fully automatic s Write whole RAM and read back s Launched by a pulse on IO pin è Boundary scan è Tester s Observable on external pins è Boundary scan è Tester s Can be launched and monitored in situ (I2C)

25. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch FENIX ASIC Status and Plans l First submitted February 2003 s Received back from foundry in May s Not yet received back from packaging è Becoming critical l Next submission after ESR s 9 th of October s Engineering run è Final design è O(3000) dies, for up to 3 CMS ECAL Super Modules s Tested chips back before end 2003 l Production s Beginning of 2004

26. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch FE Card Functionality and Performance l Full TPG in Barrel s Sum of Five filtered strip sums s Single data link to TCC and regional trigger s 11 clocks latency è FE card input to GOH connector l Partial TPG in End Cap s Five filtered strip sums s Five data links to TCC and regional trigger s 7 clocks latency è FE card input to GOH connector l Readout s Serves 25 channels s Single data link to DCC and DAQ s Dead time free è 7.2us service time, 25 primary event buffers, (10 samp/ch/evt, P(n=d) = 10 -8 *) è Null event insertion up to 127 pending events s Programmable pipeline length corresponding to level 1 trigger delay *TTS for CMS DAQ, A. Racz

27. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch FE card Layout

28. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch Tower in Super Module Trigger GOH CCU FENIX FPGA QPLL VFE LVR Readout GOH

29. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch Beam Test 2003

30. LECC 2003 September 30th 2003M. Hansen, CERN. magnus.hansen@cern.ch Conclusion l A New Readout System Architecture for CMS ECAL has been presented l The CMS ECAL Front End Card s Serving 25 readout channels è Tower in the Barrel è Super-Crystal in the End Cap l The FENIX chip s Implements the main functionality on the Front End card s Three (four) operation modes s FPGA emulator implemented s ASIC prototype implemented, Final design submitted after ESR l Prototype system successfully tested in beam