The optical readout chain for the ALICE Transition Radiation Detector Presentation at IRTG Seminar Felix Rettig 29. Juni 2007

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 2 About me 1996Abitur in Heppenheim 1997Undergraduate Studies in Physics at University of Surrey, Guildford, UK Physics Studies at Uni Heidelberg 2002-Employment at Curagita AG Physics Diploma in Heidelberg PhD Student in Heidelberg

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 3 Outline The ALICE Experiment The Transition Radiation Detector (TRD) Optical Readout Interface Board (ORI) –features, high-speed laser modulation –error rates, irradiation test results Readout datapath components –Global Tracking Unit (GTU) and its sub-components –low-latency transmission for triggering –high-bandwidth buffering of raw data –multi-event data buffering Current Status of GTU development and assembly Outlook

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 4 ALICE Experiment Pb-Pb collisions at up to 1150 TeV 8000 collisions per second Goal: Quark-Gluon- Plasma Various detectors: TPC, TRD,... A Large Ion Collission Experiment (ALICE)

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 5 Transition Radiation Detector 1.2 million channels MCMs

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 6 Transition Radiation Detector 1.2 million channels MCMs Radiator Drift Chamber Drift Ampl. z y Cathode Pads Cathode Wires Anode Wires

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 7 TRD Front-End Electronics (FEE) Multi-Chip Module (MCM) –10-bit ADC for each of the 18 channels –up to 32 time bins, 10 MHz sampling rate ADC raw data reduction to stiff "tracklets" characterized by y-, z-position and deflection (compound 32-bit value) Filter Preprocessor Processor PASA... 18x... Analog signals from 18 cathode pads Readout Tree ADC Network Interface Data Buffers Tracklet Processor (TRAP) Multi-Chip Module (MCM) deflection time bins y Local Tracking

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 8 TRD FEE Readout Tree Readout Boards (ROB) –16 MCMs (288 channels) in 4-level readout tree Module: 2 Half-Chambers –3 / 4 Readout Boards, Half-Chamber Merger (HC-MCM) –Optical Readout Interface Board (ORI) for laser trans- mission of tracklets and raw data to GTU MCM Board Merger Readout Board z y Board Merger CCCC CCCC CCCC CC C C HC Merger ORI-Board z y 2x per Module Half Chamber Merger 4 Pad Rows 18 x 4 = 72 Channels12 or 16 Pad Rows 2 x 72 Channels Laser to GTU located outside of L3 magnet

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 9 TRD FEE Readout Tree Readout Board (ROB): –16 MCMs in readout tree –4 column mergers –1 board merger Module: –2x 4 (3) Readout Boards –2 Half-Chamber Mergers –2 Optical Readout Inter- face Boards (ORI) half chamber data concen- trated by HC Merger optical transmission from ORI to Global Tracking Unit outside of L3 magnet Board Merger CCC C CCCC CCCC CCCC HC Merger ORI-Board z y 2x per Module 12 or 16 Pad Rows 2 x 72 Channels Optical laser link to GTU located outside of L3 magnet The full TRD comprises MCMs on 4104 ROBs 540 modules in 90 stacks 1080 optical readout links

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 10 Optical Readout Interface Board HC-MCM LVDS CPLD 8B/10B Serial. Laser Driver PROM Key features: conversion 120 MHz 8-bit DDR to 125 MHz 16-bit SDR 8B/10B encoding and serialization to 2.5 Gbit/s high-speed modulation of laser driving signals 850nm Laser Diode (VCSEL) with Monitoring Diode ROB 120 MHz125 MHz2.5 GHz I²C bus Laser Diode

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 11 Optical Readout Interface Board CPLD LVDS Transceivers Laser Driver Laser Diode Connectors to Readout Board 8B/10B Encoder Serializer

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 12 Laser Diode Vertical Cavity Surface Emitting Laser Diode (VCSEL) built-in monitoring diode for power regulation feedback C I forward

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 13 Laser Modulation - Shunt Switching diff. data signal DAC + - S Temp Comp Temp Sensor ADC DAC S LOG AMP CURRENT ATTENUATOR C Temp Comp Laser Driver APC Mode ADC Servo Controller Modulation CurrentI M tc1/ 2 Servo Controller Source Current I M Gain T Nom I M Nom Peaking I M TC 1,2 I MD TC 1,2 APC Gain I M Rng I MD Nom I M Rng I MD RTRT ISIS IMIM I LD Laser Diode Monitor Diode

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 14 Laser Modulation - Eye Diagrams eye closed big parameter space: 15 setup parameters in combination with various properties of board components determine low- and high-speed characteristics of laser driving signals hard to find a common setup to be used for all 1080 boards very fast signals to be optimized, intricate to measure 8 mA Laser Forward Current 250 ps clear eye openingeye closed

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 15 Laser Modulation - Eye Diagrams eye closed huge parameter space: 15 setup parameters in combination with various properties of board components determine low- and high-speed characteristics of laser driving signals hard to find a common setup to be used for all 1080 boards signals to optimize are very fast, intricate to measure 8 mA Laser Forward Current 250 ps clear eye openingeye closed 6 mA final setup with "peaking" 125 ps

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 16 ORIs in first Supermodule at CERN Number of Optical Links Optical Power [µW] First Supermodule installed at CERN high reliability of data transmission achieved: broad safety margin ensures sufficient opti- cal power in case of laser degeneration laser diodes operating in lower region of forward current and optical power output minimum power for BER < Bit Error Rates <

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 17 ORI Irradiation Tests Irradiation tests carried out at OCL 27.7 MeV proton beam All components meet safety margin of factor 4 life-time Assumptions about TRD radiation environment: D IP ~ 1.6 Gy, D tot ~ 1.8 Gy in 10 years of operation

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 18 Full TRD Readout Chain Global Tracking Unit (GTU) Track Matching Unit (TMU) Supermodule Unit (SMU), Trigger Generation Unit (TGU) ORI Module 6x Stack Trigger Design Event Buffering Design Tracklets only Tracklets & Raw Data RX TMU 5x SMU Trigger Handling & Control GTU Supermodule Segment Front-End Electronics within L3 magnet Half Chamber DCS Board TTCrx DDL SIU DATE Software DIU D-RORC Mass Storage DAQ System Central Trigger Processor TGU 1 8 x 5x Track Concentr. GTU Racks 12 fibres Racks below Muon System Storage

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 19 ALICE Trigger Timing Time after Collision CollisionLevel-0 Trigger Level-2 Trigger Window µs µs Tracklets 6.2 µs Raw Data Data Forward to DAQ Accept/Reject Level-1 Trigger

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 20 GTU Requirements 1080 optical links deliver both –tracklets:low-latency required, small volume, high rate –raw data:big volume, latency subordinate, lower rate huge bandwidth –each opical link: 1.94 Gbit/s –links into one TMU: 23.3 Gbit/s –full detector: 2.1 Tbit/s = 244 GByte/s fast track reconstruction and decision taking for TRD's level-1 trigger contribution uni-directional optical links, no flow control –buffering of incoming raw data with full bandwidth forward buffered event data to DAQ system or discard support for trigger interlacing, buffering of multiple events

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 21 TMU Requirements low-latency deserialization and 8B/10B decoding of optical data received from front-end electronics 12 independent 16-bit data streams at 125 MHz clocks single 4-Mbit SRAM with 128-bit interface at 200 MHz multi-event buffering of up to five events, appropriate data forwarding or discarding on L2 messages error detection / handling for each separate link aborting of incomplete events, discarding associated data delicate balance between speed and ressource utilization to accomodate trigger and buffer designs in one FPGA → complex FPGA design, many clock domains, highly pipelined data path design needed to allow for operation at 200 MHz

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 22 Receiving of Optical Data optical-electrical converter modules Virtex-4 FX100 FPGA 16 Multi-Gigabit Transceivers –highly configurable, ~100 parameters –clock reconstruction –deserializing –comma detection, comma aligning –8B/10B decoding –rate matching incoming 16-bit data streams synchronized to one 125 MHz clock

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 23 Event Buffering Design 16/128 Event Shaper SRAM Controller writeread 16/128 Read Address Logic & Control Data Formatting TMU/SMU Interface Readout Unit 4-Mbit SRAM SMU Control (L0-/L1-Trigger) Scheduling Memory Management Event (n, 0) Event (n+1, 0) Event (n, 11) Event (n+1, 11) Event Info FIFO Buffer Supermodule Unit Links from one Stack... 12x... SMU Control (L2-Message) Data Block, Link 0Data Block, Link x... empty address data status Data stream Merging

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 24 Data Stream Merging collation of 16-bit values to 128-bit lines for each link padding at the end of datastreams 12 data path clock domain crossings 12 independent 16-bit data streams at 125 MHz with gaps single SRAM with 128-bit wide interface at 200 MHz 128-bit Line

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 25 Data Collation = FIFO ring counter BRAM Link Data Data Valid End Marker Line Data Last Line Line Advance Line Valid read address write address write enable Block-RAM primitives –16-bit / 32-bit collation –safe datapath clock domain crossing –buffering of filled 128-bit lines until scheduled forward to SRAM write commas discarded, only valid data words are collated write enable logic performs padding without need for extra endmarker write cycles end of data flag stored explicitly, no 32-bit compares necessary in later stages 125 MHz clock domain 200 MHz clock domain Aligning Buffer

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 26 Buffer Organization SRAM divided into separate parts, one for each link blocks independently organized as ring buffers data of multiple events dynamically double write pointers allow discarding of incomplete events, directly freeing occupied memory locations overrun protection, anticipatory busy signaling 4-Mbit SRAM Event (n, 0) Event (n+1, 0) Event (n, 11) Event (n+1, 11) Data Block, Link x... empty memory Data Block, Link 0 rp end (n) end (n+1) rp end (n) end (n+1)

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 27 Buffer Address Management Event n -2 Event n-1 Event n (incomplete) + + Write Pointer A 1 Block RAM Bank B Bank A write address read address 1 0 R 18 SRAM write address offset Data Block, Link 0 Read Pointer AB Write pointers Write Pointer A 0 Write Pointer B 1 Write Pointer B x... base address event end pointer 0 2 write pointers –first word offset –current offset normal operation: –first constant –increment current event complete: –pointers swap meaning discard incomplete event: –use first word pointer, copy to current pointer only one RAM block needed for all pointers, no 12:1 multiplexers

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 28 Event Buffering Design - Detailed

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 29 Event Buffering Design - Detailed

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 30 Event Buffering Design - Detailed

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 31 Event Buffering Design - Detailed

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 32 TMU FPGA Design Current TMU design: –ressources produce new pictures!

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 33 CERN Tests

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 34 Tests and Integration Progress GTU segment running in Münster for super-module assembly and testing all segments being assembled and tested currently transfer to CERN and installation scheduled for end of july some CTP signal handling and DDL/DAQ issues open trigger design not yet integrated

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 35 Prospective Work Finish development of GTU –a lot of open detail issues –multi-event buffering under full CTP control Installation at CERN Beam and performance tests, refinements Monitoring system, GTU event display Physics applications of GTU –various trigger schemes based on comprehensive track and p t information, e.g. Jet Triggers

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 36 Thank You for Your Attention Questions?

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 37 TMU Board - Final Layout

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 38 SMU Board - Final Layout

Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar p. 39 TMU-SMU-Hybrid