Level-1 Data Driver Card (L1DDC) HEP May 2014 Naxos 08/05/2014HEP 2014, NAXOS Panagiotis Gkountoumis National Technical University of Athens 1

L1DDC Level-1 Data Driver Card Responsible for – collecting L1 data from FE boards and sending them to the counting room – Receiving Slow Control (SC) information – receiving configuration data from the counting room and sending them to the FE boards – sending timing control (TTC) signals to the FE boards – sending configuration data to the ADDC (Address Real Time) boards 08/05/2014HEP 2014, NAXOS2 On-Detector Radiation Hard Electronics Off-Detector Commercial Off-The-Shelf (COTS) GBTX GBTIA GBLD PD LD Custom ASICs Timing & Trigger DAQ Slow Control Timing & Trigger DAQ Slow Control FPGA GBT Versatile Link

FE Connectivity for MMs 08/05/20143HEP 2014, NAXOS L1DDC FE Optical link ADDC

Board dimensions 100mmx50mmx18mm (LengthxWidthxHeight) On detector – hostile radiation environment Contains 3 radiation hard ASICs – GBTx (GigaBit transceiver) – GBTIA (Trans-Impedance Amplifier) – GBLD (Laser Driver) Input voltage 24V 3 Voltage levels – 3.3V – 2.5V – 1.5V Linear LTM4619 DC-DC converter – 4.5V to 26.5V input – Dual output 0.8V to 4Amps each Linear LT3080 LDOs – Wide input voltage 1.2V to 36V – (Dropout Voltage 350mV) Power consumption ~3.3Watts Connectivity – VTRX (CERN’s custom made optical 4.8Gb/s – 8 Mini SAS 26 pins cables for the FE connection 80Mb/s, 160Mb/s, 320Mb/s) – 1 Mini SAS 26 pins cable for the ADDC connection 80Mb/s) – Low Voltage Power connector 08/05/2014HEP 2014, NAXOS Parts and connectivity 4

GBTx Gigabit Transceiver ASIC – highly flexible link interface chip Can be configured as: – Bidirectional transceiver – Unidirectional transmitter – Unidirectional receiver Very high level of error correction from SEU’s and transmission errors Radiation Hard ASIC - 130nm CMOS technology Fits in 17mmx17mm BGA package (400pins), 1mm pitch 7 power domains – Serializer (1.5V) – Deserializer (1.5V) – Clock manager (1.5V) – Phase shifter (1.5V) – Core digital (1.5V) – I/O (1.5V) – Fuses (3.3V) Max power consumption is 2 Watts (1.33Amps when all on) Electrical caracteristics – Drivers: SLVS signaling (V CM =200mV, +/-200mV) – Receivers: SLVS/LVDS signaling (V CM =1.25mV, +/-400mV) 08/05/2014HEP 2014, NAXOS5

FE Module GBTx Duplex local electrical serial links (E-Links) 1 elink consists of 3 signal lines (differential pairs). – Data down – Data up – Clock GBTx supports up 40 E-Links 8 E-Links/group E-Link data rates are programmable per group – – – E-Link clock signals independently (40MHz, 80MHz, 160MHz and 320MHz) 1 dedicated 80 Mb/s for SCA E-Links signal drivers and receivers not actively used can be powered down 08/05/2014HEP 2014, NAXOS6 FE Module Phase – Aligners + Ser/Des for E – Ports E – Port GBT-SCA E – Port data One 80 Mb/s port GBTX e-Link clock data-up data-down clocks E – Port FE Module E – Port

E-Links 08/05/2014HEP 2014, NAXOS7 VMM Com/ nion SCA VMM FE RXCLK RXDATA TXDATA RXCLK RXDATA TXDATA Companion E-Link SCA E-Link VMM FE contains 2 ASICS Companion ASIC - Collects data from VMMs and transmits them to L1DDC SCA (Slow Control Adapter) - Configures the VMMs and monitors the voltage levels L1DDC is connected to each ASIC with 1 E-Link 2E-Links/FE - 6 Lines (6 differential pairs)

E-Links Group1-4 can 80Mb/s, 160Mb/s and 320Mb/s Group5 is used for the SCA connections and always 80Mb/s The dedicated 80Mb/s can be used for the ADDC configuration data 08/05/2014HEP 2014, NAXOS8 GROUP1 C.ASIC 3 4 SCA … 1 8 GROUP2 GROUP3 GROUP4 GROUP5 C.ASIC 1 2 ADDC C.ASIC Dedicated

Mini SAS Serial Attached SCSI (SAS) Commercial cables – Round – Flat, twin axial 26 positions Shielded 8 differential pair Mini SAS 26 pins connector with cage Fixed cable length for all the FEs ( AWG 28 or 30) 08/05/2014HEP 2014, NAXOS9

Optical transceiver 08/05/2014HEP 2014, NAXOS GBTIA 0.05Amps -Power consumption 0.12Watts -Bit rate 5Gb/s -Die size: 0.75 mm × 1.25 mm μm IBM CMOS8RF- LM technology, a standard eight-metal- layer GBLD -Bit rate 5Gb/s 0.2Amps -Power consumption 0.5Watts 10

GBT System 08/05/2014HEP 2014, NAXOS11 Clock & Data Recovery. Recovers and generates a high speed clock to sample the incoming data stream Programmable in frequency and phase Decoder / Descrambler Scrambler / Encoder Serializer

Packet format (optical link) Fixed packet length: 120bits – Packet transmission rate: 1/25ns – Data transmission rate: 4.8 Gbps Header field 4 bits Forward Error Correction (FEC) field of 32bits is added resulting in a code efficiency of 70% 84bits for data transmission corresponding to 3.36Gb/s user bandwidth SC field 4 bits – Internal Control (IC) field 2 bits – External Control (EC) field 2 bits Finally only 80 bits are used for data 3.2Gb/s 08/05/2014HEP 2014, NAXOS12

Packet format (optical link) 8b/10b frame format 12 x (8b/10b) words The first one is the comma character 88/120=73% coding efficiency 3.52Gb/s user bandwidth No error correction and limited error detection Wide frame format 4.48Gb/s user bandwidth No SEU protection 08/05/2014HEP 2014, NAXOS13 8b/10b frame format Wide frame format

Encoding/decoding block diagram 08/05/2014HEP 2014, NAXOS14 Scrambler performs DC balance Double interleaved Reed-Solomon two-errors correcting code A sequence of up to 16 consecutive corrupted bits can be corrected Transmission errors or SEUs occurring in the Serializer (SER), GBLD, PIN-diode, GBTIA, Clock and Data Recovery (CDR) and De- serializer (DES) will be corrected in the decoding operation Encoding & decoding can be done in a single machine clock cycle – minimal impact on the transmission latency

Position of L1DDC in MMs FEs L1DDC 08/05/2014HEP 2014, NAXOS15 ADDC LM2 LM1 Requirement: not to lose more than one plane in case of L1DDC failure Every L1DDC is connected with 8 FEs of the same side and plane Fixed length of cables to all FEs Placement of L1DDC and ADDC only in the first plane

Position of L1DDC 08/05/2014HEP 2014, NAXOS16 FEs

Ground pins L1DDC prototype 08/05/2014HEP 2014, NAXOS17 GBTx LTM4619 Ground pins Mini SAS 26p

L1DDC prototype 08/05/2014HEP 2014, NAXOS18 Power

Calculations MMs – Per plane 2 L1DDC/16 FEs – Per Wedge 8 L1DDC/64 FEs – Per Sector 16 L1DDC/128 FEs – Per 1 Wheel 256 L1DDC/2048 FEs – Per 2 Wheels 512 L1DDC/4096 FEs 08/05/2014HEP 2014, NAXOS19 Sum 1024 L1DDC & 3.38KWatts TGCs – Per plane 2 L1DDC/6 FEs – Per Wedge 8 L1DDC/24 FEs – Per Sector 16 L1DDC/48 FEs – Per 1 Wheel 256 L1DDC/ 768 FEs – Per 2 Wheels 512 L1DDC/1536 FEs

Backup Slides 08/05/2014HEP 2014, NAXOS20

Available space 08/05/2014HEP 2014, NAXOS21