R&D Studies of the ATLAS LAr Calorimeter Readout Electronics for super-LHC Hucheng Chen On behalf of the ATLAS LAr Calorimeter Collaboration March 14th, 2009

ATLAS LAr Electronics Upgrade Outline ATLAS Liquid Argon Calorimeter Current LAr Readout Electronics LAr Electronics Upgrade Motivations R&D Studies Front-end mixed-signal ASIC design BNL, Columbia Univ., Univ. of Penn., INFN Milan, IN2P3 Radiation resistance optical link in SOS SMU High speed back-end processing unit based on FPGA Level 1 calorimeter trigger interface BNL, Univ. of Arizona, SUNY Stony Brook, IN2P3 LAPP, INFN Milan, Dresden, CERN Power supply distribution scheme BNL, Yale, INFN Milan, Univ. of Milan Summary 03/14/2009 ATLAS LAr Electronics Upgrade

ATLAS Liquid Argon Calorimeter (LAr) Barrel calorimeters in detector hall, surrounded by barrel toroids Liquid Argon Calorimeter Electromagnetic Barrel (EMB) |η|<1.375 [Pb-LAr] Electromagnetic End-cap (EMEC) 1.4<|η|<3.2 [Pb-LAr] Hadronic End-cap (HEC) 1.5<|η|<3.2 [Cu-LAr] Forward Calorimeter (FCAL) 3.2<|η|<4.9 [Cu,W-LAr] Front End Crate 03/14/2009 ATLAS LAr Electronics Upgrade

Current LAr Readout Electronics ~180,000 detector channels Front-end Electronics 58 Front End Crates 1524 Front End Boards 58 LV Power Supplies ~1600 fiber optic links between FE and BE Back-end Electronics 16 Back End Crates 192 Read Out Driver Boards 68 ROS PCs ~800 fiber optic links between ROD and ROS 03/14/2009 ATLAS LAr Electronics Upgrade

LHC Upgrade Expectation LHC upgrade includes 2 phases sLHC (phase 2 upgrade) expected to start up ~2017 ATLAS LAr calorimeter plans for LHC phase 2 upgrade New requirements to LAr calorimeter readout electronics Radiation environment assumed to scale x10 Total power consumption kept same Luminosity plot before LHC accident New injectors + IR upgrade phase 2 sLHC Start-up Early operation Linac4 + IR upgrade phase 1 Peak Luminosity: 2017: 3x1034 cm-2s-1 Integrated Luminosity: 2008-17: 650 fb-1 2013-17: 550 fb-1 03/14/2009 ATLAS LAr Electronics Upgrade

Current Front-end Architecture FEB Complexity 11 ASICs Several technologies Obsolescence of technologies (e.g. DMILL) 19 voltage regulators Analog pipelines (SCA) ~80W/board, water cooled Radiation/lifetime issues Qualified for 10 years LHC operation Limited number of spares (~6%) Other limitations L1 trigger rate <= 100kHz L1 trigger latency <= 2.5µs Consecutive L1 trigger spaced more than 125ns FEB Upgrade Component-level replacement impossible Full replacement based on current technologies No increase of power budget 03/14/2009 ATLAS LAr Electronics Upgrade

Proposed Front-end Architecture FEB “strawman” architecture keeps many options open Shaping and gain settings Analog .vs. digital pipeline On/off detector pipeline Analog .vs. digital gain selector Possibly provide analog trigger sums to decouple potential trigger upgrade FEB upgrade propagates to other boards Digitization at each bunching crossing ~100Gbps/board Higher speed, higher radiation resistance optical link LV power supplies Back-end electronics Possibly interface to L1calo digitally 03/14/2009 ATLAS LAr Electronics Upgrade

Test Structure Chiplet Analog Front-end R&D An increasing attention to SiGe technology for fast detector readout and in applications where radiation is a concern ILC R&D program in Europe Cross sub-system effort in ATLAS Inner Detector: UCSC, Barcelona LAr: BNL, Columbia Univ., Univ. of Penn., INFN-Milan, IN2P3 Characterization of IBM 8WL 0.13μm SiGe BiCMOS Foundry provided samples Gamma irradiation @ BNL Neutron irradiation @ Ljubljana, UMass Lowell Proton irradiation @ Boston (Mass. Gen. Hospital Cyclotron) IBM 8WL design kit parts Test structure Chiplet submitted on Nov. 10th, 2008 8WL design kit transistor test devices 8WL CMOS transistor array Test Structure Chiplet 2.7mm X 1.8mm 03/14/2009 ATLAS LAr Electronics Upgrade

Analog Front-end: Preamp & Shaper (RC)2 10X Shaper (RC)2-CR Preamp out 0-700μA into preamp Prototype LAr Preamp and Shaper Submitted to MOSIS on Nov. 10th, 2008 Preamp Very low noise ~0.25nV/√Hz, high dynamic range Based on low noise line-terminating preamplifier circuit topology presently used in ATLAS LAr calorimeter Shaper Fully differential Robust performance in low signal environment, excellent pickup rejection on and off chip Explored several low power non feedback approaches, none satisfied noise and linearity simultaneously LAr Chiplet 2mm X 2mm 03/14/2009 ATLAS LAr Electronics Upgrade

Mixed-signal Front-end: ADC ADC is the most technologically challenging component in the new “baseline” architecture 15(6) bit dynamic range - 12 bit resolution - 40MSPS Radiation tolerant and SEE immune Strategies being followed Find the magic bullet with commercial parts Planning to test several COTS ADCs Probably not meeting radiation requirement because bit error correction stored in rad-soft RAM Developing a custom ADC IBM 8RF CMOS as possible candidate technology Already shown to be radiation hard Lower cost compared to SiGe Lots of CERN based experience 03/14/2009 ATLAS LAr Electronics Upgrade

ATLAS LAr Electronics Upgrade Pipeline ADC R&D Pipelined ADC with digital correction Each stage resolves more bits than subsequent amplification Sample & Hold capacitor ultimately also a limiting factor because noise ~ sqrt(kT/C) Amplifier at stage output is critical Each stage is 1.5 bits Overlap between stages allows correction before outputs are “added” together If correction done on board it will need to store calibration constants in redundant rad-hard memory ADC development status Expected to need bipolar technology, started in IBM 8HP SiGe BiCMOS Decided to redesign in IBM 8RF CMOS due to cost and CERN-based experience Conversion to 8RF completed and back to target spec (12bit) Plan to submit a MOSIS test structure proposal by this June 03/14/2009 ATLAS LAr Electronics Upgrade

Digital Logic Initial Design 2 Independent clocks “TTC”-like clock for ADC and ADC multiplexer Crystal derived clock for high speed components (MUX, serializer) provides much better jitter control Gray code control to manage multiplexer addresses Minimize effect of upsets Data spends less than 8 bunch crossings in ADC multiplexer Minimize upsets Triple redundant MUX 8B/10B encoding at level of the serializer Full DAQ chain tested and debugged successfully 03/14/2009 ATLAS LAr Electronics Upgrade

ATLAS LAr Electronics Upgrade Optical Links in SOS 0.25μm UltraCMOSTM Silicon on Sapphire (SOS) by Peregrine Semiconductor Part of the Silicon on Insulator (SOI) family of CMOS technologies, and primarily used in aerospace and military applications because of its inherent resistance to radiation Low power, low cross talk, good for mixed-signal ASIC designs. Economical for small to medium scale ASIC development The shift registers are for SEE tests Characterization of radiation resistance of 0.25μm SOS Technology A dedicated test chip with transistors, ring oscillators and shift registers designed and fabricated for irradiation tests TID tests on transistors with gamma (60Co) With a grounded substrate during irradiation, there is no measurable leakage current and threshold voltage change in both NMOS and PMOS stay within fabrication variations SEE tests using shift registers with 230MeV proton No SEE was observed with flux < 1×109 proton/cm2/sec All shift registers function error free after a total fluence of 1.9×1013 proton/cm2 and ionizing dose of 106Mrad This part for TID test SOS irradiation test chip 03/14/2009 ATLAS LAr Electronics Upgrade

Link on Chip 1st Prototype (LOC1) Standard- alone Ring-Type PLL LC-tank Solid line box: implemented in LOC1 Dashed line box: implemented in FPGA A photograph of LOC1 First prototype available in June 2007 Specs for 2.5Gbps, 62.5MHz reference clock, power consumption ~200mW Large jitter is observed and understood (DJ from 4-arm 2-stage mux serializer, RJ comes mostly from the PLL), it will be corrected in the second prototype, the best BER reach is ~10-11 SEE test with 200MeV proton beam, error free for a fluence of 9x109p/cm2, SEE cross secion less than 1x10-10cm2/proton, data analysis is on-going and more tests are needed Eye diagram of a 27-1 pseudo random input data, UI = 400ps 03/14/2009 ATLAS LAr Electronics Upgrade

Link on Chip 2nd Prototype (LOC2) LOC2 consists of two parts and interface to CERN Versatile Link 5Gbps 16:1 serializer Simplify the implementation of high speed circuits Design of three critical components is in progress (PLL, Static DFF, CML driver) User interface Provide better interface to different sub-systems and flexibility to meet different requirements, will be implemented in FPGA for testing purpose Schematics level simulation finished, layout and simulations are in progress, aim for submission in June and lab test in Sept. 2009 03/14/2009 ATLAS LAr Electronics Upgrade

Readout Driver (ROD) Upgrade R&D Goals Prototype a ROD that can receive and process continuously digitized detector signals from proposed front-end architecture Investigate and evaluate several critical technologies which could be used in the design of next generation ROD R&D interests High speed optical link & FPGA SERDES Data bandwidth of entire LAr (1524 FEBs) > 150 Tbps Industry standard parallel fiber connector MPO/MTP, which has 12 fibers and is about 15mm wide Low latency lossless data compression/decompression algorithm Reduce number of links FPGA based Digital Signal Processing Take the advantage of parallel data processing Explore different system level architecture: AdvancedTCA Take the advantage of industrial standard and high availability features: redundancy, shelf management protocols, power management, fabrics Perform level 1 trigger sum digitally Flexibility: fine granularity 03/14/2009 ATLAS LAr Electronics Upgrade

ROD Upgrade Architecture 12 x 2 fibers FEB (1524 modules) ROD (218 modules) 12 x 4 fibers 12 x 8 fibers 1 Pre-Sampler 4 Middle 2 Back LVL1 Interface ROB 3 14 FPGA ~5Gbps 12 EMB ROD cfg. ATCA interface 13 12 x 14 fibers 7 Front 03/14/2009 ATLAS LAr Electronics Upgrade

Sub-ROD and Injector Development Sub-ROD Injector Features Generate programmable test data pattern for Sub-ROD Transfers data to Sub-ROD through a 12 fiber, 40Gbps optical link Uses high speed SERDES in Altera Stratix II GX Family FPGA FPGA soft core NIOS processor with Linux and Gigabit Ethernet Sub-ROD Features 40Gbps link (12x3.5Gbps) to process ¼ 128-ch FEB data Uses high speed SERDES (GTP) in Xilinx Virtex-5 FPGA Buffer data in SODIMM DDR2-533 SDRAM (on back of board) DSP algorithm implemented in FPGA embedded DSP slices Gigabit Ethernet link to transfer processed data to PC ATCA Sub-ROD schematics design finished, layout is in progress Integration Test on Sept. 17-19, 2008 2.4Gbps link running stably Fixed pattern data transmission of 36 channels (over 12 fibers) Experiment transceiver from different vendors (Reflex Photonics and Emcore) Sub-ROD Injector Sub-ROD 03/14/2009 ATLAS LAr Electronics Upgrade

Current LAr to L1calo Interface Analog transmission with bulky copper cable Off detector digitization on PPM Fixed size of trigger tower [ηxφ=0.1x0.1] L1A latency < 2.5μs 03/14/2009 ATLAS LAr Electronics Upgrade

Possible LAr to L1calo Interface Perform level 1 trigger sum digitally LAr raw data are pre-processed on LAr ROD Pre-summed trigger tower information sent to L1calo Flexible granularity Level 1 latency study We could very likely control level 1 latency < 2.5μs Possible ATLAS wide level 1 latency increase Total Latency ~ 40 BC ~ 1us 03/14/2009 ATLAS LAr Electronics Upgrade

Current LVPS System Implementation 19 voltage regulators/FEB Centralized architecture 03/14/2009 ATLAS LAr Electronics Upgrade

Strategy on Rad-Tolerant LVPS R&D Radiation environment assumed to scale x10 Total power budget of the LAr electronics assumed to remain same, but there will be different voltages/currents required Rationalization of the number and levels of the voltages Different input voltage levels and regulations Use of point of load regulators Possibility of adopting non-traditional topologies Reduced voltage across devices Optimized design of magnetics Minimize losses and heat Higher switching frequency System design Single Failure Tolerant Migration of the LVPS Architecture Distributed Power Architecture A main converter generates a distribution bus and on-boards POLs directly supply low voltages to loads Intermediate Bus Architecture In addition to the generation of a main voltage bus, as in the DPA but typically higher, a second set of bus voltages is provided, then lower voltages are given by the point-of-load converters 03/14/2009 ATLAS LAr Electronics Upgrade

Possible Architectures Distributed Power Architecture Card #3 POL niPOL Converter 280 VDC Card #2 POL niPOL Converter V1x VDC Crate POL POL Converter Card #1 Main Converter V2x VDC (ex- LVPS) Card #2 POL LDO POL Converter Card #1 Intermediate Bus Architecture 03/14/2009 ATLAS LAr Electronics Upgrade

ATLAS LAr Electronics Upgrade Summary Proposed new front-end architecture implies new requirements of the whole LAr readout electronics and many R&D interests Front-end mixed-signal ASIC design Integrated analog front-end R&D in SiGe Rad-rad ADC R&D in IBM 8RF CMOS Radiation resistance optical link in SOS High speed link to readout ~100Gbps/FEB Readout Driver (ROD) upgrade R&D High speed commercial optical link & FPGA SERDES FPGA based DSP New level 1 calorimeter trigger interface Power supply distribution scheme New power supply distribution R&D to improve power integrity We have a good number of R&D activities of LAr calorimeter readout electronics aim for ATLAS phase-2 upgrade ~2017 03/14/2009 ATLAS LAr Electronics Upgrade

ATLAS LAr Electronics Upgrade Backup Slides 03/14/2009 ATLAS LAr Electronics Upgrade

Analog Front-end: Preamp & Shaper Prototype submitted to MOSIS on Nov. 10th, 2008 Preamp Rin = 25Ω Based on low noise line-terminating preamplifier circuit topology presently used in ATLAS LAr SiGe higher base doping, lower rbb for low noise High breakdown (VB=3.6V) devices allow for higher swing to accommodate full 16-bit dynamic range Thick analog metal allows for low resistance connections to input BJTs are excellent drivers, output current ~170mA at Iin = 5mA en,equiv= 0.26nV/√Hz ENI = 73nArms (incl. 2nd stage, Cd = 1nF) Ptot = 42mW Shaper Linear Input Range 0-.35V less than 0.2% NL Maximum Input > 3V (Input diode protected) Shaping (RC)2 - CR Poles: RC = 15ns Power ~100mW (probably can be reduced 20%) Shaper Input Referred Noise ~2.2nV /√Hz Large Emitter followers on outputs allow additional external current to drive Test Loads. Intended to be compatible with 130nm CMOS ADC Op Amp Unity Gain Stable using internal compensation. Auto - Common Mode Adjust with external programming control allows tuning of Dynamic Range Shaper (RC-CR2) optimizes S/N in presence of both electronics noise and pile-up Differentiation to restore baseline Electronics : ENI = A/tp3/2 + B/√tp Pileup : ENE = C√tp At 1034 tp= 45-50ns (=13ns) optimal 03/14/2009 ATLAS LAr Electronics Upgrade

Digital Signal Processing in FPGA Energy calculation in dedicated DSP slices of FPGA Energy calculation performed at beam crossing rate Calculate energy using optimal filtering weights If E > threshold, calculate timing and pulse quality factor Use FPGA solution which allows a more parallel implementation over traditional DSP Optimal filter consumes 1 DSP48E block which runs internally at 200MHz with 2 beam crossing latency Results compared to full floating point calculations Quantization Error : < 0.02% 03/14/2009 ATLAS LAr Electronics Upgrade

AdvancedTCA System Architecture Developed by The PCI Industrial Computer Manufacturer’s Group (PICMG), an industry consortium that has standardized many popular standards such as ISA and PCI technologies for industrial backplane applications PICMG 3.0 (ATCA) High Availability, redundancy built into everything: power supplies, fabrics, etc. Boards are designed for hot swap. Multi-gigabit serial transport (no parallel busses) choice of protocols: GigE, Infiniband, PCIe, Serial Rapid I/O Shelf Manager provides intelligent diagnostics, watches over basic health of system Large form factor and power budget 8U cards 200W per slot, 3kW per chassis 16 boards per crate Integrated Shelf Manager User Defined Connector Area Front Module Size 8U x 280mm Transition Module size 8U x 70mm Dual Star Backplane Redundant Power Entry Modules, -48v 03/14/2009 Air Intake Area ATLAS LAr Electronics Upgrade

Potential MOSFET for Finger Gap LVPS Top plot shows threshold voltage of SPP07N60S5 MOSFET and the NTP1060 MOSFET used in the present power supply In this measurement threshold voltage is defined as value VGS that produces IDS = 0.25 mA, VDS = 40 Volts Unfortunately, this measurement was ended at 100 krad. As can be seen the rate of change of VGSThr is decreasing and may still be suitable at 450 krad So strategy would be to make the threshold voltage large so that the device is still usable at high total doses Bottom plot shows Single Event Burnout cross section for the same two MOSFETs Although both MOSFETs are rated for 600 volts the Infineon SPP07N60S5 shows a 50 volt advantage in de-rating behavior Unfortunately, this Infineon product is being discontinued and should not be used in a new design. However, it shows that a commercial MOSFET can exist that could be used in this application 03/14/2009 ATLAS LAr Electronics Upgrade

COTS Bulk Converters Tested Commercial Buck converters were tested with ionizing radiation until some significant changes were observed These are small converters which are often integrated in 1 chip and are board mounted for POL applications All “useful” devices tested failed at doses much lower than would be required in a sLHC environment One device which survived to very high total doses was the EN5360 produced by Enpirion in 0.25 μm CMOS technology. This buck converter is still working after 100 Mrad Commercial device is possible to be radiation hard and perhaps others in similar technologies will be also We are evaluating radiation characteristics of IHP SGB25VD technology transistors Device Irradiation Time in Seconds Dose until damage seen (krad) Observations Damage Mode TPS 62110 720 40 Increasing input current ISL 8502 730 41 MAX 8654 850 47 Loss of output voltage regulation ADP 21xx 1000 56 ST1510 2250 125 IR 3822 2500 139 EN5382 2000 111 EN5360#3 864000 48,000 Minimal Damage EN5360#2 Tested in 2007 100,000 03/14/2009 ATLAS LAr Electronics Upgrade