1. R&D Studies of the ATLAS LAr Calorimeter Readout Electronics for super-LHC
Hucheng Chen
On behalf of the ATLAS LAr Calorimeter Collaboration
March 14th, 2009

2. ATLAS LAr Electronics Upgrade
Outline
ATLAS Liquid Argon Calorimeter
Current LAr Readout Electronics
LAr Electronics Upgrade Motivations
R&D Studies
Front-end mixed-signal ASIC design
BNL, Columbia Univ., Univ. of Penn., INFN Milan, IN2P3
Radiation resistance optical link in SOS
SMU
High speed back-end processing unit based on FPGA
Level 1 calorimeter trigger interface
BNL, Univ. of Arizona, SUNY Stony Brook, IN2P3 LAPP, INFN Milan, Dresden, CERN
Power supply distribution scheme
BNL, Yale, INFN Milan, Univ. of Milan
Summary
03/14/2009
ATLAS LAr Electronics Upgrade

3. ATLAS Liquid Argon Calorimeter (LAr)
Barrel calorimeters in detector hall, surrounded by barrel toroids
Liquid Argon Calorimeter
Electromagnetic Barrel (EMB)
|η|<1.375 [Pb-LAr]
Electromagnetic End-cap (EMEC)
1.4<|η|<3.2 [Pb-LAr]
Hadronic End-cap (HEC)
1.5<|η|<3.2 [Cu-LAr]
Forward Calorimeter (FCAL)
3.2<|η|<4.9 [Cu,W-LAr]
Front End Crate
03/14/2009
ATLAS LAr Electronics Upgrade

4. Current LAr Readout Electronics
~180,000 detector channels
Front-end Electronics
58 Front End Crates
1524 Front End Boards
58 LV Power Supplies
~1600 fiber optic links between FE and BE
Back-end Electronics
16 Back End Crates
192 Read Out Driver Boards
68 ROS PCs
~800 fiber optic links between ROD and ROS
03/14/2009
ATLAS LAr Electronics Upgrade

5. LHC Upgrade Expectation
LHC upgrade includes 2 phases
sLHC (phase 2 upgrade) expected to start up ~2017
ATLAS LAr calorimeter plans for LHC phase 2 upgrade
New requirements to LAr calorimeter readout electronics
Radiation environment assumed to scale x10
Total power consumption kept same
Luminosity plot before LHC accident
New injectors + IR upgrade phase 2
sLHC Start-up
Early operation
Linac4 + IR upgrade phase 1
Peak Luminosity:
2017: 3x1034 cm-2s-1
Integrated Luminosity:
: 650 fb-1
: 550 fb-1
03/14/2009
ATLAS LAr Electronics Upgrade

6. Current Front-end Architecture
FEB Complexity
11 ASICs
Several technologies
Obsolescence of technologies (e.g. DMILL)
19 voltage regulators
Analog pipelines (SCA)
~80W/board, water cooled
Radiation/lifetime issues
Qualified for 10 years LHC operation
Limited number of spares (~6%)
Other limitations
L1 trigger rate <= 100kHz
L1 trigger latency <= 2.5µs
Consecutive L1 trigger spaced more than 125ns
FEB Upgrade
Component-level replacement impossible
Full replacement based on current technologies
No increase of power budget
03/14/2009
ATLAS LAr Electronics Upgrade

7. Proposed Front-end Architecture
FEB “strawman” architecture keeps many options open
Shaping and gain settings
Analog .vs. digital pipeline
On/off detector pipeline
Analog .vs. digital gain selector
Possibly provide analog trigger sums to decouple potential trigger upgrade
FEB upgrade propagates to other boards
Digitization at each bunching crossing
~100Gbps/board
Higher speed, higher radiation resistance optical link
LV power supplies
Back-end electronics
Possibly interface to L1calo digitally
03/14/2009
ATLAS LAr Electronics Upgrade

8. Test Structure Chiplet
Analog Front-end R&D
An increasing attention to SiGe technology for fast detector readout and in applications where radiation is a concern
ILC R&D program in Europe
Cross sub-system effort in ATLAS
Inner Detector: UCSC, Barcelona
LAr: BNL, Columbia Univ., Univ. of Penn., INFN-Milan, IN2P3
Characterization of IBM 8WL 0.13μm SiGe BiCMOS
Foundry provided samples
Gamma BNL
Neutron Ljubljana, UMass Lowell
Proton Boston (Mass. Gen. Hospital Cyclotron)
IBM 8WL design kit parts
Test structure Chiplet submitted on Nov. 10th, 2008
8WL design kit transistor test devices
8WL CMOS transistor array
Test Structure Chiplet
2.7mm X 1.8mm
03/14/2009
ATLAS LAr Electronics Upgrade

9. Analog Front-end: Preamp & Shaper
(RC)2 10X Shaper
(RC)2-CR
Preamp out
0-700μA into
preamp
Prototype LAr Preamp and Shaper
Submitted to MOSIS on Nov. 10th, 2008
Preamp
Very low noise ~0.25nV/√Hz, high dynamic range
Based on low noise line-terminating preamplifier circuit topology presently used in ATLAS LAr calorimeter
Shaper
Fully differential
Robust performance in low signal environment, excellent pickup rejection on and off chip
Explored several low power non feedback approaches, none satisfied noise and linearity simultaneously
LAr Chiplet
2mm X 2mm
03/14/2009
ATLAS LAr Electronics Upgrade

10. Mixed-signal Front-end: ADC
ADC is the most technologically challenging component in the new “baseline” architecture
15(6) bit dynamic range - 12 bit resolution - 40MSPS
Radiation tolerant and SEE immune
Strategies being followed
Find the magic bullet with commercial parts
Planning to test several COTS ADCs
Probably not meeting radiation requirement because bit error correction stored in rad-soft RAM
Developing a custom ADC
IBM 8RF CMOS as possible candidate technology
Already shown to be radiation hard
Lower cost compared to SiGe
Lots of CERN based experience
03/14/2009
ATLAS LAr Electronics Upgrade

11. ATLAS LAr Electronics Upgrade
Pipeline ADC R&D
Pipelined ADC with digital correction
Each stage resolves more bits than subsequent amplification
Sample & Hold capacitor ultimately also a limiting factor because noise ~ sqrt(kT/C)
Amplifier at stage output is critical
Each stage is 1.5 bits
Overlap between stages allows correction before outputs are “added” together
If correction done on board it will need to store calibration constants in redundant rad-hard memory
ADC development status
Expected to need bipolar technology, started in IBM 8HP SiGe BiCMOS
Decided to redesign in IBM 8RF CMOS due to cost and CERN-based experience
Conversion to 8RF completed and back to target spec (12bit)
Plan to submit a MOSIS test structure proposal by this June
03/14/2009
ATLAS LAr Electronics Upgrade

12. Digital Logic Initial Design
2 Independent clocks
“TTC”-like clock for ADC and ADC multiplexer
Crystal derived clock for high speed components (MUX, serializer) provides much better jitter control
Gray code control to manage multiplexer addresses
Minimize effect of upsets
Data spends less than 8 bunch crossings in ADC multiplexer
Minimize upsets
Triple redundant MUX
8B/10B encoding at level of the serializer
Full DAQ chain tested and debugged successfully
03/14/2009
ATLAS LAr Electronics Upgrade

13. ATLAS LAr Electronics Upgrade
Optical Links in SOS
0.25μm UltraCMOSTM Silicon on Sapphire (SOS) by Peregrine Semiconductor
Part of the Silicon on Insulator (SOI) family of CMOS technologies, and primarily used in aerospace and military applications because of its inherent resistance to radiation
Low power, low cross talk, good for mixed-signal ASIC designs.
Economical for small to medium scale ASIC development
The shift registers are for SEE tests
Characterization of radiation resistance of 0.25μm SOS Technology
A dedicated test chip with transistors, ring oscillators and shift registers designed and fabricated for irradiation tests
TID tests on transistors with gamma (60Co)
With a grounded substrate during irradiation, there is no measurable leakage current and threshold voltage change in both NMOS and PMOS stay within fabrication variations
SEE tests using shift registers with 230MeV proton
No SEE was observed with flux < 1×109 proton/cm2/sec
All shift registers function error free after a total fluence of 1.9×1013 proton/cm2 and ionizing dose of 106Mrad
This part for TID test
SOS irradiation test chip
03/14/2009
ATLAS LAr Electronics Upgrade

14. Link on Chip 1st Prototype (LOC1)
Standard-
alone
Ring-Type
PLL
LC-tank
Solid line box: implemented in LOC1
Dashed line box: implemented in FPGA
A photograph of LOC1
First prototype available in June 2007
Specs for 2.5Gbps, 62.5MHz reference clock, power consumption ~200mW
Large jitter is observed and understood (DJ from 4-arm 2-stage mux serializer, RJ comes mostly from the PLL), it will be corrected in the second prototype, the best BER reach is ~10-11
SEE test with 200MeV proton beam, error free for a fluence of 9x109p/cm2, SEE cross secion less than 1x10-10cm2/proton, data analysis is on-going and more tests are needed
Eye diagram of a 27-1 pseudo random input data, UI = 400ps
03/14/2009
ATLAS LAr Electronics Upgrade

15. Link on Chip 2nd Prototype (LOC2)
LOC2 consists of two parts and interface to CERN Versatile Link
5Gbps 16:1 serializer
Simplify the implementation of high speed circuits
Design of three critical components is in progress (PLL, Static DFF, CML driver)
User interface
Provide better interface to different sub-systems and flexibility to meet different requirements, will be implemented in FPGA for testing purpose
Schematics level simulation finished, layout and simulations are in progress, aim for submission in June and lab test in Sept. 2009
03/14/2009
ATLAS LAr Electronics Upgrade

16. Readout Driver (ROD) Upgrade R&D
Goals
Prototype a ROD that can receive and process continuously digitized detector signals from proposed front-end architecture
Investigate and evaluate several critical technologies which could be used in the design of next generation ROD
R&D interests
High speed optical link & FPGA SERDES
Data bandwidth of entire LAr (1524 FEBs) > 150 Tbps
Industry standard parallel fiber connector MPO/MTP, which has 12 fibers and is about 15mm wide
Low latency lossless data compression/decompression algorithm
Reduce number of links
FPGA based Digital Signal Processing
Take the advantage of parallel data processing
Explore different system level architecture: AdvancedTCA
Take the advantage of industrial standard and high availability features: redundancy, shelf management protocols, power management, fabrics
Perform level 1 trigger sum digitally
Flexibility: fine granularity
03/14/2009
ATLAS LAr Electronics Upgrade

17. ROD Upgrade Architecture
12 x 2 fibers
FEB (1524 modules)
ROD (218 modules)
12 x 4 fibers
12 x 8 fibers 1
Pre-Sampler 4
Middle 2
Back
LVL1
Interface
ROB 3 14
FPGA
~5Gbps 12
EMB ROD cfg.
ATCA
interface 13
12 x 14 fibers 7
Front
03/14/2009
ATLAS LAr Electronics Upgrade

18. Sub-ROD and Injector Development
Sub-ROD Injector Features
Generate programmable test data pattern for Sub-ROD
Transfers data to Sub-ROD through a 12 fiber, 40Gbps optical link
Uses high speed SERDES in Altera Stratix II GX Family FPGA
FPGA soft core NIOS processor with Linux and Gigabit Ethernet
Sub-ROD Features
40Gbps link (12x3.5Gbps) to process ¼ 128-ch FEB data
Uses high speed SERDES (GTP) in Xilinx Virtex-5 FPGA
Buffer data in SODIMM DDR2-533 SDRAM (on back of board)
DSP algorithm implemented in FPGA embedded DSP slices
Gigabit Ethernet link to transfer processed data to PC
ATCA Sub-ROD schematics design finished, layout is in progress
Integration Test on Sept , 2008
2.4Gbps link running stably
Fixed pattern data transmission of 36 channels (over 12 fibers)
Experiment transceiver from different vendors (Reflex Photonics and Emcore)
Sub-ROD Injector
Sub-ROD
03/14/2009
ATLAS LAr Electronics Upgrade

19. Current LAr to L1calo Interface
Analog transmission with bulky copper cable
Off detector digitization on PPM
Fixed size of trigger tower [ηxφ=0.1x0.1]
L1A latency < 2.5μs
03/14/2009
ATLAS LAr Electronics Upgrade

20. Possible LAr to L1calo Interface
Perform level 1 trigger sum digitally
LAr raw data are pre-processed on LAr ROD
Pre-summed trigger tower information sent to L1calo
Flexible granularity
Level 1 latency study
We could very likely control level 1 latency < 2.5μs
Possible ATLAS wide level 1 latency increase
Total Latency
~ 40 BC ~ 1us
03/14/2009
ATLAS LAr Electronics Upgrade

21. Current LVPS System Implementation
19 voltage regulators/FEB
Centralized architecture
03/14/2009
ATLAS LAr Electronics Upgrade

22. Strategy on Rad-Tolerant LVPS R&D
Radiation environment assumed to scale x10
Total power budget of the LAr electronics assumed to remain same, but there will be different voltages/currents required
Rationalization of the number and levels of the voltages
Different input voltage levels and regulations
Use of point of load regulators
Possibility of adopting non-traditional topologies
Reduced voltage across devices
Optimized design of magnetics
Minimize losses and heat
Higher switching frequency
System design Single Failure Tolerant
Migration of the LVPS Architecture
Distributed Power Architecture
A main converter generates a distribution bus and on-boards POLs directly supply low voltages to loads
Intermediate Bus Architecture
In addition to the generation of a main voltage bus, as in the DPA but typically higher, a second set of bus voltages is provided, then lower voltages are given by the point-of-load converters
03/14/2009
ATLAS LAr Electronics Upgrade

23. Possible Architectures
Distributed Power Architecture
Card #3
POL
niPOL Converter
280 VDC
Card #2
POL
niPOL Converter
V1x VDC
Crate
POL
POL Converter
Card #1
Main Converter
V2x VDC
(ex- LVPS)
Card #2
POL
LDO
POL Converter
Card #1
Intermediate Bus Architecture
03/14/2009
ATLAS LAr Electronics Upgrade

24. ATLAS LAr Electronics Upgrade
Summary
Proposed new front-end architecture implies new requirements of the whole LAr readout electronics and many R&D interests
Front-end mixed-signal ASIC design
Integrated analog front-end R&D in SiGe
Rad-rad ADC R&D in IBM 8RF CMOS
Radiation resistance optical link in SOS
High speed link to readout ~100Gbps/FEB
Readout Driver (ROD) upgrade R&D
High speed commercial optical link & FPGA SERDES
FPGA based DSP
New level 1 calorimeter trigger interface
Power supply distribution scheme
New power supply distribution R&D to improve power integrity
We have a good number of R&D activities of LAr calorimeter readout electronics aim for ATLAS phase-2 upgrade ~2017
03/14/2009
ATLAS LAr Electronics Upgrade

25. ATLAS LAr Electronics Upgrade
Backup Slides
03/14/2009
ATLAS LAr Electronics Upgrade

26. Analog Front-end: Preamp & Shaper
Prototype submitted to MOSIS on Nov. 10th, 2008
Preamp
Rin = 25Ω
Based on low noise line-terminating preamplifier circuit topology presently used in ATLAS LAr
SiGe higher base doping, lower rbb for low noise
High breakdown (VB=3.6V) devices allow for higher swing to accommodate full 16-bit dynamic range
Thick analog metal allows for low resistance connections to input
BJTs are excellent drivers, output current ~170mA at Iin = 5mA
en,equiv= 0.26nV/√Hz
ENI = 73nArms (incl. 2nd stage, Cd = 1nF)
Ptot = 42mW
Shaper
Linear Input Range 0-.35V less than 0.2% NL
Maximum Input > 3V (Input diode protected)
Shaping (RC)2 - CR Poles: RC = 15ns
Power ~100mW (probably can be reduced 20%)
Shaper Input Referred Noise ~2.2nV /√Hz
Large Emitter followers on outputs allow additional external current to drive Test Loads. Intended to be compatible with 130nm CMOS ADC
Op Amp Unity Gain Stable using internal compensation.
Auto - Common Mode Adjust with external programming control allows tuning of Dynamic Range
Shaper (RC-CR2) optimizes S/N in presence of both electronics noise and pile-up
Differentiation to restore baseline
Electronics : ENI = A/tp3/2 + B/√tp
Pileup : ENE = C√tp
At 1034 tp= 45-50ns (=13ns) optimal
03/14/2009
ATLAS LAr Electronics Upgrade

27. Digital Signal Processing in FPGA
Energy calculation in dedicated DSP slices of FPGA
Energy calculation performed at beam crossing rate
Calculate energy using optimal filtering weights
If E > threshold, calculate timing and pulse quality factor
Use FPGA solution which allows a more parallel implementation over traditional DSP
Optimal filter consumes 1 DSP48E block which runs internally at 200MHz with 2 beam crossing latency
Results compared to full floating point calculations
Quantization Error : < 0.02%
03/14/2009
ATLAS LAr Electronics Upgrade

28. AdvancedTCA System Architecture
Developed by The PCI Industrial Computer Manufacturer’s Group (PICMG), an industry consortium that has standardized many popular standards such as ISA and PCI technologies for industrial backplane applications
PICMG 3.0 (ATCA)
High Availability, redundancy built into everything: power supplies, fabrics, etc. Boards are designed for hot swap.
Multi-gigabit serial transport (no parallel busses)
choice of protocols: GigE, Infiniband, PCIe, Serial Rapid I/O
Shelf Manager provides intelligent diagnostics, watches over basic health of system
Large form factor and power budget
8U cards
200W per slot, 3kW per chassis
16 boards per crate
Integrated Shelf Manager
User Defined Connector Area
Front Module Size 8U x 280mm
Transition Module size 8U x 70mm
Dual Star Backplane
Redundant Power Entry Modules, -48v
03/14/2009
Air Intake Area
ATLAS LAr Electronics Upgrade

29. Potential MOSFET for Finger Gap LVPS
Top plot shows threshold voltage of SPP07N60S5 MOSFET and the NTP1060 MOSFET used in the present power supply
In this measurement threshold voltage is defined as value VGS that produces IDS = 0.25 mA, VDS = 40 Volts
Unfortunately, this measurement was ended at 100 krad. As can be seen the rate of change of VGSThr is decreasing and may still be suitable at 450 krad
So strategy would be to make the threshold voltage large so that the device is still usable at high total doses
Bottom plot shows Single Event Burnout cross section for the same two MOSFETs
Although both MOSFETs are rated for 600 volts the Infineon SPP07N60S5 shows a 50 volt advantage in de-rating behavior
Unfortunately, this Infineon product is being discontinued and should not be used in a new design. However, it shows that a commercial MOSFET can exist that could be used in this application
03/14/2009
ATLAS LAr Electronics Upgrade

30. COTS Bulk Converters Tested
Commercial Buck converters were tested with ionizing radiation until some significant changes were observed
These are small converters which are often integrated in 1 chip and are board mounted for POL applications
All “useful” devices tested failed at doses much lower than would be required in a sLHC environment
One device which survived to very high total doses was the EN5360 produced by Enpirion in 0.25 μm CMOS technology. This buck converter is still working after 100 Mrad
Commercial device is possible to be radiation hard and perhaps others in similar technologies will be also
We are evaluating radiation characteristics of IHP SGB25VD technology transistors
Device
Irradiation Time in Seconds
Dose until damage seen (krad)
Observations Damage Mode
TPS 62110
720 40
Increasing input current
ISL 8502
730 41
MAX 8654
850 47
Loss of output voltage regulation
ADP 21xx
1000 56
ST1510
2250
125
IR 3822
2500
139
EN5382
2000
111
EN5360#3
864000
48,000
Minimal Damage
EN5360#2
Tested in 2007
100,000
03/14/2009
ATLAS LAr Electronics Upgrade