Agata Week – LNL 14 November 2007 Global Readout System for the AGATA experiment M. Bellato a a INFN Sez. di Padova, Padova, Italy

Agata Week – LNL 14 November 2007 AGATA Experiment: 4  array of HPGe detectors for in-beam  -ray spectroscopy 36 fold segmented crystal + central core contact x 180 Digital electronics and sophisticated Pulse Shape Analysis algorithms Operation of Ge detectors in position sensitive mode  -ray tracking

Agata Week – LNL 14 November 2007 AGATA Readout: Channel per crystal: per crystal Total channel: 6660 Local Level Trigger Central core processing Central trigger processor Global Trigger Global Trigger and Synchronization control system (GTS) Global time reference The design of FE readout and L1 trigger follows a synchronous pipeline model: the data are collected at 100 MS/s and stored in pipeline buffers at the global AGATA frequency waiting the L1 decision

Agata Week – LNL 14 November 2007 Agata Front-end Model

Agata Week – LNL 14 November 2007 Choice of Readout Standard Candidates VME/VXI Past Experience with older detectors Bandwidth limited CompactPCI First version of AGATA Readout electronics Advanced TCA Present version

Agata Week – LNL 14 November 2007 Pros –Very stable –Huge choice of commercial boards –Slave implementation very simple –Wide ecosystem in HEP –Recently extended with serial switched capabilities Cons –Slow readout (max 40-80) Mbytes/s Enhancements (CBLT) not fully standard –Limited power dissipation per slot –Master implementation not trivial –If large estate needed then 9U crates expensive –Limited customizability on the bus –Crate control not standard –No power redundancy -> limited availability VME

Agata Week – LNL 14 November 2007 Pros –Fast ( > 500 Mbytes/s) –Huge choice of commercial boards –Low cost (Telecom standard) –Wide choice of IP cores available for interfacing (both initiator/target) –Extended to serial switched with CPCIExpress –Highly customizable through user reserved backplane connectors Cons –Small estate –Low power dissipation per slot –Low no. of slots per bus without bridging –No standard crate control CompactPCI

Agata Week – LNL 14 November 2007 Pros –Ultra high speed –Multi standard –Big estate / high power per slot –Choice of star or full mesh backplane –Redundant power supply –Standard crate control –Fits perfectly with modern FPGAs/ ASSPs –Gaining momentum in HEP community –Custom deployments can co-exist with standards Cons –Moderate choice of COTS cards on the market –Commercial switches exist only for 1G/10G ethernet Telecom driven Advanced TCA

Agata Week – LNL 14 November 2007 Carrier Readout (1): Main FPGA DP RAM Segment Dual star ATCA network Gigabit Ethernet Switch LLP Carrier PSA Farm

Agata Week – LNL 14 November 2007 Carrier Readout (2): Main FPGA DP RAM Segments Full mesh ATCA network Optical Transceiver PCI Express 2.5 Gbps PCI Express Endpoint LLP Carrier PCI Express optical translator Virtual I/O multiprocessor system

Agata Week – LNL 14 November 2007

Clock Distribution From GTS Tree

Agata Week – LNL 14 November 2007 MGT Clocking Layout RocketIO 101 RocketIO 101 MUX RocketIO 102 RocketIO 102 MUX RocketIO 103 RocketIO 103 MUX RocketIO 105 RocketIO 105 MUX MGTclk M34/N34 MGTclk M34/N34 MGTclk AP28/AP29 MGTclk AP28/AP29 RocketIO 106 RocketIO 106 MUX RocketIO 109 RocketIO 109 MUX RocketIO 110 RocketIO 110 MUX RocketIO 112 RocketIO 112 MUX RocketIO 113 RocketIO 113 MUX MGTclk AP3/AP4 MGTclk AP3/AP4 MGTclk J1/K1 MGTclk J1/K1 RocketIO 114 RocketIO 114 MUX ATCA FABRIC CH1-CH2 ATCA FABRIC CH3-CH4 ATCA FABRIC CH5-CH6 ATCA FABRIC CH7-CH8 ATCA FABRIC CH9-CH10 ATCA FABRIC CH11-CH12 USER SFP TRANSCEIVER RTM PCI EXPRESS LANE0 RTM PCI EXPRESS LANE5 RTM PCI EXPRESS LANE1 RTM PCI EXPRESS LANE2 100  250MHz PCI Express JITTER ATTENUATOR 100  250MHz PCI Express JITTER ATTENUATOR 200MHz GTS Clock 200MHz GTS Clock (**) The ATCA FABRIC channels are routed from CHANNEL1 to CHANNEL12 by switches (***) User SFP could be used as 1GEnet or PCIExpress DAQ without FABRIC RTM PCI EXPRESS LANE3 RTM PCI EXPRESS LANE4 ABAB ABAB ABAB ABAB ABAB ABAB ABAB ABAB ABAB ABAB INSPECTION PADS 100MHz GTS Clock 100MHz GTS Clock OPTICAL SFP OPTICAL SFP INSPECTION PADS LOCAL 100MHz (EPSON ) LOCAL 100MHz (EPSON ) PHASE LOCKED MGT clocking layout

Agata Week – LNL 14 November V DC ENABLE P3V3-5A 16.5W MEZZANINE 1 MEZZANINE 2 MEZZANINE 3 MEZZANINE 4 MAIN BOARD P3V3-7A 23.1W MAIN BOARD P2V5-7A 17.5W FPGAs CORE P1V2-7A 8.4W FPGA MGT P1V2-4A 4.8W P2V5-1.5A P1V8-0.5A PROMS VCCAUX Fpga 1 VCCAUX MGT P2V5-1.5AVCCAUX Fpga 2 P1V2-0.5A VTTTXs P1V2-0.5AVTTRXs MGT BUFFERSP1V8-6A 10.8W P12V-14.7A 176.7(160.6)W P3V3-5A 16.5W M48V-4.0A 194.4(176.7)W DC-DC Efficency is estimated at least 90% ATC210 (210W) P3V3_BOOT 4x LTM W 6x LTM W P5V0-6A 30W Carrier Power Supply

Agata Week – LNL 14 November M x 18 true dual port 100/200 MHZ 800Mb/s LVDS streaming on data channels Equalized and filtered distribution of 200MHZ GTS clock 1 PCI Express/ GE optical link 15 x Full mesh connectivity on the backplane Pervasive I2C bus for slow controls 200W power supply Multiple options for data readout Carrier main features

Agata Week – LNL 14 November 2007 Final Routing (as of april 07)

Agata Week – LNL 14 November 2007 Power and Signal Integrity Simulations

Agata Week – LNL 14 November 2007 Example Resonant mode between L6pwr/L11gnd

Agata Week – LNL 14 November 2007

CH-15 Eye Diagram with Equalization

Agata Week – LNL 14 November 2007 Int *ATCA0 = 0xfe001000; // DPRAM on board 0 Int *ATCA1 = 0xfe002000; // DPRAM on board 1 …. Fragment0 = memcpy(buffer0, ATCA0); Fragment1= memcpy(buffer1, ATCA1); PCI Express Readout Test