1. T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse, E. Delagnes, F. Druillole, A. Le Coguie, E. Monmarthe, J-M. Reymond, E. Zonca DSM/IRFU, CEA Saclay, France O. Ballester, IFAE, Barcelona, Spain

2. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr2 Plan Presentation of the T2K experiment Architecture of TPC readout electronics Principal components Current status Experience return

3. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr3 Tokai to Kamioka (T2K) experiment Main Physics Goal: neutrino oscillation μ disappearance for improved accuracy on  23 e appearance to improve sensitivity to  13 50 kT water

4. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr4 T2K Time-Projection Chambers TPC features Resolution goal: 10% for p < 1 GeV/c Double wall design; inner wall is the field cage Total ~9 m 2 instrumented with 72 Micromegas amplification modules segmented in ~7 mm x ~10 mm pads Custom readout electronics 1 of 3 TPCs shown 1728 pad (36 x 48) Micromegas module 34 cm x 36 cm CalibrationGas systemElectronics; on-line software 1 m 2 m

5. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr5 TPC Readout Challenges Environment Embedded front-end: limited access, need low power No radiation, low magnetic field (0.2 T), Japanese underground facility Highly segmented detector 124.000 channels over 9 m 2 of instrumented area → Massive replication of identical modular building blocks Extreme burstiness of data 124.000 channels; sampling: 12-bit 33 MHz → 50 Tbps → beyond capability of fully digital solution for power budget target Large raw event size but modest average dataflow 50 Tbps during ~15 µs drift time → 90 MBytes Allowable event size for storage: ~250 KBytes → data reduction of ~400 to achieve in real time Spill repetition period: ~3.5 s; Cosmic calibration: 20 Hz max. DAQ average rate: ~250 KBytes × 20 Hz = 5 MByte/s → commercial computers and networking techniques

6. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr6 Plan Presentation of the T2K experiment Architecture of TPC readout electronics Principal components Current status Experience return

7. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr7 Logical Read-Out Flow Architecture principles Custom front-end ASIC; analog memory (Switch Capacitor Array) ADC + digital buffer mounted close to the detector Multiple optical fibers send data to off-detector concentrator cards Interface to common DAQ via standard gigabit Ethernet network Pre-amp and shapers Samplers and analog memory buffers Analog to digital conversion Digital buffer – zero-suppression Data concentrator Clock/Trigger fanout ~124.000 channels 1728 ASICs On-detector 72 Optical fibers ~4 Tbaud*/s peak *1 baud = 12 bit ~2 ms retention max. 34 Gbaud/s peak 400 Gbit/s peak ~1-10 Gbit/s averaged Shared DAQ ~0.1-1 Gbit/s 432 Front End Cards 18 Data Concentrator Cards 72 Front-End Mezzanines cards Off-detector Global Clock/Trigger Gigabit Ethernet

8. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr8 Architecture Implementation 3 TPCs 1 m 2,5 m Network TCP/IP PC Linux DCCs Private Ethernet Gigabit Ethernet DAQ control Detector B Detector A Global trigger 1 of 6 TPC end-plates (12-modules) Outside magnetInside magnet 2 of 12 (or 18) Data Concentrator Cards x 6 12 duplex Optical fibres 1 of 72 modules Front End Mezzanine Card (FEM) 288 channel Front End Card (FEC) 1728 pad Micromegas plane Slow control network Optical fiber to/from DCC Low voltage Power supply 1 of 1728 Front-End ASIC “AFTER” 72 channel x 511 time buckets Switched capacitor array

9. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr9 Plan Presentation of the T2K experiment Architecture of TPC readout electronics Principal components Current status Experience return

10. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr10 ASIC “AFTER” Functional Diagram AFTER 511 cells SCAFILTER 100ns<tpeak<2us CSA 1 channel x72(76) 76 to 1 BUFFER SCA MANAGER SLOW CONTROL Serial Interface W / R Mode CK ADC TEST In Test 120fC<Cf<600fC Power SupplyReference VoltageReference Current Asic Spy Mode CSA;CR;SCAin (N°1) Power On Reset Design features 72 channels x 511 analog memory cells; F write : 1-50 MHz; F read : 20 MHz 4 Charge Ranges (120 fC; 240 fC; 360 fC & 600 fC) Supports positive or negative input signals 16 Peaking Time Values (100 ns to 2 µs) Asic For Tpc Electronic Read-out

11. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr11 The AFTER chip Design facts 0.35 µm CMOS - 500.000 transistors 7.8 x 7.4 mm die – 160-pin 0.65 mm pitch LQFP single 3.3V supply - 8 mW/channel Performance Integral non-linearity < 1.2% full range; ENC: 350 electrons (no load) Stored charge degradation after 2 ms retention: 0.18 LSB → All specifications met on first silicon! AFTER chip used to readout TPC Micromegas and FGD Silicon PM

12. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr12 AFTER chip test and production Production of the AFTER chip 5300 chip produced; 4750 OK; Yield: 89% 1800 chips delivered to T2K (TPC + FGD + monitoring chamber) LABVIEW Test Software Protection 1 Protection 2No Protection AFTER Test Card Test bench AFTER test card served as pre-prototype of Front-End Card Xilinx Virtex 2 Pro eval. kit as pre-prototype of FEM board for readout DAQ and Analysis with Ethernet PC and LabView

13. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr13 Front-End Card (FEC) Features All design concepts validated on AFTER test board 1 FEC reads out 288 channels – 6 layer PCB Consumption: 1 A – 4 W Quad-channel 12-bit ADC AD9229 Passive protection Circuits 4 AFTER chips 3.3V regulators Silicon ID chip V - I –T monitor +4V input Four dual row 80-pin 1.27 mm pitch connectors to detector Pulser Clock fanout PhotoMos relay 80-pin connector to FEM

14. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr14 Main functions Drive 6 FECs and aggregate data produced (1728 channels, 5.7 Gbps) Buffer one event (raw data), i.e. ~10 Mbit Deliver data to DCC upon request: raw data or zero-suppressed (one programmable threshold per channel) Configuration and slow control, voltage, current, temperature monitoring Front-end Mezzanine (FEM) card ZBT SRAM 2 Gbps optical transceiver CANbus slow control (RJ45) FPGA µC FEC#1 Voltage regulators JTAG Power In Connector FEC#0 BuffersFEC#2 FEC#3FEC#4FEC#5

15. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr15 Detector Module Read-out Electronics 72-channel ASIC Quad-channel ADC digital Front-end Mezzanine card (FEM) Optical Transceiver FPGA 80-pin connector 288-channel analog Front-End Card (FEC) 1728-pad detector plane Slow-control Network - CANbus Fiber to DCC Low voltage power Materializing the concept 3 year development of ~6 equivalent persons from ASIC specifications to working readout of one detector module

16. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr16 Plan Presentation of the T2K experiment Architecture of TPC readout electronics Principal components Current status Experience return

17. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr17 Integration of TPC#0 at Triumf Feb. ‘09

18. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr18 Integration of TPC#0 and #1 at Tokai Photos: Claudio Giganti

19. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr19 Latest news from T2K nd280 m TPC #2 Tested at Triumf and now in the clean room at Tokai Installation in pit on Dec’ 17th → Fully commissioned TPCs by mid-Jan’10 for closing magnet TPC #0 and TPC #1 Installed in the magnet; DAQ commissioning in progress Awaiting final gas mixture to power central cathode and detector HV → Detector commissioning starts Dec. 7th The nd280 collaboration announced the observation of the first beam neutrino candidate (in the INGRID detector) two weeks ago!

20. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr20 Plan Presentation of the T2K experiment Architecture of TPC readout electronics Principal components Current status Experience return

21. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr21 Contribution of Irfu to T2K nd280m Electronics ASIC design, production test, readout electronic cards (design and production supervision) Physics Simulation and experimental data analysis of MicroMegas detectors Detector engineering and integration Design of Micromegas detectors, production coordination, QA Cooling mechanics and structure, cabling, services (HV, LV, fibers…) Firmware and software Embedded firmware and software in the front-end, slow control, DAQ for detector test bench, firmware and software for DCCs, LV control, etc. Project management M. Zito co-leader of TPC project with D. Karlen (Canada), A. Delbart, D. Calvet coordinator of “Micromegas modules” and “Readout Electronics” → Project relies on the diversity of competence at Irfu, teams above critical size and adequate funding

22. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr22 Project Difficulties Production First batch of wafers of the After chip did not pass QA → 2 months delay to get After chips First batch of FECs failed: humidity stored in After chip package during test requires extended baking procedure → more than 2 months to understand the problem and find workaround Defects on 10% FEM for first batches (broken via below the BGA) no consensus reached between designer, PCB manufacturer, and assembly company on exact cause (coating on vias?). → three month delay: defect analysis, modifications to PCB, new batch Could not solder some connectors on FEMs using lead free process Front-end production took 1 year while 6 months had been planned

23. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr23 Project Difficulties (con’t) Back-end electronics DCC prototype board did not become functional Unanticipated contribution of Irfu to back-end HW/FW/SW and DAQ. → Back-up solution based on customized commercial evaluation boards deployed by common effort of Triumf/Irfu/Lpnhe. Commercial Xilinx Virtex 4 ML405 Evaluation board Custom Clock board Custom Optical Transceiver board FGD DCC Crate

24. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr24 What really helped Commercial FPGA evaluation boards Inexpensive, readily available, customizable hardware Can develop and debug firmware/software on proven hardware –Firmware/software development before a dedicated board is available –Simplifies dedicated board debugging because firmware/software already tested and only migration issues need to be solved Common platform for collaborative development among various group → ~45 Xilinx ML405 boards purchased by the TPC-FGD groups! Used at Saclay, Paris, Rennes, Geneva, Barcelona, Hamburg, Tokai, Vancouver $995.00 Part Number: HW-V4-ML405-UNI-G

25. Saclay 2-3 December 2009calvet@hep.saclay.cea.fr25 Summary What? A 124.000 channel readout system for the 3 TPCs of the T2K experiment How? A new 72-channel ASICs based on a 511 time-bucket SCA Custom made readout cards at the back of each 72 Micromegas detectors Extensive use of commercial FPGA evaluation boards Principal merit? High density, low power: ~16 mW/channel (2 kW for 124.000 ch.); ~3 €/ch. Summary and Status 5 year development effort from concept to installation on site → TPC commissioning at J-PARC starting in the next few days