LAr ROD R&D for sLHC Charlie Armijo, Joel Steinberg, Ken Johns (University of Arizona) Joe Mead, Hucheng Chen, Francesco Lanni (BNL) Luis Hervas (CERN) Andreas Meyer, Andy Kielburg-Jeka, Arno Straessner (Dresden) Guy Perrot (LAPP) Gloria Blasetti, Mauro Citterio (INFN, Milano) Dean Schamberger (SUNY, Stony Brook) 5/26/20091H. Chen - Brookhaven National Laboratory

Readout Driver (ROD) Upgrade R&D Goals – Prototype a ROD that can receive and process continuously digitized detector signals from proposed front-end architecture – Investigate and evaluate several critical technologies which could be used in the design of next generation ROD R&D Interests – High speed optical link & FPGA SERDES Data bandwidth of entire LAr (1524 FEBs) > 150Tbit/s Industry standard parallel fiber connector MPO/MTP, which has 12 fibers and is about 15mm wide Xilinx & Altera FPGA embedded SERDES – Low latency lossless data compression/decompression algorithm Reduce number of links between FEBs and RODs – FPGA based Digital Signal Processing Take the advantage of parallel data processing Make use of the readily available DSP resources in FPGA – Explore different system level architecture: AdvancedTCA Take the advantage of industrial standard and high availability features: redundancy, shelf management protocols, power management, fabrics High speed serial bus, PCI Express, RapidIO, 1Gbit/10Gbit Ethernet – Interface to Digital Level 1 Calorimeter Trigger Perform level 1 trigger sum digitally Flexibility: adjustable granularity, sliding window Latency: optical fiber link, energy/timing calculation, trigger sum Data distribution (real time data replication) to level 1 system – Interface to Readout Buffer (ROB) Explore the possibility to integrate ROB on ROD 5/26/20092H. Chen - Brookhaven National Laboratory

ATCA Sub-ROD Development Following up of the Sub-ROD v1.0 design with many improvements – AdvanceTCA 8U Module Fabric interface to implement Gbit, 10Gbit Ethernet, RapidIO or PCI Express protocol Update channel to facilitate the communication between adjacent modules Rear transition module interface for off crate communication – Xilinx Virtex 5 FX series FPGA (XC5VFX70T) 6.5Gbit/s RocketIO GTX transceiver PowerPC 440 microprocessor up to 550MHz operation 550MHz DSP48E slices – 75Gbit/s Parallel Fiber Optic Transceiver Reflex Photonics 75Gbit/s SNAP 12 InterBOARD parallel fiber optic transmitter and receiver Channel data rate of up to 6.25Gbit/s Long lead time – expected to be received in July – Expansion Slot on Board Samtec connectors with single ended and differential connections to on board FPGA Daughter board to implement additional functionalities 5/26/20093H. Chen - Brookhaven National Laboratory

5/26/20094H. Chen - Brookhaven National Laboratory

Sub-ROD Injector Development Sub-ROD Injector Function – Data injector for Sub-ROD – Compare/contrast Altera/Xilinx FPGAs SERDES DSP capabilities NIOS/Microblaze/PowerPC Successfully integrated with Sub-ROD v1.0 in Sept Recent work – All transmitter channels successfully 2.4Gbit/s with 2 7 PRBS – All receiver channels passed preliminary testes – Began using external memory for data – Began testing serial links for memory 5/26/20095H. Chen - Brookhaven National Laboratory

5/26/20096H. Chen - Brookhaven National Laboratory

Status of ATCA Sub-ROD & Sub-ROD Injector ATCA Sub-ROD – Fabricated and assembled – Power test is done – Ethernet test is done – Parallel fiber optic link test is going on – DDR2 memory test is going on – Tests of ATCA platform – Tests of cross board communication Sub-ROD Injector – Altera Statix GX II FPGA is capable of running Gbit/s – Tests at 3.2Gbit/s – Tests at 6 Gbit/s – Tests with 2 23 and 2 31 PRBS – Move to AMC form factor and function Integration test is planned in Aug./Sept Good Evaluation Platform for Collaborators – ATCA architecture evaluation – 12x6Gbit/s parallel optical transmission evaluation – FPGA DSP/embedded microprocessor evaluation – Flexibility to evaluate different serial protocols – Easy expansion of functionalities with daughter board 5/26/20097H. Chen - Brookhaven National Laboratory

Modeling Pile-up Generate waveforms at high luminosity to test energy and timing calculations Program – Loop over bunch crossings (BC) – For each crossing, generate Poisson random number r with mean = – For each i = 1,..,r generate energy E i from MC Convert E i to ADC (peak) Generate ADC waveform W i that spans 30 BC’s – Sum W i for all events in this BC – Sum over W i all BC – Add pedestal value Presently we are tuning the code to ensure sensible results at LHC luminosity Expect results by the next meeting 85/26/2009H. Chen - Brookhaven National Laboratory

9 EM3-Barrel Cell Energies UL (min bias), UR (j0), LL (j1), LR (ttbar) 5/26/2009H. Chen - Brookhaven National Laboratory

Waveform Shape of waveform taken from shaper plus empirical negative part 105/26/2009H. Chen - Brookhaven National Laboratory

Some Relevant Ideas Multiple Fiber Bundle – – Rapier – Easy Break-Out Cables Up to 432 fibers – Multi Loose Tube Cable Up to 720 fibers – Afumex Up to 864 fiber 5/26/200911H. Chen - Brookhaven National Laboratory

Some Relevant Ideas Optical Transceiver – MTP/MPO SNAP – Parallel Active Optical Cables – LightABLE Parallel Fiber Optical Engine FPGA SERDES – Xilinx Virtex-6: 40nm, 11.2Gbit/s Not available yet – Altera Stratix IV: 40nm, 11.3 Gbit/s Overpriced – Xilinx Virtex-5Q: Defense grade FPGA 5/26/200912H. Chen - Brookhaven National Laboratory

Some Relevant Ideas From ATCA to MicroTCA – ATCA Carriers – Advanced Mezzanine Card – AMC – PICMG AMC.0 specification – MicroTCA – PICMG MTCA.0 specification xTCA for Physics Coordinating Committee – Formed in Mar under PICMG – Coordinate Physics issues with PICMG – Growing number of labs using xTCA products need: Analog & Digital IO for AMC, ATCA, MicroTCA Software design for full xTCA management, redundancy features Fast timing and synchronization Rear Transition Module scheme for AMC/MicroTCA – “Request” design from industry If it is according to any standard, industry sees change to make profit 5/26/200913H. Chen - Brookhaven National Laboratory

Development Plan of Next Version of ROD ATCA 8U module as mother board – Serve as generic mother board – Sub-ROD daughter board – Sub-ROD Injector daughter board AMC module as daughter board – Relatively independent – Can be tested in MicroTCA chassis – Cost effective Call for user request of functionalities – Welcome contributions/ideas from collaborators – Regular phone meeting every 2 or 3 weeks – Discussion of ROD development – Discussion of pile-up simulation 5/26/200914H. Chen - Brookhaven National Laboratory