Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker Readout Effort Requirements in Instrumentation Department John Coughlan

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker Readout Summary Overview. Scope of project in INS Team members Project Status List of tasks to completion Effort required Summary & Comments

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker Readout Overview Task: To readout a very large Silicon Tracking detector. ~ 9 million Silicon Strip channels ON Detector: 73K APV25 pipeline L1 Trigger: MUX APV Frame output Analogue Data readout via Optical links (APV Frame: Header + Strip Data) => INS Effort : essentially complete OFF Detector: Front-End Drivers (FED) Digitise / Zero Suppress / DAQ readout ~ 500 x 9U VME64x boards (incl spares) 96 ADC channel boards => INS Effort : major activity DAQ Counting Room On Detector (Hard Radiation) FPGA 25 VME 9U FEDs Hybrid Front-End Hybrid Silicon Strips 70m

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED Block Diagram VME-FPGA TTCrx BE-FPGA Event Builder Buffers FPGA Configuration Power DC-DC DAQ Interface 12 Front-End Modules x 8 Double-sided board CERN Opto- Rx Analogue/Digital 96 Tracker Opto Fibres VME Interface Xilinx Virtex-II FPGA Modularity 9U VME64x Form Factor Modularity matches Opto Links Data suppression board. 8 x Front-End “modules” OptoRx/Digitisation/Cluster Finding Back-End module / Event Builder VME module / Configuration Power module Other Interfaces: TTC : Clk / L1 / BX DAQ : Fast Readout Link VME : Control & Monitoring JTAG : Test & Configuration FE-FPGA Cluster Finder TTC Temp Monitor JTAG 9U VME64x

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED Data Processing VME-FPGA TTCrx BE-FPGA Event Builder Buffers FPGA Configuration Power DC-DC DAQ Interface 12 Front-End Modules x 8 Double-sided board CERN Opto- Rx Analogue/Digital 96 Tracker Opto Fibres VME Interface Xilinx Virtex-II FPGA Data Suppression 9U VME64x Form Factor Analogue: 96 ADC channels ( MHz L1 Trigger : processes 25K MUXed silicon strips / FED Raw Input: 3 Gbytes/sec* after Zero Suppression... DAQ Output: ~ 200 MBytes/sec L1 max rate = 100 kHz) Digital Processing Flexible Digital Logic: Xilinx Virtex-II FPGAs 40K->3M gates*FPGAs programmed in VHDL & VERILOG FE-FPGA Cluster Finder TTC Temp Monitor JTAG 9U VME64x

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED Board Layout Primary Side Complex board Pushing density limits on large board for cost Opto & Analogue & Digital issues Brings together considerable INS Dept expertise and experience from several areas into one project

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED Board Layout Primary Side View thru the board Double sided

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 FEDv1 Front-End module Primary Routed Note: FPGA de-coupling OptoRx ADCs 40K FPGAs1500K FPGA OpAmps 12 optical channels Dense circuitry

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 DAQ Front-end Readout Link FRL DAQ Mezzanine Card Transition Card S-LINK64 CMS Tracker FED Transition card option TTCrx BE-FPGA Event Builder Buffers 12 TTC Due to mechanical constraints place DAQ link card on Transition card Very simple card

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED Crate Layout (illustration) 440 Boards96 ADC/Board 24 Crates 8 Racks 440 Boards96 ADC/Board 24 Crates 8 Racks 40 K ADC Channels10 Max Trigger Rate100 kHz Input Rate1.5 T Byte/s Output rate25 Gbyte/s/% 40 K ADC Channels10 Max Trigger Rate100 kHz Input Rate1.5 T Byte/s Output rate25 Gbyte/s/% 4 TTC Partitions DAQ FED

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED INS Main Deliverables (Project Spec) Includes Design of 9U FEDs. Electrical tests of design. Design & Testing Baseline FPGA Firmware. Production of prototypes for CMS silicon tests. Production and Testing of ~ 500 FEDs (incl spares.) Transition cards for DAQ links (tbd) Low level driver software. Full Documentation. Communication with CMS groups Assistance during Tracker installation & commissioning. Excludes OptoRx design & testing. Opto Test equipment for FEDs. DAQ link mezzanine card. High level software. Provision of VME crates and cabling. Other modules in FED VME crates e.g. CPUs. FED project institutes: RAL INS RAL PPD Imperial College Brunel

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED Future Team Members in INS Core Team John Coughlan Project Manager ; Stakeholders contact Software Saeed Taghavirad Hardware Design ; Firmware Ivan Church Test Engineer ; Firmware Plus Ed Freeman Firmware James Salisbury Analogue Design Rob Halsall Technical Advisor Chris Day PCB Design Others Procurement Test Section Moving from Design phase to Test phase… Principal responsibilities are shown but most personnel have flexible/multiple skills. Several with past experience on similar projects.

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED Project Status 1998->2001 FED-PMCs x 60 produced for CMS Silicon module testing. Small nr channels with limited functionality. Cheap. Very flexible. Plug in PC or use in VME crate. Very popular with CMS! (and others) 2001 Final 9U FED design started in earnest. Requirements document produced. Algorithms studied. Board interfaces investigated. Choice of (NEW) FPGA family made. (Cost critical) Cluster finding algorithms implemented in Firmware Implementation of design; 96 chan ADC on 9U. Analogue design complete (OptoRx finalised by CERN). DAQ interface baseline decided. Basic simulations and test bench measurements made. Board Design complete. Components chosen. Board layout & Routing. Final Review carried out. Manufacture of first 2 x FEDv1s imminent. Firmware for Event Building and VME continuing in parallel. Software design started. FED-PMC (FED prototype)

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED FEDv1 Status VME-FPGA TTCrx BE-FPGA Event Builder Buffers FPGA Configuration Power DC-DC DAQ Interface 12 Front-End Modules x 8 Double-sided board CERN Opto- Rx Analogue/Digital 96 Tracker Opto Fibres VME Interface Xilinx Virtex-II FPGA FE-FPGA Cluster Finder TTC Temp Monitor JTAG 9U VME64x Board Status

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED INS Dept Outline Tasks 02/03 Manufacture 2 x FEDv1. Complete VME Firmware (EF) [4 months] Complete BE Firmware (ST) [4 months] Basic board tests Power/JTAG/Analogue (JS, IC) [3 months] Primitive Driver Software (JC) [4 months] 03/04 Test of FEDv1 Design: (by Designers) Advanced board tests (Opto IC) (IC, ST) [6 months] Firmware (Front End & Event readout) (EF, ST) [6 months] VME Interface (EF) [6 months] DAQ interfaces (with Imperial) (ST, JC) [6 months] Transition board (ST) [3 months] Driver Software (JC) [6 months] Production of 20 FEDv1s for CMS tracker testing. (IC, ST, JC) [6 months] 04/05 Pre-Production and Test of 20 x FEDv2. (ST,IC,JC) [12 months] Procurement for 500 FEDs. (JC) [9 months] Set up for “automated” Production Tests. (IC,ST,JC) [12 months] 05/06 Manufacture of 500 x FEDv3. (ST,JC) [12 months] Production RAL (IC +) [12 + months] Assist Installation & CMS. (JC,ST) [6 months] 06/07 Complete CMS. (JC,ST) [9 months] Operation

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED Schedule Tracker/FED installation schedule delayed by ~ 9 months. FED production & installation now expected in 2005 DesignTest Production & Installation Pre-ProNOW-> FEDv3FEDv2FEDv1

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED INS Dept Effort Approx. 6 months behind original FED schedule. (but Tracker installation is also delayed). Spent more time and effort getting design of FEDv1 close as possible to Production requirements. (OptoRx and DAQ link specifications came later than expected). Gross manpower figures are still reasonable. But no slack. Profile needs adjustment. DesignTestProduction & Installation Includes APV effort ~ 2 SY integrated

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED INS Dept Effort Year by Year (FYs)

Instrumentation Department John Coughlan Rutherford Appleton Laboratory14 November 2002 CMS Tracker FED Conclusion & Comments INS has an excellent track record in delivering fully tested large scale electronic systems which are maintainable for the long operational lifetimes of Particle Physics experiments such as CMS. Future APV effort minimal (consultancy) Gross INS effort projections for FED stand given following assumptions... Effort profile is adjusted to reflect schedule changes (incl carry overs). No major re-design of FEDv1 to FEDv3 for production. High quality of Board manufacture. No extra deliverables from CMS (e.g. Firmware algorithms, OptoRx spec changes). No further schedule slippage (installation in 2005) Personnel with key skills (eg. FPGA firmware) stay with project. Assistance in automated FED production testing. Absolute minimial support for FEDs for CMS tracker module testing. Level of Installation & commissioning support from INS to be agreed. NB No contingency is included (e.g. for staff turnover). INS has an excellent working relationship with Imperial College and RAL PPD Many tasks are shared.