Tracker Week 21st April CCLRC, Rutherford Appleton Laboratory, Oxon, UK Imperial College, London, UK Brunel University, London, UK presented by John Coughlan RAL FED Status & Production Plans

Tracker Week 21st April U FEDv1 Production Summary Productions Jan 2003 : 2 boards. Working. June 2003 : 3 boards. Working. Oct 2003 : 6 boards. Major problems. Not for use outside UK. March 2004 : 6 boards. In production. New manufacturers. Locations of FEDs outside UK 3 at CERN in Tracker Lab. 1 at PISA since last Tracker week.  FEDv1 recent manufacture  FEDv1 test activities  FEDv2 design changes  Schedule

Tracker Week 21st April FEDv1 Recent Production 6 FEDv1s assembled by DDi Technologies, UK. Manufacture was carried out in a very professional fashion. Thorough Quality Controls. Good communications between technical staff at DDi and RAL. Testing: Automated Optical Inspection; Ersascope and X-ray on BGAs ; Flying probe tests on assembled boards. Delivered on schedule in March. All 6 boards are now fully working. 2 are at CERN since couple of weeks. Comprehensive production report also provided with suggestions for improvements for next pcb design... Conclusion: Very positive experience. Promising candidate for production manufacture.

Tracker Week 21st April FEDv1 Recent Production

Tracker Week 21st April FEDv1 Design Testing Tests completed since Feb Tracker Meeting:  Front-End FPGA Algorithms : Zero-suppression working.  S-LINK : High rate random 80MHz link readout working.  TCS : Throttle signals tested.  FPGA reprogramming: Compact Flash can now be reloaded via VME.  Baseline Firmware is complete.  Improvements are now being made to diagnostics, error handling, resets as described in URD v0.54 Hardware Testing  Power : Power & Currents well within limits. Original requirements of 300A on both 3.3V and 5V.  Temperature close to Front-End.  Protection : Over-temperature shutdown working.

Tracker Week 21st April S-LINK Tests (James Leaver) Hardware Throttle Slink Receiver Generic PCI Card VME Backplane FED Controller Slink Controller Control via VXI-MXI-2 LVDS Cable APVE Sim. FMM Sim. TCS Clock & Trigger FED Slink Transmitter Transition Card (ECAL)

Tracker Week 21st April S-LINK Tests (James Leaver) Trigger FED using APVE and read out test events via Slink – check received data for errors Current maximum Slink verification rate: No errors observed so far but at current data rate, would take ~146 days to guarantee no more than one error per week of CMS operation but at current data rate, would take ~146 days to guarantee no more than one error per week of CMS operation With random (Poisson distributed) 100 kHz triggers Sent ~1  10 8 triggers, events successfully received Verifying payload data transmitted at 100 kHz by sending triggers in controlled bursts (reducing average data rate to < 4  10 6 words/sec) No errors so far Will shortly begin testing FED performance in ‘Frame Finding’ mode, using fake data from FED Testers ‘Real World’ test of FED at high trigger/data rates Conclusion: FED can operate at 100 kHz

Tracker Week 21st April S-LINK Tests (James Leaver) FED FED Tester Sim. TCS APVE Sim. FMM Clock & Trigger from FT Hardware Throttle from FED Hardware Throttle from APVE (combines FED and APV buffer throttles) Optical (Simulated) APV Frame Data

Tracker Week 21st April Temperature Testing (Greg Iles) Will components on FED exceed operating temperature in CMS? Analogue OptoRx < 70°C Differential Buffer < 85°C ADC < 85°C Need to emulate conditions we will have in CMS LHC crate. All measurements adjusted for intake air at 18°C. 3 FEDs placed next to each other in crate. The 2 external FEDs restrict airflow and mimic to some extent heating from crate of FEDs 1 FT Ensemble used to drive all FED channels with 100kHz frames.

Tracker Week 21st April Position of temperature sensors Analogue OptoRx 0 ADC Differential buffer Analogue OptoRx 1 Analogue OptoRx 2 Not visible.... Built in FED OptoRx temperature sensor is on opposite side of the PCB

Tracker Week 21st April Temperature of ADCs & Buffers The following have temperature range of -40 to +85 °C ADC (AD9218) Differential buffer (EL2140) Predicted temperature in CMS by: Measuring temperature between 2 thermocouples One in air flow below crate One attached to top of component with araldite (near top of FED – i.e. where it gets hot) Added expected air temperature from heat exchanger (18 °C). Measured devices twice. All values with 2 °C. Worst case shown. Nominal fan speed of LHC crate = 3000rpm

Tracker Week 21st April Temperature of Opto Receivers

Tracker Week 21st April Caveats Assume heat exchanger can provide air at 18 °C. We will place a large amount of power in localised region of heat exchanger. 100kHz, “typical” frames that I have used match those in CMS. AOH driving minimum laser signal necessary for link (i.e. keeping power dissipation in experiment to minimum) FED in scope mode, 6 samples and delivering data to s-link oblivion. Only have 1 FED either side if FED under test and left hand side FED is at only ~70% power. Can we achieve a fan speed of 3480 rpm in a CMS rack. FEDs will provide a greater impedance to the air flow, but there will be more fans. How well is fan speed correlated to air flow? Heat Exchanger Hot Warm Cold FED Analogue FED Analogue

Tracker Week 21st April FEDv2 Pre-production Board Baseline functions of FEDv1 are now verified. Minimal Hardware changes for FEDv2:  Power Block : FET controller.  QDR Memory : Replacement part identified.  FE FPGA : 2M gate pin compatible with existing part.  ADC : AD9218 Device Bug. Reduce gain by half. Simple mod.  FPGA Configuration : VME Boot device reprogram via JTAG cable.  S-LINK & TCS Signals : New VME Transition Card.  TTCrx : Intermittent problems now understood. Simple fix.

Tracker Week 21st April FED Medium Term Plans Production Plans Q1/2004 : Finalise design changes for FEDv2. Sign off against FED User Requirements Document. Done. Q2/2004 : Implement changes FEDv2 and review. Q3/2004 : Manufacture couple of FEDv2s. Q4/2004 : Test FEDv2. Parts for 30+ FEDs now in hand or on order. EU Tender Q1/2004 : Place OJEC advert, invite EoI. Q3/2004 : Dispatch calls for Tender. Q1/2005 : Award contract.

Tracker Week 21st April FED Schedule

Tracker Week 21st April FED Software Enhancements 1. Interface to Fed9UDescription and Fed9UVmeDevice have been brought in line, so now you can replace your description object with a FedDevice object and it will work seamlessly. 2. First Stable release version 1 was delayed for the reason of testing the interface change thoroughly. Release will now go ahead at the end of this week 3. Fed9USupervisor now integrates the LOG4C logging and XML description configuration which has been tested in the LAB at CERN. 4. I would like everyone to move to DAQ version 2 and xerces version 2 ASAP. Then we can discontinue support of Xerces Version 1 in the release of FED software version The lib install has been tested for version 0.9 at cern in AFS, and works. So I am confident that we will not have to release source code to the main users. 6. Fed 9U Website now up and running. Contains release documentation of the high level interface to the FED software. Has links to the latest installers and to all previous install versions. Currently you have a separate installer for Xerces 1 and 2. one page will be the change log, which will be automatically updated every time we make a release. This is to be done this week to be available along with the first release. 7. Release script now uploads the latest release installer to the website, and will update the webpage contents automatically, so future management of the website content should be minimal. See Online Software Meeting

Tracker Week 21st April Summary Prototype FEDv1 6 more FEDv1’s made successfully. Commissioning tests at (CERN, Imperial & RAL) proceeding well. Baseline Hardware and Firmware design verified. Investigating Temperatures with number of FEDs in crate. Pre-production FEDv2 Implement design changes in Q2/2004 Manufacture minimum couple of boards in summer for test in Q4/2004. EU Tender Waiting for expressions of interest for 9U board production.

Tracker Week 21st April End Intentionally left blank

Tracker Week 21st April EU Tender Process EU Procurement Directives: Goods & Services > £150 K  Open Procedure : Any party can tender, suitable for off the shelf items (e.g. components)  Restricted Procedure : Only N selected parties are invited to tender. N to be specified in advance. Two part process. Market survey + Tender.  Publish OJEC Notice for Expressions of Interest. Can pre-notify selected companies. Associated Questionnaire. Rather general document. Elicit size, history of company, financial standing, standards adherence, facilities etc.  Evaluation and Selection of N companies to tender. Detailed enquiries. Visits.  Issue Tender. Detailed specification.  Still learning about these procedures. And how they fit with CERN (and CMS) procedures.  RAL is just about to issue a general pcb manufacture notice. Plan to use FED as example for tender exercise.  Some issues to decide, e.g. is separate tender for FPGAs necessary?

Tracker Week 21st April Restricted Procedure

Tracker Week 21st April Testing At Assembly plant. Do as much as possible. Discuss with companies. Positive responses. Q. Cost? Boundary Scan. Flying probe or ATE. VME Crate single card tests. (without Opto inputs, using internal test features). Major software effort to devise suitable tests with diagnostics. Draft document for “Reduced Test Setup at Assembly” just released. Full System Tests at RAL/Imperial Full crate tests. Interfaces to TTC. Full optical soak system tests before shipping. Installation and commissioning. Ship FEDs to Prevessin staging point. Repeat standalone soak tests? Install boards individually at CMS.

Tracker Week 21st April Spares Manufacture all in one go. Need 440 for CMS Tracker. 60 working spares (for lifetime of CMS). Guesstimate. Assume final 500 delivered passing all commissioning tests. Issues for later productions. Component availability. Memories. Expert availability. Design and Test. Changes in pcb and assembly processes. Standards. Markets. Pb Free Directive Failure rates Hard to determine yet. Based on previous experiments assume normal failure rate is low after commissioning, bath tub distribution (catastrophic crate loss?). Consider accelerated ageing tests offered by assembly companies? Assume most failures can be repaired. OptoRx can be removed.

Tracker Week 21st April Pb Free Directive We know it’s coming… WEEE July In principle a good thing. Japan already claims Pb Free. Military, Automotive exempt (but small markets). We may be exempt. Contract manufacturers constrained to switch for mainstream production. No drop in replacement to eutectic Sn/Pb solder. Alternatives require higher soldering temperatures. Manufacturing processes not fully understood. BGAs solder type? No long term experience of alternative solder e.g. failure in service. May require changing pcb base material from FR4. Would advise to make FEDs before industry switches…

Tracker Week 21st April First FED Prototype (Jan 2003) Primary Side (Secondary side has 1/2 analogue) OptoRx CFlash VME64x 9U board 34 x FPGAs ~40K-2M gates Analogue TTC Power Event Buffers 96 channels JTAG Boundary Scan Deliver FED “Package”: Hardware Firmware Software

Tracker Week 21st April First Fully Assembled Board Primary Side Secondary Side 9U VME64x PCB (2mm) 14 layers (incl 6 power & ground) 96 ADC channels : AD9218 Dual package MHz ~ 6 K components (smallest 0402) ; ~ 25 K tracks 1/2 Analogue circuitry on Secondary Side Highest density at Front-End Units FE Unit BGAs mm pitch Test with JTAG Boundary Scan

Tracker Week 21st April CMS Tracker FED Firmware Status Clocks Data Serial Controls VME LINK VME Bus VME SystemACE System ACE Clocks EPROM EPROM TTCrx QDR WriteQDR Read Serial Comms Headers TTC chanA VME LinkRegs S-LINKS-LINK Clocks Data Serial Comms Scope Mode Header Mode FIFOs Input Regs Serial Controls Scope Mode Frame-Findng Mode Output Input Regs Opto Rx DAC Opto Rx DAC DELAY FPGA x 3 x 8 FE FPGA x 8 BE FPGA VME FPGA ADC Under Simulation Under Test on FED Only for FEDv2 Controls Data Readout Control Throttle TCS Input Cluster Finding Mode Ed Saeed Ivan Ed Ed, John Saeed, Ivan Chan B I2C Temp “Working” on FED External Devices Temp 14th July 2003 To be Implemented QDR

Tracker Week 21st April FEDv1 Firmware Clocks Data Serial Comms VME LINK VME Bus VME System ACE System ACE EPROM TTCrx QDR Write QDRs QDR Read Serial Comms Headers TTC chanA VME LinkRegs S-LINK Clocks Data Serial Comms Scope Mode Header Mode FIFOs Input Regs Serial Comms Scope Mode Header Mode Output Input Regs Opto Rx DAC Opto Rx DAC DELAY FPGA x 3 x 8 FE FPGA x 8 BE FPGA VME FPGA ADC Under Simulation Under Test on FED Controls Data Readout Control Throttle TCS Input Cluster Mode Ed->Saeed Saeed Saeed Ed->Saeed Ed, John Saeed, Ivan Chan B I2C “Working” on FED External Devices Temp 15th March 2004 To be Implemented Spy Clocks FEDv2