1. USCMS Mayl 2003 1 TriDAS Update Drew Baden University of Maryland http://www.physics.umd.edu/hep/HTR/hcal_may_2003.pdf USCMS HCAL

2. USCMS Mayl 2003 2 Shield Wall SBSSBS HPD FE MODULE 12 HTRs per Readout Crate, 2 DCC FRONT-END RBX Readout Box (On detector) READ-OUT Crate Trigger Primitives Fibers at 1.6 Gb/s 3 QIE-channels per fiber QIE CCA GOL DCCDCC TTC GOL CCA HTRHTR HTRHTR CAL REGIONAL TRIGGER 32 bits @ 40 MHz 16 bits @ 80 MHz CCA S-Link: 64 bits @ 25 MHz Rack CPU FE/DAQ Electronics CLKCLK HTRHTR

3. USCMS Mayl 2003 3 DCCDCC  Production boards made  Logic boards made oFirmware shakedown  LRBs made  DCC motherboards successfully tested testbeam 2002 Spare Standard PMC Site (33MHz 64 bit) 3x Link Receiver TTCRx Fast Timing/ Control 235 pin 2mm Connector DAQ S-LINK64 DCC Logic Mezzanine Card

4. USCMS Mayl 2003 4 HTR Principal Functions 1. Receive front-end data for physics running  Synchronize optical links  Data validation and linearization  Form TPG’s and transmit to Level 1 at 40 MHz  Pipeline data, wait for Level 1 accept oUpon receiving L1A:  Zero suppress, format, transmit to the concentrator (no filtering)  Handle DAQ synchronization issues (if any) 2. Calibration processing and buffering of:  Radioactive source calibration data  Laser/LED calibration data 3. Support VME data spy monitoring  Will adhere to CMS VME64 standards

5. USCMS Mayl 2003 5 HTR Status Rev 1 run Summer 2002 testbeam  Board worked well – all functional requirements met  Big concern on mechanical issues for production oHad a difficult experience with previous board manufacturing Rev 2 produced March 2003  Board production changes: oNew assembler, in-house X-ray, DFM review, QC oGold plated (Rev 1 was white-tin) for better QC  Changes to HTR: oChange from Virtex 1000E FBGA (1.00mm) to Virtex2 3000 BGA (1.27mm) oAdded stiffeners oMoved all SLB/TPG output to front-panel daughterboards oModified Rx refclk scheme (the usual TTC/refclk clocking concerns)  Full 48 channel capability (Rev 1 was “half HTR”)  As of this date, no issues – this board is functionally a success

6. USCMS Mayl 2003 6 HTR Rev 3 No more design changes – this is the final HTR 30 boards delivered April 21  As of Friday (May 2) 12 have gone through final checkout oAll systems except connectivity to SLB oFiber links checked out at 1.7Gbaud bit rate (1.6Gbaud is CMS requirement)  Frame clock up to 2.0Gbaud bit rate and it stays synchronized  No BER yet…will do a lab measurement soon  12 boards x 16 links ~200 links(~5% of total) with no problems  Minor adjustments will be needed for front panels, stiffeners, etc.  Will battle test these boards this year oTestbeam to begin this month oVertical Slice tests after summer

7. USCMS Mayl 2003 7 HTR Rev 3 (cont) 16 Dual-LC O-to-E VME Deserializers Xilinx Stiffeners 6 SLBs TTC mezzanine

8. USCMS Mayl 2003 8 HCAL Clocking DCC – no difficult synchronization issues here For HTR, need 2 different kinds of clocks: 1. Synchronized LHC clock for Xilinx system clock and SLBs oMaintain phase synchronization with entire CMS pipeline oAllow SLBs to do their job oFrequency jitter requirement not critical 2. Precise 2xLHC clock for Deserializer refclk ONLY o30-40ps pkpk jitter spec oUsed ONLY for deserializers oPhase synchronicity with LHC clock not important Princeton fanout board will receive TTC, clean up clocks with QPLL, fanout signals

9. USCMS Mayl 2003 9 Princeton Fanout Board

10. USCMS Mayl 2003 10 Clock Distribution HTR TTC fiber TTC CLK80 BC0 Cat6E or Cat7 Cable (very low X-talk) CLK40 distribution to 6 SLBs and to 2 Xilinx Brdcst, BrcstStr, L1A O/E BC0 TTC FPGA.. Test Points for RxCLK and RxBC0.. 80.0789 MHz.. TTCrx to Ref_CLK of SERDES (TLK2501) CLK40 CLK80 Princeton Clock/TTC Fanout Board TTCrx QPLL

11. USCMS Mayl 2003 11 TTC receiver - TTCumd General purpose TTC receiver board (TTCumd)  TTCrx ASIC and associated  PMC connectors Will be used to receive TTC signal by HTR, DCC, and clock fanout boards No signal receivers!  Copper/fiber receivers must be on the motherboard  Signal driven through TTC connectors Tested successfully by Maryland, Princeton, BU groups

12. USCMS Mayl 2003 12 HTR Integration Goals 2003 Continued development of HTR firmware  Commission TPG path oFirmware, LUTs, synchronization, SLB output…  Monitoring, error reporting, etc. (information sent to DCC) Testbeam May 2003  Support calibration effort and continue commissioning the system  Run synchronously in May Vertical slice tests, Fall 03  Fully pipelined, monitoring, TPG, DAQ, synchronization, clocking…. Develop software to support DAQ activities  Testbeam software improvements oPrinceton group built testbeam DAQ  Software for commissioning oAllow us to verify fiber mapping oDownload LUTs, firmware version, etc.

13. USCMS Mayl 2003 13 HCAL TPG Under development…  Preliminary FPGA code for TPGs done oLUT for linearization (downloadable), 0.5GeV steps, 255Gev max E T oE to E T and sums over as many as 7 channels  Not implemented in code yet…TBD oMuon window in E oBCID filter algorithm TBD from testbeams oCompression LUTs for output to SLBs  Utilization is ~50% of Virtex2 3000 oWe are confident this chip will be sufficient  Simulation effort under way… Latency issue  See below – we are working on this…

14. USCMS Mayl 2003 14 HTR Production Full contingent of HTRs: 260 boards  Includes 10% spares, 20% spares for parts Full production will begin after:  Testbeam demonstrates I/O works under battle conditions  Successful testing of the 6 SLB daughter card functions  Understanding of how to meet latency issues o We are still some clock ticks short, but firmware is still very immature for the TPG part of the HTR (see slides below) If all goes well…sometime this summer or fall  There is no reason to hurry other than to finish with the R&D part of the project  We are confident that the current board design will be final

15. USCMS Mayl 2003 15 Overall Commissioning Schedule Summer 2003 testbeam  Repeat previous test w/production prototype boards Fall 2003 Slice tests  HCAL will join as schedule allows 2003/2004 HCAL burn-in  Continue with firmware development/integration as needed 2004/2005 Vertical Slice and magnet test  We will be ready  All HCAL TriDas production cards involved October 05 beneficial occupancy of USC  Installation of all racks, crates, and cards  We do not anticipate any hardware integration oShould be all firmware / timing / troubleshooting

16. USCMS Mayl 2003 16 ESR Review Item 1 “Use of an obsolete TI component for the data link receiver” FPGA Xilinx XC2V LC TI LC Misconception on the part of the committee oTI TLK2501 is NOT obsolete. This is a Gigabit ethernet transceiver. There is no reason to believe TI will stop making these parts. If they do, someone will make something else compatible. oStratos receivers are also NOT obsolete. Dual receivers are out of favor, Transceivers are in favor What is obsolete is our $99/part. If we need more, they will charge $133/part (or more)

17. USCMS Mayl 2003 17 ESR Review Item 2 “The random latency problem that comes with using the 8bit/10bit link protocol” The “random latency” has to do with the TI Serdes function  Two clocks here: incoming data clock and reference clock  Serdes part has an internal asynchronous FIFO to implement 8B/10B protocol  But this is NOT the fault of the protocol! oAny protocol which includes a clock, to be recovered, will have this. TI does have a 2-3 clock tick random latency with 50% probability for 2 or 3  We can use VME controllable reset and comparison to achieve the 2 clock tick lesser latency  Can readout SLBs and use relative latency to correct pointers  Can use FE BC0 signals

18. USCMS Mayl 2003 18 ESR Review Item 3 “Routing of large no. of stiff cables to the front of the HTRs versus other configurations such as a transition module”  Transition module is NOT POSSIBLE. Forget about this. oWould cost us 6 months at least (time and engineering $)  Strain relief: oEach HCAL rack will have 2 VME 9U crates oEach 9U crate will have an accompanying 6U strain relief panel oChanging to 15m quad cables (from 20m 2xdual “Wesley” cables) will greatly reduce torques on SLB cards  We will test these cables this summer – need Wisconsin Vitesse test setup oEach SLB card will be attached to the HTR front panel, and screwed into HTR motherboard  We believe this will work fine.

19. USCMS Mayl 2003 19 ESR Review Item 4 “Ensuring appropriate quality assurance and testing at the HTR board fabrication facility” We agree, this is a big worry.  Have used new high-tech assembler for Rev 3 (pre-production)  Note: almost any assembler will have startup issues oOverall techniques are more important than QA, which comes after the fact oWe have chosen an assembler with very modern (and expensive) equipment. oAn engineering review by the assembler is included in the assembly cost oOur biggest problem was fine-line BGA (1.0 mm pitch) implementation  Current version uses standard 1.27mm pitch BGA  Given current experience, we believe we have solved this…

20. USCMS Mayl 2003 20 ESR Review Item 5 “Providing sufficient FPGA excess capability against possible future enhancements to the firmware” HTR FPGA change: Virtex/1000E to Virtex2/3000  Current firmware uses o83% of all RAM resources  FIFOs, LUTs, etc. this will not change o50% of all Logic resources  Room for more logic  Room for more memory (can use distributed memory)  The sky is not the limit, but we think we’re ok here oFirmware has evolved quite far thanks to Tullio Grassi’s efforts

21. USCMS Mayl 2003 21 ESR Review Item 6 “ Minimizing the trigger latency” ItemLatency TOF.5 HCAL Optics1 FE (CCA+QIE)8-9 GOL2 Fiber Tx to HTRs18 Deserializer2-3 HTR Alignment6 HTR TPG path5-10 SLB3 TPG Cables4 TOTAL50 - 57 Current total 50 – 57 clocks  Very rough guesses oMany numbers have not been measured Optimizations:  Fiber cables need to be 90m?  HTR firmware needs optimization  Deserializer random latency fix  TPG cables changed to 15m will save 1 tick  Others…main efforts over next 6 months

22. USCMS Mayl 2003 22 TPG Path E T comp 7 10 Muon bit Sum Consecutive Time-samples 9 TP 8 QIE-data INPUT LUT Lineariz. and E t E T [9:0] 2 Compression LUT 2 Muon LUT 1 Delay to synchronize with BCID 10 L1 Filter 10 Sum in E T Peak Detection TP_Bypass 1010 2 2 2 Mask & Reset “NO-SHOWER” LUT take care of cases where showers can leak into a cell and incorrectly set the muon bit. BCID BCID avoids to flag as a muon the tail of a more energetic event

23. USCMS Mayl 2003 23 Other ESR Concerns Reliability/maintenance  Replacement of HTRs not an issue – HTRs not in hi-rad region Data link error detection  Not difficult to implement, just requires coordination.  Under consideration, schemes are evolving, dealing with e.g. oLoss of synch oTrigger acceptance violations oBuffer overflow (actual and warnings so DCC can cause L1 to throttle) oUse of BC0 from front end oInline pedestal determination oZero suppression oDAQ format