HCAL/TriDas. November, 2003 HCAL TriDAS 1 Tridas Status Drew Baden University of Maryland November 2003

HCAL/TriDas. November, 2003 HCAL TriDAS 2 HTR Rev 3 (2003) Dual-LC O-to-E VME Deserializers Xilinx XC2V Stiffeners SLBs (6) TTC mezzanine Rev 3 – 30 boards made in March 2003 –Changes from Rev 1 (2002) Full 48 channel capability –Rev 1 was “half HTR” FPGA package –Change from FBGA (1.00mm) to BGA (1.27mm) Added stiffeners SLB placement –Moved to front-panel daughterboards Clocking –TLK2501 (deserializer) “refclk” »Can use crystal oscillator or TTC80 –Fanout scheme »Receive all signals on 1 quad Cat6 –Board production changes: New assembler, in-house X-ray, DFM review, QC Gold plated (Rev 1 was white-tin) for better QC

HCAL/TriDas. November, 2003 HCAL TriDAS 3 HTR Testbeam 2003 Results Clocking issues –We believe our clocking problems were a combination of Front-end difficulties Fiber cleanliness –Front-end capability to use either TTC derived clock or crystal oscillator TTC cleanup using Cypress robochip –Not rad hard, but meets GOL 100ps jitter spec Had more success with the crystal oscillator –Possibly a very nasty EM environment as well –We advocate running with UPS powering FE boards and VME crates in H2 QPLL implementation will be next –Fiber cleanliness We underestimated how critical this is. Will be more carefully done in the future Overall HTR implementation validated –Clocking, synching, data transmission, VME, etc. –But not for TPG part Checkout of connectivity in lab Checking SLB this month Firmware completed (as much as possible) TPG is the “easy” part of validation for production

HCAL/TriDas. November, 2003 HCAL TriDAS 4 Changes to HTR for Rev4 Moved 2 LC’s down to give more clearance for fibers –Upper rear of card Spread out routing of differential pairs for 6 SLB and 2 FPGA system clocks Removed hot swapping circuits –Relax hot-swappable requirement - worry about noise Front-panel changes –Rotary switch, LEDs, change mux selects from jumper to internal, etc Miscellaneous changes –Fixed what was found to be wrong with Rev3 board, add test points, terminate all unterminated I/O lines to SLB, VME byte swapping, other minor stuff for noise considerations Schedule –HTR Rev 4 is done – reviewing now. Ready for next stage of pre-production –Will probably submit by Dec 1 to make 30 boards Will stuff 3, checkout, then stuff the rest, that should get us ready for Testbeam 2004 and Slice Tests

HCAL/TriDas. November, 2003 HCAL TriDAS 5 TODO - TPG HTR production can begin after: –SLB/HTR/Wisconsin check Check that we can maintain link Setup ready…so far so good Measure BER, etc. –SLB/HTR connectivity check Logic analyzer card on SLB site already shows connectivity Waiting for firmware for SLB to verify for sure –Should be this week or next TPG Data validation –Will build a 6U VME board with sites for the Wisconsin Vitesse receiver boards –Will fifo data and read out over VME –Use this to check data validity, try different TPG tests, etc. –Plan to start on this board this week –Ready in a few months. Will use this for the early slice tests to test TPG output. FPGA Clock Input

HCAL/TriDas. November, 2003 HCAL TriDAS 6 TTC receiver (TTC_umd) General purpose TTC receiver board TTCrx ASIC and associated PMC connectors Will be used to receive TTC signal by HTR, DCC, and Fanout boards No signal receivers –Copper/fiber receivers must be on the motherboard –Signal driven through TTC connectors Tested successfully –Maryland, Princeton, BU, FNAL –Testbeam H2 Production: –Need 1 per HTR (~260) + DCC (20) + Fanout (10) –Need ~500 TTCrx for HCAL –Will layout test board for mass testing

HCAL/TriDas. November, 2003 HCAL TriDAS 7 HCAL Clock Fanout HTR clocks provided by a single 9U VME board –Chris Tully/Jeremy Mans from Princeton –Has fiber TTC input Signals fanned out over Cat6 twisted pair: –TTC stream To be used by each HTR and by DCC to decode commands & L1A –BC0 To be used by SLBs to synchronize TPGs –“40MHz” clock To be used by FPGA and SLBs to maintain pipeline – Comes from QPLL –“80MHz” clean clock To be used for deserializer REFCLK – Comes from QPLL Changes –TTCrx reset circuitry added –VME capability added Status –Final layout in hand. Waiting till the last minute. –Production can start mid January

HCAL/TriDas. November, 2003 HCAL TriDAS 8 Clock Distribution HTR TTC fiber TTC CLK80 BC0 CLK40 distribution to 6 SLBs and to 2 Xilinx Brdcst, BrcstStr, L1A O/E BC0 TTC FPGA.. Test Points for RxCLK and RxBC MHz.. TTCrx to Ref_CLK of SERDES (TLK2501) CLK40 CLK80 Princeton Fanout Board TTCrx QPLL HTR Cat6E or Cat7 Cable

HCAL/TriDas. November, 2003 HCAL TriDAS 9 HTR Firmware

HCAL/TriDas. November, 2003 HCAL TriDAS 10 TPG Path Still under development –The following is already coded/simulated but not tested in HTR TPG path tasks (not necessarily in order) –Linearize QIE data to 10 bits With.5GeV resolution gives 512 GeV max –Apply BCID filter Probably will sum over 2 buckets and assign based on high/low patter around the bucket that has the max energy –Sum or divide depending on HB, HE, or HF –Extract a muon window for the “feature bit” –Apply logic to eliminate false muons from shower leakage, etc. –Compress and send to SLB

HCAL/TriDas. November, 2003 HCAL TriDAS 11 TPG Path Schematic

HCAL/TriDas. November, 2003 HCAL TriDAS 12 Firmware/TPG Plans Firmware –Focus on TPG Path Latency –Measure and scrub, scheme for random latencies, meeting latency budget, etc. SLB checkout –Connectivity, localbus access, timing, data integrity, etc. This will be the main activity from now until we go into production. –Bells and whistles for error reporting/recover Meetings between Maryland, Princeton, and Boston groups More to be learned in 2004 HTR boards –Make Rev4 this fall, go into production early 04 ASAP given schedule Depends on results from above

HCAL/TriDas. November, 2003 HCAL TriDAS 13 Latency Definition: from BX to input to RCT HCAL O-E QIECCA HTR SLB RCT BX TOF To RBX DataTo RCT RBX HPD or PMT (HF) 46 clocks = 1,147.7ns GOL

HCAL/TriDas. November, 2003 HCAL TriDAS 14 TOF+Y11/fiber to RBX HCAL O-E QIECCA HTR SLB RCT BX TOF To RBX DataTo RCT RBX HPD or PMT (HF) 46 clocks = 1,147.7ns GOL TOFTx to RBX (7ns/m)TOTAL HB  =1 layer 1 (1.8m) 6.1ns(4.9m) 34.1ns40.2ns HB  =15 layer 9 (4.6m) 15.4ns(0.6m) 4.2ns19.6ns HE  =34 layer 1 (4.0m) 13.3ns(3.3m) 22.9ns36.2ns HE  =17 layer 14 (5.7m) 19.1ns(0.3m) 2.0ns21.1ns HF(10.9m) 36.3ns(2m quartz) 14ns50.3ns

HCAL/TriDas. November, 2003 HCAL TriDAS 15 Inside RBX CCA delays will be set using LED system TOF+TxO-ETotal to QIEClocksQIECCA+GO L Total HB40.2(HPD) 0ns40.2ns1.6 (2) HE36.2(HPD) 0ns36.2ns1.5 (2) HF50.3(1.5kv PMT + 6m coax) 14ns + 30ns = 44ns 94.3ns3.8 (3) CCA operation needs more experience... HCAL O-E QIECCA HTR SLB RCT BX TOF To RBX DataTo RCT RBX HPD or PMT (HF) 46 clocks = 1,147.7ns GOL

HCAL/TriDas. November, 2003 HCAL TriDAS 16 Digital Data Fiber Measured  = 2/3 (51m adds 250ns delay) –Gives 4.99 meter/clock tick LHC RF frequency) Default fiber length 90m gives clocks –HE is longest due to routing around ME outer radius Change HE fiber routing to go via EE inner radius –We believe we can shave off 15-20m 3 clocks – 15 total maybe more? Thru RBXData FiberTotal HB HE HF HCAL O-E QIECCA HTR SLB RCT BX TOF To RBX DataTo RCT RBX HPD or PMT (HF) 46 clocks = 1,147.7ns GOL

HCAL/TriDas. November, 2003 HCAL TriDAS 17 Random Latencies in HTR Latency due to –TI deserializer (TLK2501) Advertised random latency with respect to reset 76 to 106 bit times –Remember, TLK2501 has 20-bit frames (80MHz frame clock) –Latency will be 47.5 to 66.9 ns random We measured the random latency at UMD –Latency difference is randomly distributed between 0 and 6ns. –Factor of ~1/3 of TI spec. Will investigate if this is “typical” –Asynchronous fifo Relative phase between recovered clock and refclk will introduce a random latency of 0 to 1 clock tick TLK2501 Common 80MHz system clock = 40MHz clock/2 from TTC Recovered clocks 80 MHz system clock

HCAL/TriDas. November, 2003 HCAL TriDAS 18 Latency in HTR Measured: –From input to GOL, through 7m fiber, through HTR firmware to SLB input connectors –Total: 395ns = 16 clock ticks –GOL takes 2 clocks, 7m fiber takes 1.4 (call it 2), remainder is ~12 clock ticks Relative phase between GOL and HTR system clock and TLK refclk varied –Latency seen is some combination of TLK and asynchronous fifo –Measured for a single Temp and VCC Need to repeat with variations Thru RBXData FiberHTRTotal HB HE HF HCAL O-E QIECCA HTR SLB RCT BX TOF To RBX DataTo RCT RBX HPD or PMT (HF) 46 clocks = 1,147.7ns GOL

HCAL/TriDas. November, 2003 HCAL TriDAS 19 SLB + TPG Cables SLB nominal latency 3 clock ticks –SLB job is to make sure all HCAL/ECAL towers from same BX arrive at RCT in synch by… Histogram TPGs to find the LHC beam structure –Find “BC0_DATA” Adjust with BC0 broadcast timing (TTCvi) so all partitions are in synch –“BC0_TTC” Delay accordingly knowing | BC0_DATA  BC0_TTC | We might have to suffer 1 or 2 clock ticks here – we don’t know yet TPG Cables –20m nominal 2xdual skew-clear: 5m/clock tick –Discussions so far indicate reducction to 15m cables quite possible. Results in 3 clock ticks for TPG cables Total –3(SLB nominal) + 1 (SLB contingency) + 3 (TPG cables) = 7 clock ticks

HCAL/TriDas. November, 2003 HCAL TriDAS 20 Grand Total Caveats and assumptions –Numbers for TOF+prog delay are correct (checked) –75m max for all fiber cables (saves 3 clocks) –CCA can be run with a latency of 3 and still do its job (5 max for the chip) –3m TPG cables –Summing in HTR firmware does not add to latency –Total delay through SLB < 4 (1 for contingency) Need experience from this fall’s tests Thru RBX Data Fiber HTRSLB/T PG Total HB HE HF Possible further scrubbing –CCA can perhaps take up some of the slack for some of the non-integer TOF+propogation delays (1clock) –Digital fiber cabling (every 5meters = 1 clock!) –“Tricks” in HTR firmware Some parallelism, routing constraints, faster LUT access… –Other clever ideas Possible further additions to total latency –TOF+propogation (HF delays thru PMT and coax, need measurements) –Digital fiber cabling could go back to 90? HCAL policy: 75m cables or less –Summing in HTR firmware Need a MC study to tell us Default should be no summing unless MC study tells us otherwise –Difficulties with HCAL/SLB timing Need experience

HCAL/TriDas. November, 2003 HCAL TriDAS 21 Project Timeline Firmware TB 03 firmware should be adequate Boards Can use current crop of Rev3 To do: Commission QPLL, global clocking. Level 1 Trigger: SLB, TPG, latency… “Vertical Slice” (~March) 2005 HB, HE, and ME plus Level 1/TPG Fiber synchronization Check in alcove…compare environments Firmware TPG firmware has to be ready. Fall 03 task. Boards Must have Rev4. How many TBD. FE Commissioning (~Feb) “Magnet” Tests (no HCAL) CMS Magnet Integration System Test Boards Will have Rev4. H2 Testbeam Fiber synchronization Check in H2…compare environments Boards Will have Rev4. How many TBD.