Links & Clock distribution: toward the TDR A. Aloisio, R. Giordano aloisio@na.infn.it, rgiordano@na.infn.it University of Naples ‘Federico II’ and INFN ETD Meeting - CERN - Nov. 17th, 2011

ETD Meeting - CERN - Nov. 17th, 2011 Overview Toward the TDR Does a Golden Solution exist ? What if ... Making the system working Manpower Discussion ETD Meeting - CERN - Nov. 17th, 2011 2

ETD Meeting - CERN - Nov. 17th, 2011 Toward the TDR Setting the scene of links and clock distribution in the ETD Many key design elements are still missing (rad dose, partitioning, technology) Devices non yet available today will be obsolete in a few years from now Trends, way-of-designing, paradigms survive better, can contribute indeed to the TDR ETD Meeting - CERN - Nov. 17th, 2011

Links & Clocks: taking the golden section In our view, high speed serial links should be the golden solution to distribute ALSO the clock In our view, their implementation should rely on FPGA as much as possible IDEA DESIGN ETD Meeting - CERN - Nov. 17th, 2011

ETD Meeting - CERN - Nov. 17th, 2011 FPGA pros and cons Pros: Uncountable... Cons just one: Rad-tolerance Rationale A) test on beam the newest FPGAs B) keep an eye on the silicon vendors C) Check what ‘competitors’ do D) Have ready a benckmark design E) Keep simulating the background ETD Meeting - CERN - Nov. 17th, 2011

Testing FPGA ‘a-la-carte’ First test on Xilinx V5 – Jul. 10 Tested TMR vs. noTMR moderation Config. Memory cross-section System cross-section of a real design Power issue: negligible TID effects: negligible Latch-up: None SEFI: minor effects VERY detailed fault analysis presently being conducted (see next slide) Three more testbeams coming: Dec. 10 (V5-V6) then Mar 2012 (V6), Jul 2012 (V6-V7) No more beam will be available after that ETD Meeting - CERN - Nov. 17th, 2011

Fault Analysis (from last TB) OPEN PIP: net "GLOBAL_LOGIC0_451" gnd, outpin "XDL_DUMMY_INT_X2Y49_TIEOFF_X2Y49" HARD0 , inpin "data_out(13)_dup_117" DX , inpin "data_out(13)_dup_117" SR , pip CLBLL_X2Y49 SITE_BYP_B6 -> M_DX , pip CLBLL_X2Y49 SITE_CTRL_B3 -> M_SR , pip INT_X2Y49 BYP6 -> BYP_B6 , <<<<<<< pip INT_X2Y49 CTRL3 -> CTRL_B3 , pip INT_X2Y49 FAN5 -> FAN_BOUNCE5 , pip INT_X2Y49 FAN6 -> FAN_BOUNCE6 , pip INT_X2Y49 FAN_BOUNCE5 -> BYP6 , pip INT_X2Y49 FAN_BOUNCE6 -> CTRL3 , pip INT_X2Y49 GND_WIRE -> FAN5 , pip INT_X2Y49 GND_WIRE -> FAN6 , ; CLB 3 CLB 2 CLB 1 CLB 6 CLB 5 CLB 4 CLB 9 CLB 8 CLB 7 We know exactly where we failed Routing seems to be the problem most of the time: we can moderate such issues Analysis is performed by Prof. Violante & Prof. Sterpone (Politecnico Torino, Italy) ETD Meeting - CERN - Nov. 17th, 2011

ETD Meeting - CERN - Nov. 17th, 2011 From the vendor: V6 vs.V5 ETD Meeting - CERN - Nov. 17th, 2011

ETD Meeting - CERN - Nov. 17th, 2011 From the vendor: V7 Starting with V7, it should/could be possibile to correct-on-the-fly a single SEU. Waiting for more info from Xilinx about that... Trend is clear: SEU will be handled in the future more and more carefully by specific on-chip logic ETD Meeting - CERN - Nov. 17th, 2011

ETD Meeting - CERN - Nov. 17th, 2011 What BELLEII is doing ? Help yourself ... ETD Meeting - CERN - Nov. 17th, 2011

ETD Meeting - CERN - Nov. 17th, 2011 Benchmark design We have it Fixed latency Low phase noise on recovered clock Tested, presented many times, published ETD Meeting - CERN - Nov. 17th, 2011

Background simulation Vital Protons? Neutrons? E spectra? More info needed to drive the tests ETD Meeting - CERN - Nov. 17th, 2011

ETD Meeting - CERN - Nov. 17th, 2011 What if ... ... FPGAs do not fulfill our expectations on-detector Keep sending both clock and data Use COTS SerDes (like National, GOL for read-out only, ...) Design a Rad-Hard SerDes (see next slide) If (really) unfeasible, then: Send clock via specific fibers (see later) Design low speed SerDes (500Mbit/s) with ACTEL-like devices ETD Meeting - CERN - Nov. 17th, 2011

Designing our own SerDes We are in touch with IMEC (B) to explore the feasibility of a custom made SerDes tailored to our needs Nov.22 we will talk to R&D Managers from IMEC for details About costs: 500k€, about time: 1 year Too expensive? Not really... EU programs, ESA interested, ETD Meeting - CERN - Nov. 17th, 2011

ETD Meeting - CERN - Nov. 17th, 2011 Sending the clock We do not like that... ... but we know how to do it and we have already did a few tests (just in case ...) Jitt = 25 ps RMS Jitt = 9 ps RMS CERN TTC – 40MHz Clock driven by off–dector FPGA, to be send across fiber - 40MHz ETD Meeting - CERN - Nov. 17th, 2011

Making the system working TDR should define also important guidelines: In our opinion, ALL links (FCTS, Read-out) should be easy upgradable, BOTH ends (TX,RX) Then, they should be always hosted on mezzanine cards, BOTH ends (TX,RX) ETD Meeting - CERN - Nov. 17th, 2011

Links should talk each other Tower of Babel - Bruegel Like the ride from Eden to Babel ? In our view, one team does one link front-to-back, see ‘Facts and Opinions ...’ presentation at Elba Crucial issue, to be discussed in the view of TDR The Garden of Eden - Bosch ETD Meeting - CERN - Nov. 17th, 2011

ETD Meeting - CERN - Nov. 17th, 2011 Manpower ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ETD Meeting - CERN - Nov. 17th, 2011

ETD Meeting - CERN - Nov. 17th, 2011 Discussion …. …. Is now open …. ETD Meeting - CERN - Nov. 17th, 2011