Serial link Loss-of-Lock impact on trigger distribution dead time 24/01/2018 Serial link Loss-of-Lock impact on trigger distribution dead time Sergio Cavaliere Department of Physics, University of Napoli “Federico II”, Italy and INFN Sezione di Napoli, Italy e-mail: sergio.cavaliere@na.infn.it In this talk I will present some results from a preliminary study of Forward Correcting Codes for the SuperB serial links, simulations results and tools built for the purpose. It’s actually an ongoing work which will require soon a closer integration to the architecture which are being studied for the actual link. XIV SuperB Workshop – LNF Frascati - Sept, 2010

XIV SuperB Workshop – LNF Frascati - Sept, 2010 Introduction Recall the error sources due to SERDES Serializer-Deserializer in a radiation hard environment Describe the problem under analysis, that of Loss Of Lock due to bit failures in specific bit positions Describe some features of the Poisson statistics of this source of errors or better missed data Describe the parameters analyzed in order to evaluate the impact of LOL events Describe the simulation performed and the histogram and charts useful in order to evaluate the impact of LOL events on the performance of the overall architecture Conclusion and future work XIV SuperB Workshop – LNF Frascati - Sept, 2010

SerDes in rad enviroments Rad hard Fixed latency under test Connections: From the FCTS to the detector From the detector to the subdetectors Layout by D. Breton LNF, Dec.09 FCTS link Tight latency requirements DS92lv18 SuperB Workshop - SLAC, Oct.09

Which bits may be protected against errors? not of concern today start stop only these bits may be protected bit 0 -- 17 20bit symbol: start, 18bit payload, stop Guaranteed transitions at the symbol boundary No encoding provided, pattern is transmitted as-it-is 61.2 MHz x 20 -> 1.2 Gbit/s, 1.1 Gbit/s user bandwidth, 1 UI = 817 ps SuperB Workshop - SLAC, Oct.09

Loss-of-lock: no recovery is possible Error injection Error flag RX recovered clock LOCK* Loss-of-Lock Lock achieved Each and every missing start/stop triggers a loss-of-lock Then, the RX automatically tries to lock on the stream Lock is (should be) guaranteed by SYNC patterns: BELLE reports a few problems with older components, no issues have been seen during our tests SuperB Workshop - SLAC, Oct.09

Loss-of-lock recovery sequence Transmission restarts ECS signals lock to the TX via Ethernet TX RX ECS ETHERNET LOL signalled to TX and later Resynch happened Synchronization command Red: Loss Of Lock handling Black: normal operation commands RX reveals LOL event ECS acknowledges LOL and signals it to the TX via Ethernet which in turn issues a re-synch command RX starts relocking T2 T1a T1b T3 XIV SuperB Workshop – LNF Frascati - Sept, 2010

XIV SuperB Workshop – LNF Frascati - Sept, 2010 24/01/2018 Involved times T1 it is the time needed to traverse the message trough Ethernet. It depends on the performance of the Ethernet, the communication procedure, etc. We will take into consideration the following figures T1 = 5ms 20ms 50ms 75ms 100ms +/-10% T2 is the the time needed to the receiver to inform the ECS that lock was lost 0.1 ms -0.5 ms -1.0 ms T3 is the recovery time in the RX . Uniform distribution between 500ns e 1000ns XIV SuperB Workshop – LNF Frascati - Sept, 2010 7

Loss Of Lock: statistics of the recovery duration 24/01/2018 Loss Of Lock: statistics of the recovery duration T1 = 5ms 20ms 50ms 75ms 100ms +/-10% Ethernet +ECS T2 time to signal lock lost to the ECS 0.1 ms -0.5 ms -1.0 ms T3 lock time in RX. Uniform pdf 500ns e 1000ns The three times add up to a pdf resulting from the convolution of the three pdfs Since T3 has uniform pdf 500ns to 1000ns it may be neglected Overall pdf is essentially determined by the sum of T1 and T2, both normal. It will be normal showing parameters σ1232=σ12+σ22 e µ123= µ1 + µ2 XIV SuperB Workshop – LNF Frascati - Sept, 2010 8

Loss Of Lock: pdf of the recovery duration 24/01/2018 Loss Of Lock: pdf of the recovery duration (s) T1 = 5ms 20ms 50ms 75ms 100ms +/-10% T2 = 0.1 ms -0.5 ms -1.0 ms T3 = 500ns ---- 1000ns XIV SuperB Workshop – LNF Frascati - Sept, 2010 9

Loss Of Lock events: Poisson statistics 24/01/2018 Loss Of Lock events: Poisson statistics Loss of Lock has the usual statistics giving the number of events which take place in a radiation hard environment one after the other and indipendently each other the average number of events in unit time is constant, equal to λ. λ is the average number of events in unit time (frequency or rate) μ =1/ λ is the average time distance from one event to the next λ is the average number of events in unit time (frequency or rate) μ =1/ λ is the average time distance from one event to the next XIV SuperB Workshop – LNF Frascati - Sept, 2010 10

Loss Of Lock events: diffferences from thePoisson statistics 24/01/2018 Loss Of Lock events: diffferences from thePoisson statistics After a Loss Of Lock events takes place a recovery procedure is needed before the link is active again and a new LOL may happen. Time distance between one event and the next cannot be less than recovery time. The statistics is slightly modified in the origin of the times but Recovery time max is 100ms Average time distance ranges from 1 hour 10 days The statistics is practically unaffected XIV SuperB Workshop – LNF Frascati - Sept, 2010 11

XIV SuperB Workshop – LNF Frascati - Sept, 2010 24/01/2018 Loss Of Lock impact on average for each single link Foreseen range [1h 2h 4h 12h 1day 5days 10days] From the test beam we know the average no of LOL events for a link Because of n LOL events in unit time a link will be off for n*LOL_duration seconds We may define the ratio: 75 50 20 10 5 ms 1 LOL event each 1/ day 10ms LOL recovery duration toff/ton = 4E-4% Average time distance for LOL events (s) BUT What happens when we have 500-1000 links working at the same time in response to a flow of triggers in the 100 - 150kHz range? This requires thorough simulations. XIV SuperB Workshop – LNF Frascati - Sept, 2010 12

The problem to be faced 1/2 24/01/2018 The problem to be faced 1/2 Here is a sketch in the case of 3 links only rad events and LOL recovery intervals rad events and LOL recovery intervals rad events and LOL recovery intervals link 1 link 2 green: LOL events affecting 2 links Blue: LOL events affecting 3 links link 3 XIV SuperB Workshop – LNF Frascati - Sept, 2010 13

The problem to be faced 2/2 24/01/2018 The problem to be faced 2/2 LOL event 100kHz Trigger stream LOL recovery interval If a channel has its links impaired in some time instant it will not deliver the command to acquire data for that trigger If many links are likely to be impaired at the same time, data quality will be strongly affected because data from many channels will be missing We would like to know: How many triggers are canceled for too many links thus causing significant loss of information for those triggers? What is the statististics of the number of links impaired (because of a LOL event) for the same trigger? These question may be answered by means of long simulations, as follows XIV SuperB Workshop – LNF Frascati - Sept, 2010 14

A single situation (intermediate case) 24/01/2018 A single situation (intermediate case) LOL recover time 20ms Trigger frequency =150 kHz Average time interval between two consecutive LOL events 1day Simulation: 1 year of the apparatus Each trigger is sent through 1000 links to 1000 channels So it is important to undertstand the statistics of LOL in the different links As a result of simulations in this case the percentage of triggers with 1 link impaired and then data missing from 1 channel is 0.02% of the overall triggers 2 links impaired and then data missing from 2 channels is 2.7E-6% 3 links impaired and then data missing from 3 channels is 6E-8% 4 links impaired and then data missing from 3 channels is 2E-8% 5 links impaired and then data missing from 5 channels is 0.0% Likely: No more than 4 links for the same trigger are unable to transmit the trigger command. XIV SuperB Workshop – LNF Frascati - Sept, 2010 15

Histogram for the intermediate case on a log scale 24/01/2018 Histogram for the intermediate case on a log scale LOL recover time 20ms Trigger frequency =150 kHz Average time interval between two consecutive LOL events 1day Simulated time of the apparatus 1year Trigger with impaired links 1 link 0.02% 2 links 3E-6% 3 links 6E-8% 4 links 2E-8% 5 links 0% Trigger frequency =150 kHz 6E-8% 2.7E-6% 2.0E-8% 0.023% 0.0% LOG SCALE No. of links impaired 0.0% XIV SuperB Workshop – LNF Frascati - Sept, 2010 16

XIV SuperB Workshop – LNF Frascati - Sept, 2010 24/01/2018 Intermediate case: LOL recovery lasts 20ms LOL interval 1h 2h 4h 12h 1day 5days 10days 0.55% 0.001% 3E-6% 5E-7 0.55 0.02% 6E-8% 2E-8% XIV SuperB Workshop – LNF Frascati - Sept, 2010 17

XIV SuperB Workshop – LNF Frascati - Sept, 2010 24/01/2018 Intermediate case: this equivalent chart allows easily reading histogram values XIV SuperB Workshop – LNF Frascati - Sept, 2010 18

Worst case : longest LOL recovery time and shortest LOL interval 24/01/2018 Worst case : longest LOL recovery time and shortest LOL interval LOL recover time 100ms Trigger frequency =150 kHz Average time interval between two consecutive LOL events 1hour Simulated time of the apparatus: 1year 2.7% Trigger with impaired links 1 link 2.7% 2 links 0.04% 3 links 4E-4% 4 links 4E-6% 5 links 0% 0.04% 4E-4% 4°-6% XIV SuperB Workshop – LNF Frascati - Sept, 2010 19

Worst case: LOL recovery lasts 100ms 24/01/2018 Worst case: LOL recovery lasts 100ms LOL average time distances: 1h 2h 4h 12h 1day 5days 10days 0% 0.009% 4E-5% 1e-6% 1.4% 6e-7% 03e-8% 9e-9 0.01% 6e-5% 3e-7% 9e-8% 0.1% 0.04% 4E-4% 4E-6% 2.7% XIV SuperB Workshop – LNF Frascati - Sept, 2010 20

XIV SuperB Workshop – LNF Frascati - Sept, 2010 24/01/2018 Worst case: this equivalent chart allows easily reading histogram values XIV SuperB Workshop – LNF Frascati - Sept, 2010 21

Best cases: LOL ricovery time is 5ms 24/01/2018 Best cases: LOL ricovery time is 5ms Data from the green histogram: Average rad interval 12 hours simulated for 1year Trigger with missing links 1 link 0.012% triggers 2 links 8E-7% in 3 links 3E-8% in 4 links 9°-9% XIV SuperB Workshop – LNF Frascati - Sept, 2010 22

Best case: LOL ricovery time is 5ms 24/01/2018 Best case: LOL ricovery time is 5ms LOL recover time 5ms Average rad interval 1day simulated for 1year Trigger with missing links 1 link 0.005799% triggers 2 link 1.161e-007% in 3 link 1.513e-008 in 4 link 4.679e-009% Trigger frequency =150 kHz XIV SuperB Workshop – LNF Frascati - Sept, 2010 23

Conclusion and perspectives 24/01/2018 Conclusion and perspectives Statistical analysis of the impact of Loss Of Lock on data quality has been set up Many alternatives has been analyzed in order to understand the impact of the problem on design parameters To be done: Understand the impact on fata quality and front end synchronization More thorough error analysis is on the way and some results have been obtained Impact on the error performance of the choice between uncoded, Hamming coded or ReedSolomon coded data transmission in terms of coded error probability. XIV SuperB Workshop – LNF Frascati - Sept, 2010