Understanding Readout Issues with the Pilot System October Fpix Upgrade Meeting Bora Akgün Will Johns Karl Ecklund Helmut Steininger Jordan Tucker Nathan Mirman Satoshi Hasegawa Sergey Los Jan Troska 3 Nov 2014Pilot Readout Issues - w.johns1

Nature of the problem FED looks for good phase choices in the 400 MHz data stream – FED has a fixed reference clock to the LHC – Uses idle pattern to find good phases 8 choices in 2.5 ns Look for adjacent good choices, try to choose center Phase choice becomes unstable with triggers – Hard to do calibrations Where is the jitter in the pilot system? 3 Nov 2014Pilot Readout Issues - w.johns2

A quick scan of jitter 3 Nov 2014Pilot Readout Issues - w.johns3 Looking at readout dependability – Some interesting observations on jitter of 400 MHz – So we expect to lose about 1 ns of 2.5 ns, and we see an “eye” of about 1 ns (this was before triggers) – We also found that we could get less system jitter with a TTCex replacing the TTCci mFEC TTCrx TTcci TTCrx QPLL Detector Piggy 340 ps 360 ps 340 ps 240 ps 450 ps 600 ps

Looking at FED Phases Jitter Study 3 Nov 2014Pilot Readout Issues - w.johns4 Looking at 400 Mbd signal triggering on 400 MHz clock on the scope with idle pattern : TTCex reduces the jitter TTCci TTCex

Looking at FED Phases Jitter Study 3 Nov 2014Pilot Readout Issues - w.johns5 Looking at 400 Mbd signal triggering on 400 MHz clock on the scope with 100KHz triggers: With triggers, there is little improvement TTCci TTCex

We did get some calibrations done We asked Helmut for a firmware where we can fix a phase rather than rely on the automatic phase finding system – Still get lots of readout errors (BER Xe-4 ish) 3 Nov 2014Pilot Readout Issues - w.johns6 This is a very nice pixel alive. We can’t get the same nice results every time we do it, and not all ROCs are as nice, but that could be tuning.

Looking at FED Phases Jitter Study 3 Nov 2014Pilot Readout Issues - w.johns7 We have measured the jitter on the clock to the module and the idle signal while triggering with TTCci (it is easier to do this when the detector is open) (All jitter is quasi peak-to-peak) The idle pattern jitter from the module, measured at the FED: – when idle the jitter is ~730ps – with 100KHz triggers the jitter is ~1.2 ns, We also measured jitter on delay25 clock in and clock out: A-23(CLK+2) and A-22(CLK-2) and also A-21(CLK+1) and A- 21(CLK-1) for both of them – when idle the jitter is ~630ps – with 100KHz the jitter is ~1.15ns – At this point we concluded the extra jitter is coming from the DOH and/or TPLL

Looking at FED Phases Jitter Study 3 Nov 2014Pilot Readout Issues - w.johns8 We have measured the jitter with the spare port card – DOH output ~600 ps w/o triggers ~700 ps w/ triggers – Delay25 input (TPLL output) ~820 ps w/o triggers ~1.26 ns w/ triggers Jitter is shifting only in one direction with triggers Checked the same thing w/ the current detector port card – Similar results Ask PLT (Dean Hidas) to look at their TPLL, they see similar

Looking at FED Phases Jitter Study 3 Nov 2014Pilot Readout Issues - w.johns9 We tried playing with TPLL settings in “test mode” – Never got the clock back! We tried playing with 400 MHz and 160 MHz pll settings on the TBM – Looking for the source of 2.5ns shifts we saw in the idle pattern at lower (-9C) Blade temperature – We able to induce and kill shifts with 400 MHz pll settings – Jitter measurements were done with none of the 2.5 ns shifts Sergey Los tried changing the pattern of phases we send to the TPLL to get the trigger – Didn’t work so well

Looking at FED Phases Jitter Study 3 Nov 2014Pilot Readout Issues - w.johns10 Would bypassing closer to TPLL power pins help? – Probably not, saw this on all the hardware we tried Helmut says maybe we could get ~100ps back with FED tricks, but this is not really enough – His estimate is close to what we see when the QPLL cleans up the jitter on the FEC At this point we started to really sound the alarm and got some help from Jan Troska – He confirmed what we were seeing – He strongly urged us to fix this on the detector – He said ECAL has seen this and fixed it

Nice pictures from Jan Best case: trigger right from TTCci 3 Nov 2014Pilot Readout Issues - w.johns11 Notice jitter seems biased in one direction

3 Nov 2014Pilot Readout Issues - w.johns12 Data period is not a constant Jitter is biased The QPLL gets rid of the other freqs I believe the blue should be 0 for a good eye

3 Nov 2014Pilot Readout Issues - w.johns13 The challenge of phase finding Trigger on the data signal, and look at the delayed data signal. -Will be tough to find a phase if the data is not stable -We did try several methods No Delay 100 ns 800 ns 200 ns 1 ms This suggests we must sample constantly or fix the jitter. Tough to sample constantly. (Data changes)

What about CDR? This has been suggested by several people now. – Resources are not suitable in the Altera chips Helmut has been using – Better possibilities in XILINX FPGA Helmut will try, but is skeptical due to the idle pattern – May not be enough info fo a CDR – CDR chip? 400 MHz chip is obsolete (and big) Can try for fun 3 Nov 2014Pilot Readout Issues - w.johns14

What did ECAL do? They put a QPLL after the TPLL on the clock – The QPLL has its own oscillators and can clean jitter – Jan says ECAL cleans 1.5 ns jitter to several 100 ps CERN is building us a little board we can try inserting in the port card – About 2 weeks eta – They will look at SEU rates, Rad hardness FNAL is pursuing a new port card – Could take a few weeks 3 Nov 2014Pilot Readout Issues - w.johns15

Another possibility 3 Nov 2014Pilot Readout Issues - w.johns16 mFEC Delay 25 TPLL QPLL POH mFEC Delay 25 TPLL POH mFEC Delay 25 TPLL POH TPLL Can try this if middle choice has too much jitter. Saves us the trouble of running a separate clock. NOW ECAL NEW

Other things to try Can probably clean up the mFEC jitter 100ns Perhaps clean up the piggy by 100 ns w/QPLL Have tried using the mFEC return clock to help decode – Jitter compared to data is much smaller (~600 ps) – Too much jitter to replace FPGA clock Works better than normal clock if no triggers though… – Maybe we can use the return clock to find dynamic updating phase Need to turn into opto signal for FED or find FPGA input pins Constant signal that phase shifts with the data 3 Nov 2014Pilot Readout Issues - w.johns17

Outlook We must address this issue It must be present in the BPiX readout We have several approaches – There is still time for a hardware intervention on the pilot – We need a long term solution for the phase 1 detector – We will pursue all the approaches we can until installation 3 Nov 2014Pilot Readout Issues - w.johns18