By: Bas van der Heijden & Martin van Beuzekom Timepix3 Readout By: Bas van der Heijden & Martin van Beuzekom 3/20/2013 Nikhef

MPX3 and TPX3 readout -> SPIDR Speedy PIxel Detector Readout Development on a Xilinx evaluation board design a custom (small form factor) FPGA board at a later time Aim to read out 1 Timepix3 at full (80 Mhits/s) speed, or multiple Timepix3 chips at lower speed (less links per TPX3) Readout bandwidth at least 5 Gbits/s -> use 10 Gigabit ethernet currently using 1 Gigabit ethernet 10 Gbe VHDL code development started MPX3.1 and RX readout up and running (low DAQ speed) Re-use ‘back-end’ code for TPX3 readout 3/20/2013 Nikhef

Xilinx ML605 development board Medipix3 chipboard Xilinx ML605 development board 3/20/2013 Nikhef

By: Bas van der Heijden Nikhef Timepix3 chipboard HV for sensor/ grid test fixture for diff. probe No routing TPX3 lemo00 Power supply 1.5V Test fixture 11x 150mm 69mm FMC connector 3/20/2013 By: Bas van der Heijden Nikhef

By: Bas van der Heijden Nikhef Timepix3 chipboard FMC (high pincount 8gbt’s) Mux for selecting between gbt and regular IO pins on fpga (PI2PCIE2412ZHE) ADC & DAC for timepix I2C Lemo’s (trig/clk/busy/?) VDD(1.5V) VDDA(1.5V) VDDPLL(1.5) (from one DC/DC? CERN sm01c) VDDA(3.3 efuse from FMC) U&I monitor (ina219) CERN rad-hard powersupply’s? Flash for storing pixel mask (mp25p32 32Mb) Simple bias supply for Si sensor? (max668) (12-100V) Extra pad and connector/pad for HV bias (1000V) (thick sensor/grid+cathode) Probing fixtures for 8x data_out/data_in/clk_in/enable_in No routing behind chip (allows removing/thinning of PCB for beamtest) Chip cooling? 3/20/2013 By: Bas van der Heijden Nikhef

Fast data transmission path 64MHz (recovered clock) 125MHz 156MHz MAX data rate is 5.12Gbit (80MHz*64bit@8 lanes) TPX RST TPX CLK 8x TimeStamp 14+20bits 2^34*25ns ≈ 7min Unique timestamps@40MHz Pixel data to Ethernet packet State Machine timestamp 20b FIFO 68b Depth=4097 8x36Kb EBR Data 64b Data 64b Data 68b Data 68b Data 64b TX BUF-FER 10GB MAC 10GB PCS SerDes 640Mbit CLK 64MHz 48b + 20b timestamp Pixel word collector CLK 10MHz FIFO 68b (depth=16) WORD MUX Data 68b sop Port 1 sop Data 8b Data 68b Empty Control word DEMUX Empty Empty eop eop Empty RdReq RdReq RdReq IP MUX K-char WRen RdReq XGMII Remove control header Send packet on timeout or max size 10G Base X Almost full Almost full RX BUF-FER eop sop Control data FIFO 64b Throttle control CPU Port n Pixel word 68b Crtl 4b Address 16b ToA 14+4b ToT 10b Coarse Timestamp20b 3/20/2013 Nikhef

Fast data path summary Add 20 bit timestamp to 48 bit pixel packets from TPX3 data of up to 8 serial links merged into one fifo (round robin) strip off 4 bit ctrl header in ethernet packet builder use 4 bit ctrl header to route data (to GBE / CPU) GBE packet transmitted when either max. length reached or when time-out (configurable) max. collection time given by 14+20 timestamp @40 MHz -> 7 minutes 3/20/2013 Nikhef

control data path 125MHz 156MHz 40/80MHz IP MUX RX Statemachine Packet decoder Enable Data out Front end Data Port 1 ClkIn MUX 10GB MAC TX BUF-FER 10GB PCS 64b IP MUX XGMII 10G Base X Throttle control Veto 64b RX BUF-FER T0 sync Control signals CPU Shutter CPU Port TPulse ext_T0 & shutter 3/20/2013 Nikhef

control path summary Configuration / control via soft-core CPU (Leon) Configuration directly from GBE also possible via Rx statemachine Rx statemachine also for throttling (pause frames) The Soft-CPU + firmware take care of the HW specific details Testpulse generator Shutter control 3/20/2013 Nikhef

Throttle structure FPGA PC Pause frame DAQ busy packet Timepix3 FIFO TX State-Machine MAC+PCS Network card DAQ application 10G Base X Shutter FIFO almost full Throttle control Pause frame FrontEnd disable Other detectors DAQ busy packet (continue after …) FE disable from other detector 3/20/2013 Nikhef

Hardware status + plan for next 3 months 2 hardware engineers full time + support for PCB design MPX3+RX readout working 1 lane to MPX3, 1 Gbe to PC not yet all modes are implemented/tested further VHDL development is temporarily stopped; focus on TPX3 specific parts Aim to have ready before TPX3 returns from fab: Chipboard 10 GBE (fallback to 1 GBE possible) VHDL statemachine, interface to TPX3 Pixel packet receiver and packet builder for 10 GBE Later this year: features and testing max. data throughput custom FPGA board tileable chipboard for MPX3 and TPX3 (2 versions) 3/20/2013 Nikhef

Software status and plans Simultaneous code development for soft-CPU in FPGA and driver library at PC side Library (“API”) for MPX3+RX functions almost complete, being tested/debugged at the moment Low level TPX3 calls will be added the coming months Testprograms (“scripts”) are simply a collection of calls from the library Calls like: “SetDACs”, “openShutter”, “ReadFrame” etc. written in C/C++, compile before running (MS Visual Studio Express 2010, free version) Pixel data written to disk and /or on primitive display decoder for offline display (Root) will be provided Additional layer of code will be added to adapt to Pixelman API High speed DAQ program will follow later this year 3/20/2013 Nikhef

Software tools (V)HDL graphical entry tool: Ease from HDLworks Simulation: Questasim latest version Synthesis: Xilinx ISE version 14.x Use of SVN Software development: MS Visual Studio Express 2010 free version 3/20/2013 Nikhef

We would like to have/know: Final pinout / dimensions of TPX3 for chipboard Up to date manual (working document?) Simulation model of TPX3 Other S-LVDS issue? converter needed? 3/20/2013 Nikhef