1. Time Division Multiplexing School of Physics and Astronomy Department of Particle Physics Elissavet Papadima 29/5/2014

2. System for Testing

3. What do we need the TDM for? The existing system uses a Quad Module with four FE-I4Bs The I/Os of each FE-I4 are connected to a RJ45 for transmission. With the multiplexing of the four streams to one stream, the data rate is increased by four and the cost of the cables for the transmission is reduced by four.

4. The Quad Module Each RJ45 port has the connections that are shown in the schematic. Schematic: http://icwiki.physik.uni-bonn.de/twiki/bin/view/Systems/UsbPix#Hardware

5. System for testing (I) In order to test the connectivity through the GLIB board the first test includes both a multiplexing and a demultiplexing core in Virtex 6 FPGA.

6. System for testing (II) The second test includes the multiplexing core in Virtex6 FPGA and the demultiplexing core in Virtex4 FPGA.

7. Comments on the second test: Configuration of the Quad Module with 40MHz frequency and the HSIO with 160MHz. The multiplexed stream is at 160MHz and it is going to be received as an 160MHz stream in the first RJ45 of the HSIO. The packets are not from the same FE-I4 Chip so the result is not going to be the same as if a single stream at 160MHz was sent if the demultiplexing core is not included in Virtex4 FPGA. A Chipscope core has been inserted to display the streams and check the functionality of the system.

8. Do we need the demux in the Quad Module? Three kind of commands: “Trigger”, “Fast” and “Slow” For the slow commands we don’t need to use demux because the command will be decoded but ignored if the chip ID (field 4 of the command) doesn’t match the geographical address of the FE. For the rest of the commands (Trigger, Bunch Counter Reset, Event Counter Reset and Calibration Pulse) we have to use demux. For this implementation though we can assume that these commands are going to be applied on all the four FE-I4 chips simultaneously and design the demux later.

9. The clock information In this implementation the external clock information is not used in the design The clock signals pass through the FPGA from the transmitter to the receiver as if the device was a bridge (transparent). The design uses a slow clock (40MHz) and a fast clock (160MHz). In the future the data speed will be increased up to 640MHz. The design has its own system clock and a pll to generate the second clock and uses registers in order not to lose data. In the PCB that is going to replace the FPGA the clock is going to be according to this design. In the new Quad Module the clock could be according to this design, or if a new version of FE-I4  FE-I5 is used that is going to have an output port for the clock, this clock information could be used as the system clock. The point of the TDM design is to reduce the cables needed for the transmission so apart from multiplexing the four streams to one, a new method of transmission could also be used in order to insert the information of the clock in the height or the width of the command pulses. Challenging task because all the command, data and clock signals in the transmission are differential and low-voltage.

10. Other issues: The start time of the streams defines whether the multiplexer will start from 00, 01, 10 or 11 state. Solutions: Trigger signal from the FE-I4 to the multiplexer Initial packet in the transmission with the information of the counter (chip ID equivalent) Add logic in the Quad Module and the HSIO Decoder to include the information of the chip ID

12. FPGA - Gigabit Link Interface Board (GLIB) According to the latest manual of GLIB: https://svnweb.cern.ch/cern/wsvn/ph- ese/be/amc_glib/trunk/glib_v3/doc/glib_v3_user_manual.pdf

13. Powering and connection with the PC According to the latest manual of GLIB: https://svnweb.cern.ch/cern/wsvn/ph- ese/be/amc_glib/trunk/glib_v3/doc/glib_v3_user_manual.pdf

14. How to start-up an ATX Power supply outside a computer According to the latest manual of GLIB: https://svnweb.cern.ch/cern/wsvn/ph- ese/be/amc_glib/trunk/glib_v3/doc/glib_v3_user_manual.pdf

15. Powering the GLIB and connection with the PC According to the latest manual of GLIB: https://svnweb.cern.ch/cern/wsvn/ph- ese/be/amc_glib/trunk/glib_v3/doc/glib_v3_user_manual.pdf The J12 JTAG connector is used for the FPGA configuration. ATX power supply Xilinx Platform Cable USB II

16. GLIB interface Card

17. The reset button and the I/Os According to the latest manual of GLIB: https://svnweb.cern.ch/cern/wsvn/ph- ese/be/amc_glib/trunk/glib_v3/doc/glib_v3_user_manual.pdf The reset button GLIB Interface Card Supports connection to both the FCM sockets.

19. Xilinx ISE Development Suite GLIB compatible with ISE version 14.5 Use the example project “glib_v3_basic” making changes in the user_logic_basic.vhd Download and install the files needed for the project to run according to page 24 of the GLIB manual: https://svnweb.cern.ch/cern/wsvn/ph- ese/be/amc_glib/trunk/glib_v3/doc/glib_v3_user_manual.pdf https://svnweb.cern.ch/cern/wsvn/ph- ese/be/amc_glib/trunk/glib_v3/doc/glib_v3_user_manual.pdf Under user_logic_basic.vhd insert the verilog design as a new component The project “glib_v3_basic”is in VHDL and the design is in Verilog – the mixed language synthesis is supported The Verilog design should be tested in simulation separately from the “glib_v3_basic” project with the according simulation testbench and in the implementation along with the “glib_v3_basic” project with the according implementation testbench

20. The test vectors for the testbench The test vectors for the testbench have been created in C in files memory_1.txt, memory_2.txt, memory_3.txt, memory_4.txt along with the expected results in memory_exp.txt The starting point of the streams defines which stream is multiplexed first so this is a parameter that should be taken into account in the creation of the memory_exp.txt

21. Simulation results

22. Data MUX

23. Virtex 6 FPGA

24. The output stream starts from stream_3 because the select signal of the multiplexer has the value 11 when the input streams are ready in the registers. The output data rate has Tclock = 1.5625ns  f = 640MHz In the real system there are delays that may affect the results.

25. Simulation results

26. Data DEMUX

28. The output stream starts from stream_2 because the select signal of the multiplexer has the value 11 when the input stream is ready in the REG_IN. Therefore, the first data of stream_o/_1 are not valid values 0. That is not going to affect the functionality because the processing of each stream is going to be done separately and not simultaneously (so there is no need of synchronization among the streams) and there is a specific bit sequence that signifies the start of transmission. The output data rate has Tclock =6.25  f = 160MHz In the real system there are delays that may affect the results.

29. Simulation results

33. Testbench

34. The input streams are stored in memory_0-memory_3. Each memory has 1000 data. The output stream is stored in memory_out and has size 5000 data. A pointer defines from which address the data in the memory are valid. The input streams start at 637ns (arbitrary selected time) so the start pointer is 347.The valid results are compared with the expected ones. If any error occurs, the signal error_detected is ‘1’.

35. Testbench Module on FPGA

36. Testbench Module top_level monitor tdmux_top

38. Goto Next slide

40. clock_monitor stream_out rst clk error_detected

41. The expected sequence is sel = 1, 2, 3, 0.

42. Belkeley LAB Johannes Agricola

43. Parallel Multi-Chip Readout with USBpix by Johannes Agricola 40MHz The only implementation so far must be with the FPGA It is not known how they have designed the circuit in the FPGA but maybe they have used buffers SiliconBlue iCE mobile FPGA

44. The Multiplexing Board of Berkeley LAB 40MHz single ended CMOS level multiplexer that could be run at less than 2V with less than 20 ns switching speed a buffer and inverter to generate a differential pair along with resistors to create the correct current levels for the LVDS receiver on USBpix first version second version has a lowest recommended power supply of 2.35V, but has switching speeds < 1ns only uses 1 RJ45 on the bottom instead of 2; data can be selected from the two identical circuits (one is a copy of the other)

45. Initial results with 2 single Chip Cards v2 (I) ANALOG SCAN WITHOUT MUX ANALOG SCAN WITH MUX

46. Initial results with 2 single Chip Cards v2 (II) DIGITAL SCAN WITHOUT MUX DIGITAL SCAN WITH MUX

47. Comments on the Multiplexing Board of Berkeley LAB The order of the Single Chip Cards are reversed between tests  so it is not a problem of the transmission This implementation uses 2 2x1 MUX circuits and the results are for one of this 2 circuits The synchronization must be done with the Clock signal. Both the analog and the digital tests results are not identical apart from the pixels that are disconnected. Could it be noise because of the MUX circuit?

48. Other

49. Serial Voltage Supply (n Modules) Issues: Manchester encoding in the output of each module so that the signal can be capacitively coupled Detect overcurrent shut down the erroneous module