Xiangming Sun1PXL Sensor and RDO review – 06/23/2010 STAR XIANGMING SUN LAWRENCE BERKELEY NATIONAL LAB Firmware and Software Architecture for PIXEL L. Greiner, H. Matis T. Stezelberger M. Szelezniak C. Vu H. Wieman …

Xiangming Sun2PXL Sensor and RDO review – 06/23/2010 STAR Outline Requirements for event readout mode Event readout mode Other firmware functions Firmware and readout software structure Summary

Xiangming Sun3PXL Sensor and RDO review – 06/23/2010 STAR Chip set sensoroutputfrequency Mimostar2analog50MHz Mimostar3analog50MHz Phase1/24 LVDS pairs and analog 160MHz Ultimate2 LVDS pairs160MHz

Xiangming Sun4PXL Sensor and RDO review – 06/23/2010 STAR  Individual sensor test - A single sensor is connected.  Probe test - The sensors are put on probe station and the power supply is controlled.  Infrastructure Test Board (ITB) test - 10 sensors are connected in parallel, long cables are used.  Beam test - 4 sensers are built into a telescope. The trigger is included.  Phase-2 sector test - More than 1 sector is connected. The trigger is included.  Final version - 40 sensors are connected. Long cables are used. The trigger is included. Firmware structure is designed to handle all these tests. The set of firmware modules for the various tests will be very similar with only minor task dependant differences. Test set

Xiangming Sun5PXL Sensor and RDO review – 06/23/2010 STAR Rrequirement for event readout mode 40 sensors per readout board( Each sensor has 2 LVDS pairs. 80 LVDS pairs in 160MHz). Sensor digitizes signal and records hit address in on-chip memory, then transfer hit address serially. Trigger is STAR DAQ1000(about 1KHz). DAQ DDL assumed throughput is 80MB/s on average. (DDL link rated for 200MB/sec). requirement : mass terminition board mother board xilinx V5 PRO SIU/DDL DAQ PC

Xiangming Sun6PXL Sensor and RDO review – 06/23/2010 STAR Data is buffered in FPGA STAR and DAQ policy: 1 trigger per event DDL peak speed(2Gb/s) is lower than data generation peak speed (12.8Gb/s). TPC can buffer 8 triggers when DDL is not available. PIXEL should buffer ≥8 triggers frame1 frame2 frame3 Trigger timeEvent data

Xiangming Sun7PXL Sensor and RDO review – 06/23/2010 STAR Event format 7 For sector header ender Data block For ladder1 Data block For ladder2 Data block For ladder3 Data block For ladder4 Header token Trigger info system status temperature Firmware version Hardware ID ……. 10 words reserved Block startData length … Hit address end token sensor ID is encoded in four unused bits

Xiangming Sun8PXL Sensor and RDO review – 06/23/2010 STAR Ladder#1Ladder#2Ladder#3Ladder#4 Event Builder #1 (head token added) Event Builder #2 Event Builder #3 Event Builder #4 (end token added) Event fifo SIU Event mode Readout structure for one sector

Xiangming Sun9PXL Sensor and RDO review – 06/23/2010 STAR Event Builder Serial to parallel conv … 10 sensors data memory Trigger time memory start frame memory Builder state machine To event fifo circular buffer Sensor 1 output1 output2

Xiangming Sun10PXL Sensor and RDO review – 06/23/2010 STAR Serial to parallel converter Serial to parallel conv … 10 sensors data memory Serial-to-parallel converter 1, convert 160MHz 1 bit data to 10MHz 16bit parallel data. 2, add sensor ID into the data. Data memory scans Serial-to-parallel converter one by one every 10 ns (100MHz). It starts from sensor 1 and returns to sensor 1 after reaching sensor 10. From sensor1 From sensor2 From sensor… From sensor10 From sensor… From sensor1 Sensor 1 output1 output2 From sensor3

Xiangming Sun11PXL Sensor and RDO review – 06/23/2010 STAR Circular buffers data memory Trigger time memory start frame memory frame counter when trigger comes index of start frame memory index of data memory for the first frame index of data memory for the second frame 64K words 512 words trigger token and buffer full frame1 frame2

Xiangming Sun12PXL Sensor and RDO review – 06/23/2010 STAR Data check out data memory Trigger time memory start frame memory frame1 frame2 get trigger time and start frame index get buffer index for 2 frames Check out region where hit address bigger than trigger time for frame1 and smaller for frame2 Release unused memory.

Xiangming Sun13PXL Sensor and RDO review – 06/23/2010 STAR Buffering capability 19K words/trigger (684kb).( buffer length in chip is assumed to be 500 words and 150 word for inner and outer ladder from simulation ). Current FPGA has 3Mb for event mode. XC5VLX330 has 8Mb. 256 continous triggers can be stored (data memory may be full already) bottleneck is DDL. 39MB/s for trigger DAQ1000 deviceTotal roomRoom usedusage sensor2000word500 / 15025% / 7.5% FPGA data8Mb684kb*217% FPGA trigger512 word2word*21% DDL80MB/s39MB/s48% In normal condition, memory and DDL usage 10 triggers can be buffered when the DDL link is busy.

Xiangming Sun14PXL Sensor and RDO review – 06/23/2010 STAR Other firmware functions JTAG function (all sensors) Analog full frame readout (Mimostar3 and Phase1) Digital full frame readout (Phase1 and Ultimate) I 2 C device control (Phase1 and Ultimate) System monitoring (Phase1 and Ultimate)

Xiangming Sun15PXL Sensor and RDO review – 06/23/2010 STAR Firmware structure 15 System control can access every module DDL/USBPC sensor

Xiangming Sun16PXL Sensor and RDO review – 06/23/2010 STAR JTAG function (all sensors) JTAG is used for initializing the sensor. JTAG is done by the FPGA in Virtex5 development board. We reserve 5 FPGA pins as JTAG pins which connect to sensor’s JTAG pin serially. The JTAG data is sent from PC to FPGA SRAM. The JTAG module in FPGA will read the data from FPGA SRAM and transmit the data to sensor by JTAG pins. sensor

Xiangming Sun17PXL Sensor and RDO review – 06/23/2010 STAR Analog full frame readout analog signal is digitized in ADC ADS5272 sensor ADS MHz, IOdelay is used data is buffered in SRAM

Xiangming Sun18PXL Sensor and RDO review – 06/23/2010 STAR IO delay for digital data alignment 1, set sensor to pattern mode, a single pulse is sent from sensor for each row. 2, scan IO delay setting from 0 to 63. readout data in event mode. 3, search for the pulse in data taken. plot the relation between pulse address and IO delay setting. 4, find the transition region in above plot. set IO delay by shifting a half clock. Digital signals pass 7 meters before arriving FPGA. How the digital signals from all sensors are aligned? Scan steps 063 N N+1 half clock pulse 1 clock cycle Transition region Iodelay set

Xiangming Sun19PXL Sensor and RDO review – 06/23/2010 STAR Digital full frame readout sensor

Xiangming Sun20PXL Sensor and RDO review – 06/23/2010 STAR I 2 C voltage control and measurement

Xiangming Sun21PXL Sensor and RDO review – 06/23/2010 STAR System monitoring We have a separate data channel to monitor the system in physics run. This channel includes a information builder module in FPGA, USB data channel and a monitor PC. If something wrong happens, we can easily see it from monitor PC. sensor

Xiangming Sun22PXL Sensor and RDO review – 06/23/2010 STAR Event mode

Xiangming Sun23PXL Sensor and RDO review – 06/23/2010 STAR Readout software structure command: generate commands in windows and linux usb.exe :upload commands to FEE in windows download_data_block_to_FEE: upload commands to FEE in windows getdata:get frame data in windows and linux rorc_receive: get event data in linux IO delay package: tune IOdelay for each digital channel I2C package: control I2C device Design Goal: portable between Linux and Windows every test can be scripted

Xiangming Sun24PXL Sensor and RDO review – 06/23/2010 STAR System Control command Hex file usb.exe download_data_block_to_FEE 0x0402fffd 0x1d82ff3f 0x1502ffcf 0x2642ffff 0x2642fdff 0x2202feff 0x0c03fff0 0x1547ffff 0x1547ffdf 0x0cc7ffff …………. Linux PC Linux PC rorc_receive getdata Windows PC Windows PC

Xiangming Sun25PXL Sensor and RDO review – 06/23/2010 STAR Summary The design meets the requirements as stated. The full set of expected operation modes are addressed.

Xiangming Sun26PXL Sensor and RDO review – 06/23/2010 STAR backup

Xiangming Sun27PXL Sensor and RDO review – 06/23/2010 STAR I 2 C voltage control