Software Implementation of USB 3.0 Stack for Upgraded Data Link Interface on IBR-2 Reactor Spectrometers in FLNP S.M.Murashkevich, V.A.Drozdov Frank Laboratory of Neutron Physics Joint Institute for Nuclear Research, Dubna, Russia NEC’2017

Data acquisition systems for spectrometers of the IBR-2 reactor MPD board for data acquisition from point detectors Maximum number of detectors – 240 Time discretization frequency of an event appearance – 62.5 MHz (16 ns) Total maximal data flow rate – 8 Mevent/s Additional signals subject to time discretization – start of the reactor or a chopper, beginning and end of a registration time window during a reactor cycle, rising and falling fronts of 6 additional external signals (for instance, signals from a Fourier-chopper encoder), etc. Histogram memory (64 Mb) NEC’2017

Data acquisition systems for spectrometers of the IBR-2 reactor De-Li-DAQ-2D board for data acquisition from PSD 8-channel time-code converter (TDC-GPX), resolution 80 ps Histogram memory (HM), 1 Gbyte : - X-Y-TOF spectra , up to 512х512х1024 32-bit words The real count rate >=  106 events/s NEC’2017

Characteristic features of MPD and De-Li-DAQ-2D boards The blocks include FPGA where the firmware perform logical operations, selection and filtering of data events Data transmission between DAQ blocks and the PC is carried out through a serial optical fiber line and a specially developed adapter module FLINK with USB2.0 interface Optical transceivers and code converters (parallel code to serial and vice versa) located in the interface blocks of FLINK adapters and modules allow working at speeds up to 1.25 Gbit /s. The distinctive feature of DAQ blocks is to use a standardized fiber optic serial interface All functions and parameters of the modules are programmable Basic modes of data acquisition are: - histogram mode (on-line sorting and collection of spectra in HM) - list mode (“Raw” data) Build-in test generator (imitates the system operation without a detector input ) Standard NIM NEC’2017

Need for improving the Data Link Interface Current trends towards increasing the number of detector channels and the volume of recorded and accumulated information in real time in experiments on the spectrometers of the IBR-2 reactor require increasing the bandwidth and reliability of the communication channel. The main restriction on the speed of operation using the modules was an insufficient throughput provided with the USB2.0 interface. At the same time MPD and De-Li-DAQ-2D modules themselves have interface units allowing a higher speed. With the original version of the adapter, it was not possible to transparently transmit and receive the packets received and generated by the module. The communication interface between the modules and the adapter used a more reliable data transmission Protocol than the interface between the adapter and the PC. To completely exploit the blocks already installed on the spectrometer it was necessary to upgrade the communication adapter and the data transmission Protocol from the adaptor to the PC. NEC’2017

Data transmission between PC and DAQ board: old FLINK adaptor version FPGA1 (master for USB-controller) : - initiates internal registers of USB-controller (CY7C68001 chip, Cypress) - realizes the data interchange protocol via optical fiber and TLK1501 chip with DAQ board Transceivers TLK1501- convert parallel data into serial data stream and vice versa USB-controller has 4 end points: EP2, bulk, outFIFO, 512 bytes x 2, writing data to HM EP4, bulk, outFIFO, 14 bytes, command block determines a type of data interchange: - read/write to HM - read/write registers - write constant/erase HM - read status register EP6, bulk, inFIFO, 512 bytes x 2, reading data from HM, reading registers EP8, bulk, inFIFO, 512 bytes x 2, reading “raw” data NEC’2017

Data Transmission Protocols: old FLINK adaptor version Data Transmission Protocol: adapter < - > DAQ board Control and identification of commands received from the PC and data transmitted via USB channels from the DAQ board to the PC and vice versa are carried out directly in the FPGA chip of the adapter. Upon receiving a command or data packet from the PC, the adapter adds the corresponding header ID to the beginning of the block of the command or data packet, and CRC-code at the end, and in this form transmits data to the DAQ board. To control the execution of commands, a response packet of acknowledgment (NACK, ACK) and board state (BUSY, READY, ERR) is used. Data Transmission Protocol: adapter < - > PC When data is transferred from the adapter to the PC from the full package, only data without ID and CRC-code is written to the USB channel. Thus, the software could not control receiving of commands and data by modules, and also control the absence of loss of data packets in the “raw” data mode. NEC’2017

Data transmission between PC and DAQ board: new FLINK adaptor version FT600 USB3.0-FIFO Bridge chip (FTDI SuperSpeed USB device) instead CY7C68001 chip 4 endpoints are used: EP2, bulk, outFIFO, 2048 bytes, command block EP3, bulk, outFIFO, 2048 bytes, data from application EP82, bulk, inFIFO, 2048 bytes,data from HM, reading registers EP83, bulk, inFIFO, 2048 bytes, reading “raw” data Interface USB3.0 (5 Gb/s) instead USB2.0 (480 Mb/s) Important properties of the interface USB3.0 - SuperSpeed supports Streaming for bulk endpoints (important when receiving “raw”data) - USB 2.0 is a half-duplex broadcast bus while SuperSpeed is dual-simplex unicast bus which allows concurrent IN and OUT transaction - SuperSpeed supports continuous bursting NEC’2017

Data Transmission Protocols: new FLINK adaptor version Data Transmission Protocol: PC < - > adapter < - > DAQ board Transmission of packets from the PC to the modules and back is carried out in a unified format. Packet checking is performed only on the side of the module and in the PC. When sending commands and data from the PC to the USB, a two-byte ID code is added to the beginning of the packets, and a CRC-code is added to the end of the packets. Then the response packet is analyzed. “Raw” data, in addition to the ID and checksum code, have the current packet number after ID, which makes it possible to control the absence of packet loss. NEC’2017

Upgrading software for MPD and De-Li-DAQ-2D boards Use USB3.0 D3XX driver (proprietary interface specifically for FTDI SuperSpeed USB devices (FT60x series) , FTDI company) instead of USB2.0 CyUSB driver (Cypress company) Use API of FTD3XX library instead CyAPI.lib library The library contains all necessary functions for operation with endpoints Principal functions of the library that are used: read and write BULK endpoint FT_WritePipe and FT_ReadPipe for synchronous and FT_ReadPipe and FT_SetStreamPipe function for asynchronous transfers. When FT_SetStreamPipe function is used FT_ReadPipe no longer sends a session command to the chip because chip already knows how much data is requested . The programms use synchronous transmissions for: - sending command - reading HM and regiters - writing in HM asynchronous transmissions for: - reading “raw” data NEC’2017

Upgrading software Sending commands to blocks and receiving data from blocks are implemented according to the new protocol Software of the control units are a part of the software package SONIX+ Was keep the same functionality of the software: - initialization of the blocks - set the needed parameters and modes of blocks - start and stop of exposition - reading histogram memory and the registers - obtaining “raw” data - saving data on the hard disk - et al. NEC’2017

Conclusion Upgrading of the adapter and improvement of software for a new application-layer protocol have resulted in an increase of the bandwidth and reliability of the communication channel. NEC’2017

Thank you for your attention! NEC’2017