1. nSYNC and Data Format S. Cadeddu – INFN Cagliari P. Ciambrone – INFN LNF

2. nODE 4 nSYNC @ 48 channels 4 GBTx for hit+ TDC data – Slave GBT 1 GBTx forTFC/ECS – MasterGBT 1 GBT-SCA 2 VTTx 1 VTRx 07/07/2015S. Cadeddu: nSYNC & data format nSYNC 48Input ch Hit map TDC out VTTx GBTx VTTx GBTx ECS VTRx GBT SCA nSYNC 48Input ch Hit map TDC out nSYNC 48Input ch Hit map TDC out nSYNC 48Input ch Hit map TDC out GBTx 2

3. nSYNC block diagram 48 input channels – Each channel equipped with its own TDC – TDC resolution programmable – TDC output: Hit/NoHit + Signal phase BX synchronization Frame header builder TDC Zero suppression DAQ GBT interface controller TFC interface decoder ECS interface: I 2 C, SPI or both 07/07/2015S. Cadeddu: nSYNC & data format3

4. Fixed Frame – Our choice 07/07/2015S. Cadeddu: nSYNC & data format Header 48 Hit bits Data Frame: 112 bits 8 edac 3 3 3 3 3 3 3 3 3 3 Header 48 Hit bits 4 4 4 4 4 4 4 4 Header 48 Hit bits 5 5 5 5 5 5 8 edac 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 Frame using the 8 bits Hamming/EDAC, hits NZS, three possibility for TDC data (depending on the resolution), 16 bits for the Header – 3 TDC bits: max 13 channels (27%), one bit “empty” – 4 TDC bits: max 10 channels (20,8%), “full frame” – 5 TDC bits: max 8 channels (16,6%), “full frame” Header 48 Hit bits Data Frame: 112 bits 3 3 3 3 3 3 3 3 3 3 Header 48 Hit bits 4 4 4 4 4 4 4 4 Header 48 Hit bits 5 5 5 5 5 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 3 3 3 3 3 3 4 4 4 4 5 5 Frame using no protection, hits NZS, three possibility for TDC data (depending on the resolution), 16 bits for the Header – 3 TDC bits: max 16 channels (33,3%), “full frame” – 4 TDC bits: max 12 channels (25%), “full frame” – 5 TDC bits: max 9 channels (18,7%), three bits “empty” 4

5. Fixed Frame – Reasons of our choice We choose to use the Fixed Frame with Full Hit Map plus TDC data ZS. Data Frame (Header + HitMap + TDC) can be protected by an Hamming/EDAC code. This possibility will be programmable. We have always the Hit Map (i.e. the “physics” information) whatever the occupancy will be. The worst we truncate TDC data. – The HitMap scheme was approved by trigger people that said (Julien Cogan): ”it is much better to have the hit map separated from the hit times which are not used in the trigger. It would help if the hits are sorted by station and region to speed up the sorting in the software.” This solution guarantee the maximum occupancy. The GBT links are always aligned. Less resources in the nSYNC Less resources in the TELL40 FPGA. 07/07/2015S. Cadeddu: nSYNC & data format5

6. Technical details The Header start at the MSB position of the GBT frame ( bit 111 in our case) This is also true for the fields inside the Header Our frame is packed as following: – Header – Hit Map – TDC data – Hamming bits 07/07/2015S. Cadeddu: nSYNC & data format6

7. Questions Time: This is a list of open point for which we would like to have some feedback from experts. Header definitions: – BCID field, in our case we foreseen 12 bits. – From docs: “Information field is mandatory since the no data bits is mandatory”: Is this true also in case of Fixed Frame? It should be related to the Header only commands. The “truncation data bit” must be included in this field? Same information can be extracted from the Hit Map – Data_length field is mandatory only in the case of Variable Frame. We’ll use the Fixed Frame so we haven’t to implement this field: is it right? SYNCH command: – From docs: When TFC sends a SYNCH command, the FE must send a header containing just the full 12 bits BCID and the synch pattern. Our header is foreseen to be 16 bit wide and it should be of Fixed Header type: during synch command we have to “reduce” this size? Is it mandatory or is it foreseen for Variable Header only ? Header Only and BX veto command: – How must these command implemented with our fixed frame scheme? 07/07/2015S. Cadeddu: nSYNC & data format7

8. Data Frame: 112 bits Header 48 Hit bits 8 edac 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Header 48 Hit bits 4 4 4 4 4 4 4 4 8 edac 4 4 4 4 4 4 4 4 4 4 4 4 Header 48 Hit bits 5 5 5 5 5 5 8 edac 5 5 5 5 5 5 5 5 5 5 Data Frame: 112 bits Header 48 Hit bits 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Header 48 Hit bits 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Header 48 Hit bits 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Questions Time: In the Fixed Frame scheme, if some data bits are not used, can they be fixed to “0”? We foreseen to protect the frame with a Hamming/EDAC code. This can be programmable – Is there any potential problem in the TELL40 FPGA firmware side? TDC has a programmable resolution, the output can be 3,4,5 bits wide, The TELL40 should collect all the data from all the input links, probably it has to perform a new ZS before to repack data for output in order to optimize the bandwidth: – Is there any problem in this “variable size” from the TELL40 firmware point of view? 07/07/2015S. Cadeddu: nSYNC & data format8

9. Questions Time: ZS data transmission: NZS data transmission (?): – Our Hit Map (which contains physics information) is always NZS. – TDC data are used mainly for monitoring purposes. – We can give the possibility to fix (via ECS) the window of TDC data instead of having the ZS TDC (this can be used for special runs like PDM or calibration runs) – This means we ignore the NZS TFC command 07/07/2015S. Cadeddu: nSYNC & data format Head X Hit Map X ZS TDC data X Head X+1 Hit Map X+1 ZS TDC data X+1 Head X+2 Hit Map X+2 ZS TDC data X+2 Head X Hit Map X NZS TDC data-W1 X Head X+1 Hit Map X+1 NZS TDC data-W1 X 9 Head X Hit Map X NZS TDC data-W2 X Head X+1 Hit Map X+1 NZS TDC data-W2 X Head X Hit Map X NZS TDC data-W3 X Head X+1 Hit Map X+1 NZS TDC data-W3 X

10. TFC commands: Preliminary thought We start to think about the TFC commands and they can fit with the present Fixed Frame data format, in preparation with the meeting with Federico. Header Only: – we are not sure this make some sense with our data format. Bunch Crossing Veto: – Same as Header Only. In this case we can think to force all hits to zero, in order to speed-up the analysis (trigger etc). Calibration Type: – To be studied Non-Zero Suppressed Mode (NZS): – How to manage in Fixed Frame. – Hit map is always NZS. – The TDC may be appended in the following frames (in this case we can manage one NZS event every n bunches (n around 6) Synch command – There should be no problem Snapshot Command – There should be no problem 07/07/2015S. Cadeddu: nSYNC & data format10

11. SPARE 07/07/2015S. Cadeddu: nSYNC & data format11

12. nSYNC TDC Technology: UMC 130nm Size: (90 x 171)  m 2 Working Power consumption: <100  W It was submitted in February 2015 in the new ADV2 test chip. 07/07/2015S. Cadeddu: nSYNC & data format12

13. Variable Frame – FEC Frame using the 32 bits FEC, hits NZS, three possibility for TDC data (depending on the resolution), 16 bits for the Header – 3 TDC bits: max 5 channels (10,4%), one bit “empty” – 4 TDC bits: max 4 channels (8,33%), “full frame” – 5 TDC bits: max 3 channels (6,25%), one bit “empty” 07/07/2015S. Cadeddu: nSYNC & data format Header 48 Hit bits Data Frame: 112 bits 32 FEC bits 3 3 3 3 3 3 3 3 3 3 Header 48 Hit bits 32 FEC bits 4 4 4 4 4 4 4 4 Header 48 Hit bits 32 FEC bits 5 5 5 5 5 5 Header 9 Ch bits Data Frame: 112 bits 32 FEC bits Header 32 FEC bits Header 32 FEC bits 9 Ch bits 10 Ch bits 11 Ch bits Frame using the 32 bits FEC, Full ZS with 6 bits channel address, three possibility for TDC data (depending on the resolution), 16 bits for the Header – 3 TDC bits: max 7 channels (14,6%), one bit “empty” – 4 TDC bits: max 6 channels (12,5%), four bits “empty” – 5 TDC bits: max 5 channels (10,4%), nine bits “empty” 13

14. Variable Frame - Considerations With the variable frame the only choice we have is to use a full ZS (channel address + tdc data) in order to maximize the occupancy sustainable. Pro – Bandwidth optimized. – Full event NZS can be transmitted Con – Max occupancy at the limit (about 12% in the case of nominal TDC resolution of 4 bits). – If we have higher occupancy we loose physics and not only TDC information. – The TELL40 FPGA resources needed to manage this kind of format are very high, also because we cannot fulfill the request of “byte modularity” to minimize them. – The previous point means that we’ll need more TELL40 (i.e. more money). 07/07/2015S. Cadeddu: nSYNC & data format14

15. Fixed frame – EDAC 07/07/2015S. Cadeddu: nSYNC & data format Header 48 Hit bits Data Frame: 112 bits 7 edac 3 3 3 3 3 3 3 3 3 3 Header 48 Hit bits 4 4 4 4 4 4 4 4 Header 48 Hit bits 5 5 5 5 5 5 7 edac 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 Frame using the 7 bits EDAC, hits NZS, three possibility for TDC data (depending on the resolution), 16 bits for the Header – 3 TDC bits: max 13 channels (27%), two bits “empty” – 4 TDC bits: max 10 channels (20,8%), one bit “empty” – 5 TDC bits: max 8 channels (16,6%), one bit “empty” Header 9 Ch bits Data Frame: 112 bits Header 9 Ch bits 10 Ch bits 11 Ch bits 7 edac 9 Ch bits 10 Ch bits 11 Ch bits Frame using the 7 bits EDAC, Full ZS with 6 bits channel address, three possibility for TDC data (depending on the resolution), 16 bits for the Header – 3 TDC bits: max 9 channels (18,15%), eight bits “empty” – 4 TDC bits: max 8 channels (16,6%), nine bits “empty” – 5 TDC bits: max 8 channels (16,6%), one bit “empty” The last case is discarded: we have less max occupancy and the risk to loose physics 15

16. Head X-1 Hit Map X-1 ZS TDC data X-1 Questions Time: ZS data transmission: NZS data transmission (?): – Our Hit Map (which contains physics information) is always NZS. – TDC data are used mainly for monitoring purposes. – We can give the possibility to fix 07/07/2015S. Cadeddu: nSYNC & data format Head X Hit Map X ZS TDC data X Head X+1 Hit Map X+1 ZS TDC data X+1 Head X+2 Hit Map X+2 ZS TDC data X+2 Head X Hit Map X NZS TDC data-1 X Head X+1 Hit Map X+1 NZS TDC data-2 X Head X+2 Hit Map X+2 NZS TDC data-3 X Head X+3 Hit Map X+3 NZS TDC data-4 X Head X+4 Hit Map X+4 ZS TDC data X+4 16