1. FTK-ROD FLOW CONTROL Anton Kapliy Enrico Fermi Institute University of Chicago March 30 2012 1

2. Design Objectives ATLAS Upgrade Week at Stanford. A.Kapliy - 03/30/2012 2 Send out one more (identical) copy of ROD data stream to FTK via a second S-Link fiber Allow FTK to assert flow control through the S-Link return channel, but otherwise have no impact on data-taking Operate in normal (single-channel) mode when FTK is disconnected / absent Provide a resilience mechanism against spurious data coming from the FTK link ◦ In the initial commissioning period, FTK may be power-cycled or re-loaded with bad firmware

3. Bird’s-eye view: S-LINK forward transmission ATLAS Upgrade Week at Stanford. A.Kapliy - 03/30/2012 3 FIFO32->16 D[32] @ 50 MHz D[16] @ 100 MHz TX wrapper TX duplicator TLK2501 transceiver Forward data flow: Identical data is sent through both optical fibers Data is sourced from a single FIFO and duplicated just before entering transceivers Downstream consumers of this data are: ROS’s ROBIN card FTK_IM mezzanine card Xilinx transceiver Serial connection via optical transceiver FTK_IM D[1] @ 2 GHz (after 8b/10b) S-Link firmware inside HOLA D[32] @ 40 MHz Data from the ROD comes in through a parallel port DAQ (ROS+ROBIN)

4. Bird’s-eye view: S-LINK return channel & XOFF ATLAS Upgrade Week at Stanford. A.Kapliy - 03/30/2012 4 FIFO32->16 D[32] @ 40 MHz D[32] @ 50 MHz D[16] @ 100 MHz Data from the ROD comes in through a parallel port TX wrapper TX duplicator RX wrapper (x2) TLK2501 transceiver Xilinx transceiver Serial connection via optical transceiver FTK_IM read-enable XOFF_FTK XOFF_DAQ Return data flow: Return channel allows ROS->ROD communication: Link-down, link-reset, XOFF, general return lines DAQ channel implements full S-Link protocol: Responsible for link startup & link reset FTK channel only implements XOFF part of S-Link protocol: FTK XOFF is effectively OR’ed with DAQ XOFF But see additional details in a later slide D[1] @ 2 GHz (after 8b/10b) DAQ (ROS+ROBIN) S-Link firmware inside HOLA If FIFO is half-full, assert LFF (Link-full flag) OR

5. FTK plays no role in link startup ATLAS Upgrade Week at Stanford. A.Kapliy - 03/30/2012 5 S-Link is a stateful protocol that requires a handshake with the remote side before the link can come up ◦ Handshake uses LDOWN and RRES signals from the ROS return line FTK does not participate in handshake: ◦ S-Link will come up even without FTK ◦ But: link will not work without a connected ROS On ROD side, we can start listening for FTK XOFF commands as soon as we see recovered clock on the FTK return line On FTK side (downstream), we use a modified S-Link receiver that does not require the initial handshake ◦ Just plug the fiber and it’s ready-to-use almost right away

6. Additional information on FTK XOFF ATLAS Upgrade Week at Stanford. A.Kapliy - 03/30/2012 6 FIFO RX wrapper (x2) TLK2501 transceiver Xilinx transceiver FTK_IM read-enable Actual flow control logic is: XOFF = (XOFF_FTK and FTK_XOFF_ENA) or (XOFF_DAQ) In other words, FTK will not participate in flow control at all – until the FTK_XOFF_ENA register has been set by the downstream FTK_IM board. If DAQ link goes down: the entire S-Link stops functioning until the initialization handshake is repeated with the ROS If FTK link goes down: XOFF_FTK signal will be automatically set to 0 if there is no recovered clock from FTK side. Effectively, the link will continue operating normally in DAQ-only mode. DAQ (ROS+ROBIN) FTK_XOFF_ENA (register) FTK_LRL[4] FTK_XOFF_ENA XOFF_FTK XOFF_DAQ OR FTK_XOFF_ENA register Set to 0 on power-up Set to 1 after the following 16-word sequence of 4-bit patterns is received on FTK_LRL line: d1e2 a3d4 b5a6 b7e8 Can be manually set to 0 again by setting FTK_LRL to 0xf (1111)

7. Firmware ATLAS Upgrade Week at Stanford. A.Kapliy - 03/30/2012 7 Original S-Link code from CERN assumes that the transceiver is implemented in an external device (TLK2501 chip) Chicago made the following changes: ◦ Utilized internal Altera FPGA transceivers ◦ Added a second channel, as described above INFN also has a version of S-Link firmware for Xilinx transceivers Feel free to email us with any technical questions, or if you need a copy of the code: ◦ Anton (HOLA & Altera S-Link): kapliy@gmail.comkapliy@gmail.com ◦ Alberto (FTK_IM & Xilinx): alberto.annovi@lnf.infn.italberto.annovi@lnf.infn.it