CMX status Yuri Ermoline for the MSU group Mini-TDAQ week, CERN, 9-11 July 2012,

1/12 CMX: CMM upgrade CMX replaces CMM in L1Calo crate n Modern hardware implementation JEM/CPM to CMX (backplane) rate 40 →160Mbs  New data format n Crate CMX to System CMX (cables) rate 40 →160Mbs n Cluster information sent to TP (Topological Processor) over optical links (12-fiber nOptional partial TP capability included n Standalone TP is now planned Some TP capability still desired for CMX platform (flexibility) CMX

2/12 VME-- Interface TP-CMX FPGA Virtex-6 LX550T-FF1759 TP-CMX FPGA Virtex-6 LX550T-FF1759 Base-CMX FPGA Virtex-6 LX550T-FF1759 Base-CMX FPGA Virtex-6 LX550T-FF x Optic OUT 12x Optic IN DAQ ROI DAQ ROI TCM Interface VME-- TCM Inputs from all JEM or CPM processors from this crate LVDS cables From Crate To System CMX 400 single 160Mbps 3x27 LVDS up to 160 Mbps 2x 12-fiber ribbons OUT 3x 12-fiber ribbons IN 6.4 Gbps outputs to Standalone TP and/or TP-CMX 2x G-Link Out 6.4 Gbps inputs re-bundled from up to 12 Base-CMX CTP output 2x33 LVDS 40 Mbps (from TP-CMX via Base-CMX) CMX overview: modular design (Base + TP)

3/12 Most cards assembled without TP functionality VME-- Interface Base-CMX FPGA Virtex-6 LX550T-FF1759 Base-CMX FPGA Virtex-6 LX550T-FF x Optic OUT DAQ ROI TCM Interface VME-- TCM Inputs from all JEM or CPM processors from this crate LVDS cables From Crate To System CMX 400 single 160Mbps 3x27 LVDS up to 160 Mbps 2x 12-fiber ribbons OUT 6.4 Gbps outputs to Standalone TP and/or TP-CMX 2x G-Link Out CTP output 2x33 LVDS 40 Mbps (System CMX only)

4/12 CMX development activities + CMX input module n Philippe, Dan, Chip and Wojtek n Real-time data path layout - backplane to FPGA signal routing n Input module firmware currently under test - data capture at 160 MBit/s n Prototype blank card mechanical testing nVME/ACE/TTC (VAT) n Yuri + Seth Caughron (recently joined the efforts from software side) n 6U VME test card – PCB ready for testing in July nPower budget estimate n Measurements using 6U VME test card ðEstimation vs. real consumption

5/12 CMX development schedule (still valid) n2011: Project and engineering specifications n CMX project Preliminary Design Review (Done) n Preliminary design studies (Ongoing) n Test rig installed, checked out at MSU (postponed until 2012) n2012: Prototype design and fabrication n CMX schematics and PCB layout (ongoing) n CMM firmware ported on CMX n Prototype fabrication (fall 2012) n Basic tests for backward compatibility in test rig at MSU n Production Readiness Review n2013: Prototype testing/installation/commissioning, final fabrication n Full prototype tests in test rig at CERN n CMX firmware development and test n Test in the L1Calo system during shutdown n Fabricate and assemble full set of CMX modules n2014: Final commissioning in the L1Calo trigger system in USA15

6/12 CMX development timeline

7/ nPrototype design and fabrication n CMX schematics and PCB layout (ongoing) ðDual FPGA design: Base-CMX & TP-CMX (RAL Feb 2012) ðReal-time data path layout (ongoing) ðInput module FW is currently under development and test ð6U VME test card for VME/ACE/TTC (VAT) interface (PCB ready for testing) ðPower budget estimate n CMM firmware ported on CMX ðMost of CMM data path work completed at Stockholm (Pawel & Sam) needs to be adapted to final package choice ðFirmware for VAT interface (2 CPLD + TTC FPGA) to Spartan-3AN (Yuri) n Prototype fabrication ðmechanical testing (blank card, backplane connectors, front panel tests) Will happen before real prototype board, but not urgent yet. n Basic tests for backward compatibility in test rig at MSU n Production Readiness Review (L1Calo/TDAQ) Fall/Winter ðAfter the prototype is tested - final check before going into full production

8/12 Backplane data transfer nDesign challenges: n 25 backplane 160 MHz from 16 sources to CMX n Absolute and Relative skew of incoming Processor input signals ðMax skew within signals from one source processor module (2 ns ?) ðUnequal length of the traces (source/backplane/destination) ðThere will only be a small difference in trace length on CMX, and it will likely not be the dominant source of skew among signals coming from a given processor module n3 possible schemes: n forwarded coded clock/parity + 24 data (96 bits) n forwarded clock + 24 data (96 bits including 1 to 4 parity bits) n parity + 24 data (96 bits, no forwarded clock - global clock from TTC) nAvailable resources in FPGA: n Data phase correction in IODELAY (per trace) – up to 2.4ns n Clock phase adjustment in MMCM (per source) n Clock capable pins, clock buffers and clock distribution networks

9/12 Forwarded coded clock/parity + 24 data nData/clock/parity need to be brought in alignment at the FPGA inputs n PCB/backplane traces length difference compensated by IODELAY ðImplemented in BLT, software based procedure, done once nCoded clock/parity – signal used as clock and as data n Need MMCM (located far from IO pins) to recover clock ð16 MMCMs out of 18 used, Extra clock delay n “Hand-made” FPGA layout nNeed to re-synchronize to CMX internal clock n96 data bits available

10/12 Forwarded clock + 24 data nImplemented in BLT n Tested with up to 8 sources in the crate nData/clock/parity need to be brought in alignment at the FPGA inputs n PCB/backplane traces length difference compensated by IODELAY ðsoftware based procedure, done once nForwarded clock used directly in IO blocks n No need for MMCMs n Regional clock distribution networks nNeed to re-synchronize to CMX internal clock nPenalty: < 96 data bits available n 1 to 4 parity bits n  Defined as preferable data transfer scheme n PCB/FPGA layout should be able to support all 3 schemes ðregional clock input for all input signals from same CPM/JEM ðcleanest/most direct access possible to one MMCM if needed

11/12 Test card for VME/ACE/TTC (VAT) part of CMX nRedesign HW with new components nFit design into single device n6U VME test card: n HW/SW implementation n Main FPGA re-configuration n To be merged into CMX design nPCB is ready for testing, next: n VME/ACE/TTC Firmware for VAT and Virtex6 FPGAs:  Specify new VME register model n Test Stand and software VME CPLDI2C CPLDACE FPGA

12/12 Conclusions nOriginal schedule is maintained n With minor adjustments nPrototype effort proceeding in parallel on 3 fronts according to plan n Providing necessary information for technical decisions ðIncreasing level of details nTechnical decisions are made n Upon agreement within L1 Calorimeter Trigger ðCMX Base/TP functionality, FPGA choice, link speed, data transfer scheme nGeneral outlook is optimistic