Presentation for the Exception PCB February 25th 2009 John Coughlan Ready in 2013 European X-Ray Free Electron Laser XFEL DESY Laboratory, Hamburg Next Generation Systems

Presentation for the Exception PCB February 25th 2009 John Coughlan X-RAY Pixel Detectors DAQ For XFEL Up to 16M? tiled pixel detector at 30K frames/s -> 160 GBytes/sec Starting with 1 M pixel prototypes -> 10 GBytes/sec On Detector up to 256 x FPGA 40nm + 10G links Off Detector DAQ next gen Advanced Telecoms ATCA crate 1 8 SFP+ FPGA Detector Pixel Sensors 10G Fibre 30 m 10 GBytes/sec x N cards ATCA 8U Image Builders

Presentation for the Exception PCB February 25th 2009 John Coughlan Power ConnectorJTAG Front End Card FEE On Detector (16 cards per 1 MPixel Detector) Main Components and Interfaces 10 Gb Data To XFEL Train Builder Changed to 1 FEM per Super Module (128 ASICs). 16 FEMs in total. Readout with 10 Gb Optical links. Memory to balance data flow from ASICs to Optical Links. Fast Timing and Controls. Expect a common system to be defined. Drawing is Not to Scale Slow Controls LAN <~ 200 mm FPGA Virtex 5 Cfg EPROM Common Fast Timing/Controls DC-DC Linears Output Buffers Clock, Reset Record Reg Out Input Muxes ASIC status 128 x ASICs Osc Opto on FMC Mezzanine? Opto Module 10 Gb SFP+ SFP+ PHY DDR2/QDR2 <~ 125 mm Slow Controls 1 GbE

Presentation for the Exception PCB February 25th 2009 John Coughlan X-RAY Pixel Detectors DAQ For XFEL Up to 16M? tiled pixel detector at 30K frames/s -> 160 GBytes/sec On Detector 256 x FPGA 40nm + 10G links Off Detector DAQ next gen Advanced Telecoms ATCA crates … Scientific requirements ? 1 8 SFP+ FPGA Detector Pixel Sensors 10G Fibre 30 m 10 GBytes/sec x N cards ATCA 8U Image Builders

Presentation for the Exception PCB February 25th 2009 John Coughlan ~40% scaling Cross point Confign Zone 1 Zone 3 SFP+ PHY SFP+ PHY SFP+ PHY SFP+ PHY “Simplest” Train Builder Implementation. All FPGA / No Mezzanines. 10 Gb Inputs From Detectors 10 Gb Outputs To Processors Zone 2 DC-DC SFP+ PHY SFP+ FPGA SDRAM SFP+ PHY SFP+ FPGA SDRAM SFP+ PHY SFP+ FPGA SDRAM SFP+ PHY SFP+ FPGA SDRAM FPGA SDRAM FPGA SDRAM FPGA SDRAM FPGA SDRAM ½ MPix ATCA 8U Train Builder. Matched to 2D 8 x FEE outputs RTM

Presentation for the Exception PCB February 25th 2009 John Coughlan Image Builder for XFEL Demonstrator Advanced Mezzanine Card 180 mm RAM CONNECTORCONNECTOR CONNECTORCONNECTOR CONNECTORCONNECTOR CONNECTORCONNECTOR KEEP OUT FPGA FMC 2 x SFP+ 10 Gbps FMC 2 x SFP+ 10 Gbps FPGA KEEP OUT MGTMGT MGTMGT MGTMGT MGTMGT MGTMGT MGTMGT MGTMGT MGTMGT KEEPOUTKEEPOUT KEEPOUTKEEPOUT CONNECTORCONNECTOR CONNECTORCONNECTOR X- point TXTX TXTX TXTX TXTX RXRX RXRX RXRX RXRX FMC 2 x SFP+ 10 Gbps FMC 2 x SFP+ 10 Gbps AMC Form Factor. Migrating to 8 FPGAs on 8U ATCA? FPGA ~ 16 x 3-6 Gbps serial links Analogue Cross Point for Image Building. 3-6 Gbps DDR2/3 ~ 2-4 GBytes B/W 1-2 GBytes/sec In & Out VITA57 FMC Mezzanine I/O 2 x SFP+ opto TRx 10 Gbps (XAUI or RXAUI PHY) mTCA serial backplane 148 mm

Presentation for the Exception PCB February 25th 2009 John Coughlan Next Generation Board Issues High speed diff pairs 3-6 Gbps. BGA pitch < 1 mm? Reduce Fabrication Risk. Advanced PCB design and construction techniques. Vias in pad, micro vias, Laser drill. Incremental build up layer PCB. FPGA 60-40nm generation? FPGA to Memory interface. SO-RDIMMs WASSO Memory controllers Hard/Soft IP from FPGA vendors 10 Gbps optical interfaces 3-6 Gbps Serial Backplanes. Power. Multiple POL. Analogue. Decoupling caps. next gen FPGA packages. Pb Free manufacture? Tools Signal Integrity analysis, how to measure eye diagrams on board?

Presentation for the Exception PCB February 25th 2009 John Coughlan Train Builder Development Plan 1. Demonstrator AMC card with CrossPoint. Prototype system using MicroTCA crate. Optical Mezzanine on AMC? On Detector cards Mega Pixel detectors : Full ATCA board Prototypes. With RTMs Mega Pixel detectors: Full ATCA board Production with RTMs Multi MegaPixel detectors ? AMC FPGA SDRAM SFP+ PHY SFP+ FPGA SDRAM SFP+ PHY SFP+ FPGA Virtex 5 Cfg EPROM DC-DC Linears Osc Opto Module 10 Gb SFP+ SFP+ PHY DDR2/QDR2 Slow Controls 1 GbE