M. Bellato INFN Padova and U. Marconi INFN Bologna Status of the ROM M. Bellato INFN Padova and U. Marconi INFN Bologna SuperB meeting - Frascati 4/7 April 2011

SuperB meeting - Frascati 4/7 April 2011 Parameters. The estimated event size at the FE is 500 kB. The maximum trigger rate is 150 kHz. We set the aggregate ROM input rate to 10 Gb/s. Number of FEE 2Gb/s data links : 325 With these assumptions the number of ROMs needed to manage the SuperB data flux can be estimated to be of the order of: (8. × 500. ×103 ) × (150. × 103)/(10. × 109) = 60. boards Each ROM will handle ∼10 kB of fragment size at 150 kHz. SuperB meeting - Frascati 4/7 April 2011

18 June 2009

SuperB meeting - Frascati 4/7 April 2011 ROM implementation. We foresee two possible approaches to the implementation of the ROM. Based on custom electronics and field busses: using FPGA to get data from the FEE, perform “simple” synchronous processing, and format data according to a suitable industrial standard. Based on custom electronics and host PC/CPUs: FPGA being used to get data from the FEE (possibly perform synch. Processing) and inject them into the PC via PCIe. CPUs will perform complex data processing and data transfer, using standard protocol and on board network interface cards. SuperB meeting - Frascati 4/7 April 2011

SuperB meeting - Frascati 4/7 April 2011 Implementation (I) Based on a field bus (VME, xTCA, …) 5 x FEE optical links at 2Gb/s as input 1 x 10Gb/s as output FCTS interface is not an issue FPGA centric design Inputs deserialization Synchronous processing (feature extraction) Fragment formatting according to event builder protocol (TCP/UDP, Converged Ethernet, custom) Output serialization Once design is fixed, changes are difficult Improvements are limited, if not foreseen in advance SuperB meeting - Frascati 4/7 April 2011

Implementation (I) – Block Diagram SuperB meeting - Frascati 4/7 April 2011

SuperB meeting - Frascati 4/7 April 2011 Implementation (II) Solution based on FPGA and PC/CPUs appears to be more versatile than the one based on stand alone custom board. The required functionality implemented by means of high level languages. Transmission protocol to the HLT farm is no more a constraint. The trigger rate of 150 kHz constraints the available processing time per CPU to NCPU / 150. ms NCPU being the number of CPU cores per PC box. A processing time of the order of 1 ms per CPU would require therefore order of 100 CPU per box. Availability of many-cores CPUs …………? see next slides. SuperB meeting - Frascati 4/7 April 2011

Implementation(II) – Block Diagram Power Management SFP RTX FCTS I/F SNAP12 RX Front-Ends FPGA – Virtex 6/7 PCIexpress 4x Dual port RAM Clock Management SuperB meeting - Frascati 4/7 April 2011

+ How to get High Performance and Energy Efficiency for highly Parallel Workloads? high F.P. performance (VPU/SIMD) + many integrated small energy efficient and high-performance cores small extreme energy efficient core The Newest Addition to the Intel Server Family. Industry’s First General Purpose Many Core Architecture

Knights Ferry - Aubrey Isle Processor Memory Controller System & I/O PCIe Interface Function Fixed Multi-Threaded Wide SIMD I$ D$ . . . Shared coherent L2 Cache GDDR GDDR . . GDDR GDDR http://download.intel.com/technology/architecture-silicon/Siggraph_Larrabee_paper.pdf Multiple IA cores 16-wide vector units (512b) 1024-bit ring bus - In-order, short pipeline - Extended instruction set GDDR memory - Multi-thread support Fully coherent caches - Supports virtual memory Standard IA Shared Memory Programming Future options subject to change without notice.

>50 Intel Architecture cores The “Knights” Family Future Knights Products Knights Corner 1st Intel® MIC product 22nm process >50 Intel Architecture cores Knights Ferry Development Platform Future options subject to change without notice.

“Knights Ferry” Development Platform Software Development Platform Growing availability through 2011 Up to 32 cores, up to 1.2 GHz Up to 128 threads at 4 threads / core Up to 8MB shared coherent cache 1-2 GB GDDR5 shared memory PCIe Card Bundled with Intel HPC SW tools Software development platform for Intel® MIC architecture

SuperB meeting - Frascati 4/7 April 2011 ROM PCIe Architecture The interface board (FEE to PCIe) plugged into the PC mother board. PCIe Gen2 x4 lanes has enough bandwidth: (4.×5.×109 bit/s) (150 kHz)-1 = 16.7 KB ( > fragment size ~ 10 KB) Adequate cooling of the interface board may be an issue (but estimated power consumption < 40 W). Mechanical fit is poor FCTS interface should be integrated in the ROM ? SuperB meeting - Frascati 4/7 April 2011

ROM PCIe Architecture (II) Event builder farm may fit the ROMs as plug-in cards Exploit existing hardware and reduce costs Event builder farm may fit also many-cores CPU cards as add-on HLT may be performed on the same farm + filter farm SuperB meeting - Frascati 4/7 April 2011

SuperB meeting - Frascati 4/7 April 2011 Ongoing Tests Proof of concept performed on Xilinx ML605 with Virtex-6 LX240T and PCIe Gen2 4x SuperB meeting - Frascati 4/7 April 2011

SuperB meeting - Frascati 4/7 April 2011 Ongoing tests SuperB meeting - Frascati 4/7 April 2011

SuperB meeting - Frascati 4/7 April 2011 Plans Legacy implementation based on field busses requires investigation on the 10Gb/s output links in terms of protocol implementations on FPGA PCIe based implentation requires Development of a Linux driver to test the performance. Evaluate process latency of getting and transmitting data: useful time for processing. Evaluate Knights Ferry or GP-GPU for feature extraction ? SuperB meeting - Frascati 4/7 April 2011