The trigger-less readout for the Mu3e experiment Dirk Wiedner On behalf of the Mu3e collaboration 31 March 20161Dirk Wiedner

Readout Requirements 31 March 2016Dirk Wiedner2 100 MHz muon decays 50 ns readout frames (pixel) O(5000) pixel chips O(7000) scintillating fibers O(7000) timing tiles Online filtering

Timing Detectors 31 March 2016Dirk Wiedner3 Scintillating fiber hodoscope Timing tiles On detector zero- suppression O(7000) fibers O(7000) tiles

Silicon Pixel Detector 31 March 2016Dirk Wiedner4 Inner double layer Outer double layer Re-curl layers o Both sides (x2) Sensor size o 2x2 cm inner sensors 4680 outer sensors

HV-MAPS 31 March 2016Dirk Wiedner5 H igh V oltage M onolithic A ctive P ixel S ensors HV-CMOS technology Reversely biased -85V o Charge collection via drift  Fast O(1 ns) o Thinning to 50 μm by Ivan Peric I. Peric, A novel monolithic pixelated particle detector implemented in high- voltage CMOS technology Nucl.Instrum.Meth., 2007, A582, 876

HV-MAPS 31 March 2016Dirk Wiedner6 H igh V oltage M onolithic A ctive P ixel S ensors HV-CMOS technology Reversely biased -85V o Charge collection via drift  Fast O(1 ns) o Thinning to 50 μm by Ivan Peric I. Peric, A novel monolithic pixelated particle detector implemented in high- voltage CMOS technology Nucl.Instrum.Meth., 2007, A582, 876

HV-MAPS 31 March 2016Dirk Wiedner7 H igh V oltage M onolithic A ctive P ixel S ensors HV-CMOS technology Reversely biased -85V o Charge collection via drift  Fast O(1 ns) o Thinning to 50 μm Integrated readout electronics o Zero suppression o 1.25Gbit/s serial LVDS outputs by Ivan Peric I. Peric, A novel monolithic pixelated particle detector implemented in high- voltage CMOS technology Nucl.Instrum.Meth., 2007, A582, 876

HV-MAPS 31 March 2016Dirk Wiedner8 H igh V oltage M onolithic A ctive P ixel S ensors HV-CMOS technology Reversely biased -85V o Charge collection via drift  Fast O(1 ns) o Thinning to 50 μm Integrated readout electronics o Zero suppression o 1.25Gbit/s serial LVDS outputs by Ivan Peric I. Peric, A novel monolithic pixelated particle detector implemented in high- voltage CMOS technology Nucl.Instrum.Meth., 2007, A582, 876

Pixel Readout Scheme 31 March 2016Dirk Wiedner9

Pixel Readout Scheme 31 March 2016Dirk Wiedner10 Pixel logic o Pixel address (8 bit) o Fine time (8 bit) o 50 ns frames Column logic o Pixel data o Column address o Coarse time Readout buffer Serializer and fast link(s) Pixel address Pixel Logic Column Logic Frame logic Readout buffer Serializer Fine time Coarse time Coarse time Column address Column address

Pixel Readout Scheme 31 March 2016Dirk Wiedner11 Pixel logic o Pixel address (8 bit) o Fine time (8 bit) o 50 ns frames Column logic o Pixel data o Column address o Coarse time Readout buffer Serializer and fast link(s) Pixel address Pixel Logic Column Logic Frame logic Readout buffer Serializer Frame logic Readout buffer Serializer 8 bit Fine time 8 bit Coarse time Coarse time Column address Column address

Pixel Readout Scheme 31 March 2016Dirk Wiedner12 Pixel logic o Pixel address (8 bit) o Frame number (4 bit) o 50 ns frames Column logic o Pixel data o Column address o Coarse time Readout buffer Serializer and fast link(s) Pixel address Pixel Logic Column Logic Frame logic Readout buffer Serializer Frame logic Readout buffer Serializer Fine time Coarse time Column address 8 bit 32 bit

Pixel Readout Scheme 31 March 2016Dirk Wiedner13 Pixel logic o Pixel address (8 bit) o Frame number (4 bit) o 50 ns frames Column logic o Pixel data o Column address o Coarse time Readout buffer Serializer and fast link(s) Pixel address Pixel Logic Column Logic Frame logic Readout buffer Serializer Fine time Coarse time Coarse time Column address Column address 32 bit 4 x 1.25 Gb/s

Data Link Scheme From detector slices to time slices 31 March 2016Dirk Wiedner14

Link Overview 31 March 2016Dirk Wiedner15 Front end links o Pixel sensor to on-detector FPGA 1.25bit/s LVDS o Timing detector readout Optical links from detector o Front end FPGAs o … to readout boards o 5 Gbit/s Optical links in counting room o Off-detector read out boards o …to PC Farm

Link Overview 31 March 2016Dirk Wiedner16 Front end links o Pixel sensor to on-detector FPGA 1.25bit/s LVDS o Timing detector readout Optical links from detector o Front end FPGAs o … to readout boards o 5 Gbit/s Optical links in counting room o Off-detector read out boards o …to PC Farm Pixel Sensor Pixel Detector FPGAs Readout board x12 Readout board x12 PC x48 PC x48

Link Overview 31 March 2016Dirk Wiedner17 Front end links o Pixel sensor to on-detector FPGA 1.25bit/s LVDS o Timing detector readout Optical links from detector o Front end FPGAs o … to readout boards o 5 Gbit/s Optical links in counting room o Off-detector read out boards o …to PC Farm Pixel Sensor FiberTile Pixel Sensor FiberTile Pixel Sensor FiberTile Pixel Sensor FiberTile Silicon FPGAs x60 Fiber FPGAs x48 Tile FPGAs x48 Readout board x16 Readout board x16 Readout board x8 Readout board x8 Readout board x8 Readout board x8 x5000x7000 PC x48 PC x48 O(8Tbit/s)

Tile Link Overview 31 March 2016Dirk Wiedner18 Front end links o Pixel sensor to on-detector FPGA 1.25bit/s LVDS o Timing detector readout Optical links from detector o Front end FPGAs o … to readout boards o 5 Gbit/s Optical links in counting room o Off-detector read out boards o …to PC Farm Pixel Sensor Fiber Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Silicon FPGAs x60 Fiber FPGAs x48 Fiber FPGAs x48 Tile FPGAs x48 Tile FPGAs x48 Readout board x16 Readout board x16 Readout board x8 Readout board x8 Readout board x8 Readout board x8 PC x48 PC x48 x240

Link Overview 31 March 2016Dirk Wiedner19 Front end links o Pixel sensor to on-detector FPGA 1.25bit/s LVDS o Timing detector readout Optical links from detector o Front end FPGAs o … to readout boards o 5 Gbit/s Optical links in counting room o Off-detector read out boards o …to PC Farm Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Silicon FPGAs x60 Fiber FPGAs x48 Tile FPGAs x48 Readout board x16 Readout board x16 Readout board x8 Readout board x8 Readout board x8 Readout board x8 PC x48 PC x48 x240x192 O(4Tbit/s)

Link Overview 31 March 2016Dirk Wiedner20 Front end links o Pixel sensor to on-detector FPGA 1.25bit/s LVDS o Timing detector readout Optical links from detector o Front end FPGAs o … to readout boards o 5 Gbit/s Optical links in counting room o Off-detector read out boards o …to PC Farm Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Silicon FPGAs x60 Silicon FPGAs x60 Fiber FPGAs x48 Fiber FPGAs x48 Tile FPGAs x48 Tile FPGAs x48 Readout board x16 Readout board x16 Readout board x8 Readout board x8 Readout board x8 Readout board x8 PC x48 x192 x96 O(4Tbit/s)

Front End FPGAs 31 March 2016Dirk Wiedner21 FPGAs on detector o 60 (+96) pieces Receive sensor data o 45 LVDS inputs 5 Gbit/s outputs o 8 optical links o … to counting house Switching data between readout boards farms A-D Front end FPGA 1.25 Gbit/s LVDS in x 45 5 Gbit/s optical Readout board A Readout board A Pixel Sensor Readout board B Readout board B Readout board C Readout board C Readout board D Readout board D

Front End FPGAs 31 March 2016Dirk Wiedner22 FPGAs on detector o 60 (+96) pieces Receive sensor data o 45 LVDS inputs 5 Gbit/s outputs o 8 optical links o … to counting house Switching data between readout boards farms A-D Front end FPGA 1.25 Gbit/s LVDS in x 45 5 Gbit/s optical Readout board A Readout board A Pixel Sensor Readout board B Readout board B Readout board C Readout board C Readout board D Readout board D

Front End FPGAs 31 March 2016Dirk Wiedner23 FPGAs on detector o 60 (+96) pieces Receive sensor data o 45 LVDS inputs 5 Gbit/s outputs o 8 optical links o … to counting house Switching data between readout boards farms A-D Front end FPGA 1.25 Gbit/s LVDS in x 45 5 Gbit/s optical Readout board A Readout board A Pixel Sensor Readout board B Readout board B Readout board C Readout board C Readout board D Readout board D

Front End FPGAs 31 March 2016Dirk Wiedner24 FPGAs on detector o 60 (+96) pieces Receive sensor data o 45 LVDS inputs 5 Gbit/s outputs o 8 optical links o … to counting house Switching data between readout boards farms A-D Front end FPGA 1.25 Gbit/s LVDS in x 45 5 Gbit/s optical Readout board A Readout board A Pixel Sensor Readout board B Readout board B Readout board C Readout board C Readout board D Readout board D

Front end FPGA Switching Board 31 March 2016Dirk Wiedner25 FPGA readout boards o 4 per sub-detector 5 Gbit/s optical inputs o 48 inputs 10 Gbit/s optical output o 12 outputs to PCs Switching network o A-D sub-farms o One output per PC Switching Board 5 Gbit/s Optical x28 PC 10 Gbit/s Optical PC Sub-farm A Front end FPGA Front end FPGA Front end FPGA PC x12

Switching Board 31 March 2016Dirk Wiedner26 FPGA readout boards o 4 per sub-detector 5 Gbit/s optical inputs o 48 inputs 10 Gbit/s optical output o 12 outputs to PCs Switching network o A-D sub-farms o One output per PC Front end FPGA Switching Board 5 Gbit/s Optical x28 PC 10 Gbit/s Optical PC Front end FPGA Front end FPGA Front end FPGA PC Sub-farm A x12

Switching Board 31 March 2016Dirk Wiedner27 LHCb PCIe40 PCB Marseille First prototypes available Front end FPGA Switching Board 5 Gbit/s Optical x28 PC 10 Gbit/s Optical PC Front end FPGA Front end FPGA Front end FPGA PC Sub-farm A x12

GPU-PC 31 March 2016Dirk Wiedner28 PC with GPU 10 Gbit/s Fiber input o 8 inputs from sub- detectors Data filtering o Timing Filter on FPGA o Track filter on GPU o Data to tape < 100 MB/s FPGA PCIe board GPU computer Optical mezzanine connectors

GPU-PC 31 March 2016Dirk Wiedner29 PC with GPU 10 Gbit/s Fiber input o 8 inputs from sub- detectors Data filtering o Timing Filter on FPGA o Track filter on GPU o Data to tape < 100 MB/s GPU computer

Readout board GPU-PC 31 March 2016Dirk Wiedner30 PC with GPU 10 Gbit/s Fiber input o 8 inputs from sub- detectors Data filtering o Timing Filter on FPGA o Track filter on GPU o Data to tape < 100 MB/s PC 10 Gbit/s Optical x8 PC 10 Gbit/s Optical x8 A Readout board B x8 Readout board

Vertex Filter 31 March 2016Dirk Wiedner31 Entire event on GPU Large target o Large spread of muons o Easy vertex separation Reject data without three tracks o … inside area interval on target 1 3 2

Vertex Filter 31 March 2016Dirk Wiedner32 Entire event on GPU Large target o Large spread of muons o Easy vertex separation Reject data without three tracks o … inside area interval on target 1 3 2

Summary Mu3e has 300M 10 8 muons/s 1 Tbit/s data 0-suppressed serial data from active pixel sensors Switched optical network GPU filter farm with optical inputs 31 March 2016Dirk Wiedner33 ++