1. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 TTC upgrade The TTC-PON project 1

2. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 Defining ‘TTC upgrade’ for this talk Upgrade Requirements From TTC to PON TTC-PON project TTC for Alice - ideas 2 Outline

3. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 CANDIDATE FOR TTC UPGRADE Definition 3

4. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 4 Generic TTC in experiment CTP LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC … OFF detector ON detector

5. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 5 Candidate for Versatile Link upgrade CTP LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC … OFF detector ON detector

6. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 6 Candidate for TTC upgrade CTP LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC … OFF detector ON detector

7. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 7 Candidate for TTC upgrade CTP LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC … OFF detector ON detector

8. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 TTC UPGRADE Requirements 8

9. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 Bi-directionality (see next slides) Increased bandwidth Backward compatibility with legacy TTC system Common system Scalability Partitioning flexibility 9 Preliminary Requirements Upgrade Requirements Defined by the TTC upgrade working group in 2009-2010

10. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 Downstream, broadcast/unicast: – Machine Timing Signals with programmable phase, low, deterministic and fixed latency: Bunch Clock (40.078MHz) – low jitter required Orbit (11.2 kHz) = BCR – Experiments’ Synchronous Signals with low, fixed and programmable latency: Triggers (max 40.078MHz), together with minimum 8 bits of Trigger Type Event Counter resets (ECR) Calibration Pulses – Low to medium rate Asynchronous Signals: Slow control Region of Interest Readout (ROI) Event routing information (IP address?)... 10 Bi-directionality Upgrade Requirements

11. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 Upstream: – Feedback and acknowledgment for control – latency controlled signals (calibration requests) – Busy/throttle signals transmitted with short and bonded latency – Fiber latency monitoring –... 11 Bi-directionality Upgrade Requirements

12. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 FROM TTC TO PON What is a PON? The PON-TTC proof of concept 12

13. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 The upgraded TTC should be… Bidirectional Point to Multipoint Scalable … and requires High bandwidth downstream for broadcast (Gbps) Relatively lower bandwidth upstream 13 TTC-PON: a natural answer…  typical topology of a PON system + constraints on latency and clock recovery From TTC to PON

14. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 14 PON = a mature technology FTTH/B vs VDSL subscribers worldwide, 2011-2016, millions Ranking of the 10 main FTTH/B players in the world, in late 2011. FTTH/B Subscribers worldwide (millions) Major topology of FTTH/B

15. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 15 What is a PON? OLT ONU1 ONU2 ONU3 FTTH FTTC FTTB Passive Optical Network Major topology of growing Access Network Market (FTTx) Point-to-MultiPoint (P2M) One single fibre in charge of both downstream and upstream transmissions

16. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 16 What is a PON? Downstream: 1490nm @ 1.25 Gbps OLT ONU1 ONU2 ONU3 FTTH FTTC FTTB 123113 123113 1 2 3 1 1 3 1 2 3 1 1 3 111 2 33 1490nm Example of 1G-(E)PON

17. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 17 What is a PON? Upstream: 1310nm @ 1.25 Gbps OLT ONU1 ONU2 ONU3 FTTH FTTC FTTB 222 1 1 1 3 3 3 111 222 3 33 1310nm 222 1 1 1 3 3 3 Example of 1G-(E)PON

18. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 18 What is a PON? Upstream: 1310nm @ 1.25 Gbps OLT ONU1 ONU2 ONU3 FTTH FTTC FTTB 123113 123113 1231 13 1231 13 111 2 33 1490nm 222 1 1 1 3 3 3 111 222 3 33 1310nm 222 1 1 1 3 3 3 Example of 1G-(E)PON

19. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 19 What is a PON? OLT ONU1 ONU2 ONU3 FTTH FTTC FTTB 123113 123113 1231 13 1231 13 111 2 33 1490nm 222 1 1 1 3 3 3 111 222 3 33 1310nm 222 1 1 1 3 3 3 Example of 1G-(E)PON

20. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 20 TTC-PON proof of concept [2010] Study conducted by I. Papakonstantinou (Aceole Fellow, now at UCL), Csaba Soos (CERN/PH/ESE) and the Opto Team of PH/ESE/BE. And documented : Passive Optical Networks in Particle Physics Experiments, Ioannis Papakonstantinou, 24 th November 2009, PH-ESE Seminar Passive Optical Networks in Particle Physics Experiments A Fully Bidirectional Optical Network with Latency Monitoring Capability for the Distribution of Timing-Trigger and Control Signals in High-Energy Physics Experiments, Ioannis Papakonstantinou et al., IEEE TNS, Aug. 2011. A Fully Bidirectional Optical Network with Latency Monitoring Capability for the Distribution of Timing-Trigger and Control Signals in High-Energy Physics Experiments, OLT ONU1 ONU2 1490nm, 1.6Gbps 1310nm, 800Mbps 1B2KB1B2KB 1B2KB1B2KB 1B2KB1B2KB OLT ONU1 12 2 1 B2KBB2KB 1BKB1BB

21. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 21 TTC-PON proof of concept [2010] From TTC to PON

22. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 TTC-PON proof of concept [2010] Raw rate 1.6Gbit/s Arbitrary protocol for 64 ONUs Synchronous transmission of super-frames with a period of 1625ns = 65*25ns at 1.6Gbit/s 8b/10b encoding (1Field = 1 symbol) 590.8 Mb/s broadcast only 9.23 Mb/s per ONU FieldOperation Frame alignment and synchronization L1A trigger accept decision, real time transmission Auxiliary, to be defined in the future (could be used for upstream channel arbitration to reduce buffering capacity), Broadcasted/individually addressing commands (depending on first bit of D1) Upstream channel arbitration (address of the next ONU allowed to transmit upstream) TTC individual addressing frame length = 1050ns) 22 Spare slides Downstream Protocol

23. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 Channel arbitration at the OLT (based on a simple round-robin scheme) Slave N1 receives an R/F character with its address and switches its laser ON IFG between successive emissions allows receiver to adapt between bursts Long sequence of idle bytes for CDR & frame alignment 4 bytes of payload Total BW 800Mb/s Latency not fixed but bonded Optimized latencies with 1G-EPON components – if 64 ONUs: 14 us (BW/slave=2.2Mb/s) – If 32 ONUs: 7 us (BW/slave=4.5Mb/s) – If 8 ONUs: 2 us (BW/slave=18Mb/s) – … TTC-PON proof of concept [2010] ~300ns for high dynamic range systems (20dB) 50ns for low dynamic range (<6dB) 23 Spare slides Upstream Protocol

24. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 24 TTC-PON proof of concept [2010] Properties1G-PON P ROOF - OF - CONCEPT TTC LEGACY  Clock Rate Distance Splitting Ratio Encoding 40 MHz 100 m - 1000 m Flexible, Up to 64 NRZ 8b/10b 40 MHz 100 m -1000m Flexible, up to 32 BPM  Bit Rate Wavelength Trigger Rate Trigger Type Latency Error Detection Error Correction Broadcast Payload Individual Payload Recovered Clock Jitter 1.6 Gb/s 1490 nm 40 MHz, unconstrained Synchronous, 8 to 32 bits Fixed and Deterministic, 10BX 1 bit (8b/10b) - 590Mbps 9.2Mbps guaranteed per ONU if 64 ONUs 25ps rms out of the FPGA (no extra PLL) 160 Mbps 1310 nm 40MHz, but max 11 consecutive Asynchronous Fixed and Deterministic, 4BX 2 bits 1 bit (Hamming) 20Mbps 7.6Mbps to be shared between Rx 25ps rms out of the TTCrx (no QPLL)  Bit Rate Wavelength Latency Payload BW Allocation 800 Mb/s 1310 nm Bonded, split ratio related. 1:8 => 2us,1:64 => 14 us Split ratio related. 1:8 => 18 Mb/s, 1:64 => 2.2Mb/s TDMA, Fixed, Round Robin X Compared Performance wrt TTC legacy

25. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 25 TTC-PON proof of concept [2010] Properties1G-PON P ROOF - OF - CONCEPT TTC  Clock Rate Distance Splitting Ratio Encoding 40 MHz 100 m - 1000 m Flexible, Up to 64 NRZ 8b/10b 40 MHz 100 m -1000m Flexible, up to 32 BPM  Bit Rate Wavelength Trigger Rate Trigger Type Latency Error Detection Error Correction Broadcast Payload Individual Payload Recovered Clock Jitter 1.6 Gb/s 1490 nm 40 MHz, unconstrained Synchronous, 8 to 32 bits Fixed and Deterministic, 10BX 1 bit (8b/10b) - 590Mbps 9.2Mbps guaranteed per ONU if 64 ONUs 25ps rms out of the FPGA (no extra PLL) 160 Mbps 1310 nm 40MHz, but max 11 consecutive Asynchronous Fixed and Deterministic, 4BX 2 bits 1 bit (Hamming) 20Mbps 7.6Mbps to be shared between Rx 25ps rms out of the TTCrx (no QPLL)  Bit Rate Wavelength Latency Payload BW Allocation 800 Mb/s 1310 nm Bonded, split ratio related. 1:8 => 2us,1:64 => 14 us Split ratio related. 1:8 => 18 Mb/s, 1:64 => 2.2Mb/s TDMA, Fixed, Round Robin X Preliminary results based on the simple proof of concept

26. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 THE TTC-PON PROJECT Investigation paths Roadmap 26

27. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 First prototype very promising Still in the investigation phase Many ideas to optimize this early proof-of-concept – 1G-TDMA protocol optimization, specific TIA for OLT – 10G-TDMA Commercially available products for FTTx higher payload, lower latency, TTC legacy & TTC-PON overlay? – Other multiplexing techniques Still prototyping stage for FTTx Wavelength multiplexing (WDM), Orthogonal Codes (CDMA) 27 TTC-PON investigation paths From 1G to 10Gλ multiplexing

28. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 Objectives: – Explore all the paths to push the performance of the TTC-PON to its maximum – Get as much feedback as possible from experiments Regular reports/demo to experiments Specification group to validate major steps – Install a prototype hosted by CMS trigger upgrade for evaluation (FMC based) – Propose one or several products to experiments end of 2015 Using several PON technologies Form factor: optical components + small reference designs + firmware IP blocks Crew – TTC and PH-ESE opto team – A new doctoral student from Ioannina University (Greece) – CERN-UCL collaboration for potential WDM investigations 28 TTC-PON Project The TTC-PON Project

29. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 29 TTC-PON Project Roadmap 2012 2013 2014 2015 2016 2017 2018 2019 2020 LS1 LS2 TDMA investigations (using evaluation kits) Prototyping, qualification and commissionning of a demonstrator within CMS TTC upgrade (FMC based) Present final prototypes @ TWEPP15 Investigation on other multiplexing schemes with UCL (CDMA, WDM) Prototyping, pre-production The TTC-PON Project Specification with experiments, reviews, etc

30. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 TTC-PON FOR ALICE - IDEAS 30

31. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 31 What PON could bring to ALICE… CTP LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC …

32. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 32 This … CTP LTU TTCoc Read Out Crate FEB GBT LTU TTCoc Read Out Crate FEB GBT LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC … OLT PON

33. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 33 This … CTP LTU TTCoc Read Out Crate FEB GBT LTU TTCoc Read Out Crate FEB GBT LTU TTCvi TTC ex TTCoc Read Out Crate FEB … Busy Data TTC … OLT PON Busy BusyType Calibration Request Feedback GBT status ONU status Latency measurement L1A ECR BCR TrigType Calibration Pulse Configuration Slow control

34. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 34 Or this … CTP LTU TTCoc Read Out Crate FEB GBT LTU TTCoc Read Out Crate FEB GBT LTU TTCoc Read Out Crate FEB … Data TTC … OLT PON OLT To be checked

35. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 35 Or this … CTP TTCoc Read Out Crate FEB GBT TTCoc Read Out Crate FEB GBT TTCoc Read Out Crate FEB … Data TTC … OLT PON OLT … One OLT per partition

36. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 36 Or even this … CTP TTCoc Read Out Crate FEB GBT Read Out Crate FEB GBT TTCoc Read Out Crate FEB GBT … OLT PON OLT … … One OLT per sub-detector Software partitionning

37. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 37 We are in specification stage Every feedback is most welcome! A preliminary project description is available herehere

38. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 SPARE SLIDES 38

39. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 39 Generic TTC in experiments – block diagram Det. Module Det. Module LTU TTCex FE driver TTCrx Det. Module Det. Module TTCrx Det. Module Det. Module TTCrx Det. Module Det. Module FE Controller TTCrx FE driver TTCrx Det. Module Det. Module 1 10 1 16/32 Counting room Detector DAQ Introduction

40. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 40 Candidate for GBT & VERSATILE LINK upgrade Det. Module Det. Module LTU TTCex FE driver TTCrx Det. Module Det. Module TTCrx Det. Module Det. Module TTCrx Det. Module Det. Module FE Controller TTCrx FE driver TTCrx Det. Module Det. Module 1 10 1 16/32 Counting room Detector DAQ TTC upgrade overview

41. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 41 Candidate for GBT & VERSATILE LINK upgrade Det. Module Det. Module LTU TTCex FE driver TTCrx Det. Module Det. Module TTCrx Det. Module Det. Module TTCrx Det. Module Det. Module FE Controller TTCrx FE driver TTCrx Det. Module Det. Module 1 10 1 16/32 Counting room Detector DAQ TTC upgrade overview Not the subject of this talk

42. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 42 Candidate for TTC upgrade DAQ Det. Module Det. Module TTCrx Det. Module Det. Module TTCrx Det. Module Det. Module TTCrx Det. Module Det. Module Det. Module Det. Module Counting room Detector DAQ LTU TTCex FE driver TTCrx FE Controller TTCrx FE driver TTCrx 1 10 1 16/32 Introduction

43. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 TTC-PON 2010 Demonstrator Performance C URRENT TTC AND PON-TTC D OWNSTREAM LATENCY CHARACTERISTICS PON-TTCL ATENCY Current TTCL ATENCY GTX T X 75 nsTTCex25 ns EPON OLT T X 2.11 ns EPON ONU R X 2.16 nsTTCrx65-85 ns GTX-GTP R X 137.5 ns Total216.8nsTotal90-110 ns Optical Fiber5ns/mOptical Fiber5ns/m Point of Measurement RMS C2C J ITTER ( PS ) Ref 40MHz3.17 Recovered 40 MHz, ONU1 Filtered 40 MHz, ONU1 36.72 3.6 Recovered 40 MHz, ONU2 Filtered 40 MHz, ONU2 53.12 3.8 Field 1G-EPON (C URRENT ) 10G-EPON2GPON10GPON Overlapping IFG (ns)50 16 Training (ns)12512.562.512.5 4B Payload (ns)404204 Total per frame (ns)21566.598.532.5 No. of bunch-cycles between transmissions9342 M IN U PSTREAM F RAME D URATION B REAKDOWN A NALYSIS PON-TTC J ITTER CHARACTERISTICS Spare slides 43

44. Sophie Baron – TTC upgrade – PON concept – ALICE - 29 October 2012 44 TTC-PON proof of concept (2010)