1. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 20031 Optical Data Links in CMS ECAL CMS ECAL architecture and needs Optical Data Links in HEP Experiments ECAL Data Link system description Data Link components Data Link system Opto-Hybrid assembly, Quality Assurance Schedule

2. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 20032 The CMS Detector Silicon Tracker ECAL HCAL Solenoid Muon Chambers Return Yoke Barrel Preshower + Endcap

3. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 20033 CMS ECAL front-end architecture ECAL has ~77,000 lead tungstate crystals arranged in supercrystals of 25. Front-end electronics of each supercrystal send the data off-detector via the optical links. The architecture and the needed data rate require ≥600 Mb/s with modularity: for Data: 1 link / supercrystal for Trigger: 1 link / supercrystal (barrel) 5 links/supercrystal (endcap) Total data + trigger: ~9000 links Supercrystal Data Trigger sums APD/VPT Preamp ADC Front-end board

4. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 20034 Optical Links and HEP Generic architecture of an optical link: TxRx fiber “Optical Layer” SerDeser “First Overlay” MUXDEMUX “Framing” e/o, transport, o/e encoding/decoding, usually performed in same chip as serialization/deserialization Multiplexing multiple “conversations” in each link. Not usually relevant for HEP since systems are designed to run each link continuously at maximum data rate. data indata out

5. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 20035 Optical Layer considerations - 1 Tx considerations: Edge-Emitting: linear response (relevant if you design an analog link) VCSEL: said to be more rad-resistant due to smaller active area, but practical experience is that fractional change of behavior per unit irradiation is typically similar. VCSEL is cheaper to produce and test on wafer. Was once thought that they would entirely replace Edge-Emitting. But benefit is diluted since packaging dominates the cost of both. In any case: device chosen must be consistent with the system speed and environmental specifications (often unique to HEP; more on this later). Tx: In HEP, usually either Edge-Emitting Laser Diode or VCSEL. Rx: Typically photodiode or photodiode array (typically of 12), followed by digital amplifying ASIC. TxRx fiber “Optical Layer” e/o, transport, o/e Rx considerations: Speed, wavelength, saturation and sensitivity limits must be consistent with system specifications. VCSEL diagram Edge-Emitting diagram test on wafer

6. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 20036 Optical Layer considerations - 2 Optical fiber considerations: Two principle kinds: Single Mode (SM) and Multi Mode (MM). Differences in impurities in the silica and differences in core and cladding diameters give different spectral transmission characteristics. SM: lower attenuation, is the favored choice for telecom (longhaul) distances (10 km and greater). Uses smaller core to allow only one mode to propagate, making connector alignment more critical (and expensive). MM: favored for “datacom” (LAN) distances (~5 to 10 km). In principle cheaper than SM, but fiber itself may be more expensive since market is smaller (datacoms did not take off). HEP distances are short (~100m), so SM/MM choice is not critical. Usually free to select the cheapest solution satisfying environmental specifications. Qualification is essential: Early ECAL qualification of MM fiber of various suppliers: Good fiber loses <50% transmission after 10 13 p/cm 2. Bad fiber turns black. P. Denes

7. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 20037 First Overlay considerations Serializer/deserializer considerations: Encoding: Two principle protocols are CIMT (“G-Link”, Agilent proprietary) and 8-bit/10-bit. Both share the following characteristics: introduce an overhead of two bits per 8 bits of data are usually employed continuously transmitting, either data or idle pattern aid in maintaining synchronization of transmitter and receiver allow some error checking stabilize 0 and 1 levels by keeping number of bits of each the same Principle protocol differences: G-Link is considered more robust in terms of maintaining and recovering synchronization. But it is proprietary. At the moment the fastest CIMT deserializer runs up to 1.4 Gb/s. In general there is no guarantee that a company employing proprietary technology will continue developing new products with it, nor any guarantee that it will not discontinue existing ones. Speed and bits serialized: Typical scheme at LHC experiments is one (two) 16-bit words serialized at LHC clock rate of 40 MHz implying optical link speed of 800 (1600) Mb/s of which 80% is data and 20% is overhead. TxRx fiber SerDeser “First Overlay” encoding/decoding, usually performed in same chip as serialization/deserialization

8. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 20038 General considerations In designing an optical link for a HEP experiment: Typically start with a shopping list of needs. From that define specifications and select/develop technology. End up with a laundry list of “issues”. The typically O($1M) procurement for a HEP experiment is relatively small for any company large enough to collaborate in the task. Development teams in laser transmitters are smaller than one might think, typically 10-30 people for a large company. => The experiment will incur significant development and qualification effort and costs. HEP experiments typically have many specifications which are unusual in industry. The closer the transmitter is to the center of the detector, the more likely this is to be true. Every need has its consequence(s): radiation resistance, low power (heat), temperature cycle, non-magnetic materials extensive qualification of several products (variation of behavior between different products can be large) limited space custom laser package, with consequences for hermeticity and mounting techniques long (3-10 yrs) development time possibility that a product will be discontinued 100% reliability of ~10k links over 10 years sleepless nights

9. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 20039 Optical Data Link System Diagram 12 Rx module 12 1 GOH 12 1 96 Distributed Patch Panel Back-end Patch Panel Off Detector Front End Pigtail fiber Ruggedized ribbon Dense multi- ribbon cable GOL Laser diode In-Line Patch Panel CMS PIN photo- diode array Digital amp. ASIC 12 Edge-Emitting Laser Diode: ST Microelectronics Digital Optical Receiver: NGK GOL ser. ASIC: CERN MIC GOL Opto-Hybrid (GOH): ECAL designs, prototypes, qualifies, defines manufacturing specifications, procures the manufacture, tests samples during production. 12-channel NGK Rx: Off-the-shelf component: ECAL qualifies, procures, tests samples during production. Fiber, connectors and adaptors: ECAL uses solutions already developed and procured for CMS Tracker.

10. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 200310 Components: Transmitter - 1 The transmitter of the data link is the GOH. Its principle components: Some specifications of the GOH: Receives LV, clock, control and 16-bit parallel digital input at 40 Mb/s from FE board via connector. Transmits serialized optical output at 1310 nm, 800 Mb/s, either protocol, via pigtail fiber. Output signal power ~-6dBm, depending on bias levels chosen. (0 dBm = 1 mW)GOH Designed by CERN Microelectronics group. Implemented in 0.25μm CMOS technology employing radiation-tolerant layout practices. Speeds of 0.8 and 1.6 Gb/s. ECAL will run the GOH at 0.8 Gb/s. Two protocols (G-Link and 8b/10b). ECAL will use 8b/10b for data and G-Link for trigger. The Gigabit Optical Link (GOL) serializer and laser driver ASIC The laser diode, made by ST Microelectronics, based on a Mitsubishi die Custom-made for CMS Tracker (linear response), suitable also for digital use in ECAL (rise time consistent with 800 Mb/s operation). Die wafer lots are radiation-qualified (gammas and neutrons) before assembly into laser diodes. Pigtail fiber is attached and calibration data are taken by ST.

11. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 200311 Components: Transmitter - 2 Status of the GOH: The design is complete, prototypes exist. Functions well within the full data link (more on this later). Driven by GOH evaluation board (a modified GOL eval board), gives a clean eye diagram at 800 Mb/s.

12. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 200312 Components: Fiber and Connectors Fiber and connectors are adopted from the CMS Tracker system All specifications consistent with use in ECAL as well (e.g. temperature, rad-hardness, attenuation, safety). ECAL benefits from commercial work, qualification and acceptance test setups already a part of Tracker program. ECAL’s contribution is the in-kind to staffing of acceptance tests. 12 1 96 MU-sMU adapter (Sumitomo) MFS adapter (Diamond) MFS-MPO multi-ribbon cable (Ericsson, Diamond) sMU-MFS fanout (Ericsson, Sumitomo, Diamond)

13. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 200313 Components: Receiver 12-Channel digital NGK Rx COTS device; ECAL tasks are to qualify and procure. Specifications are consistent for use within the full data link, e.g.: wavelength: 1310 nm speed: up to 1.25 Gb/s saturation: -5 dBm sensitivity: -18 dBm Functions well within the link (see following slides). Rx module 12 PIN photo- diode array Digital amp. ASIC 12

14. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 200314 Data Link System Tests - 1 Early system tests involved: GOH driven by GOH evaluation board Rx on Rx evaluation board Deser (G-Link or 8b/10b) on Deser eval board counted Deser word errors (frame errors)… Rx Err Rx Out Scope Clock Generator Counter Attenuator GOH Evaluation Board Rx Evaluation Board Deser Evaluation Board …in this case, to study effect of crosstalk in the Rx Conclusions: Link functions well, BER < 10 -13 should be obtainable Rx is well within sensitivity spec (-18 dBm) Crosstalk costs about 2 dB of the optical power budget ~2 dB Channel under study Noise data generated on neighboring channels

15. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 200315 Data Link System Tests - 2 High = Des “Data Valid” (Four) Trigger-generating pulses High = Des “Rx Err” Four triggers generate four bursts of data. Each trigger involves the absence of one Control clock cycle. Valid data seen by deserializer, except occasionally soon after the missing clock cycle(s). It is expected that this will be resolved when QPLL (CERN MIC) is integrated into Front- end and Off-detector boards. The good news is that FE + Data Link works! First test of the Data Link with the prototype FPGA version of the Front-End board (mid-Feb 03):

16. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 200316 Optical Power Budget Launched from GOH-6 dBm --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Laser diode efficiency 1 dB Fiber and connector interfaces 3 dB Rx crosstalk 2 dB --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Effective signal power at Rx-12 dBm Rx sensitivity limit-18 dBm* *Rx sensitivity spec is “-19 dBm typical, -18 dBm minimum”. We have so far observed better than –20 dBm. => At least 6 dB of margin in the power budget

17. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 200317 GOH Assembly Manufacturing Specifications document for the GOH is complete. We plan to submit the RFQ as soon as possible to have the maximum amount of time to test the first 100 (Engineering Run). GOH assembly involves a number of steps: Wire-bond protective cover attachment Laser diode attachment and wire-bonding Fiber clamp attachment GOL BGA mounting PCB manufacture, passive component mounting

18. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 200318 GOH QA and Link System Tests - 1 Construction of first 100 GOH (Engineering Run) will begin when all of these are ready: RFQ completed and manufacturer selected Laser diodes are available Tested GOL’s are available We plan thorough tests to verify functionality and reliability. The tool for these tests will be our Bit Error Rate Test system. Based on PC, CMS “GIII DAQ” cards and extensive Altera programming. Principle is the same as for the frame error tests: plot BER vs. optical power, make a quantitative measure of a stress on the system in terms of a “penalty” on the optical power budget. Advantages over frame errors are that it is bit-by-bit, independent of protocol. BERT system has been working since beginning of the year. Presently runs at 100 Mb/s: BER of <10 -13 in ~1 day. Present effort is to automate it to perform many measurements with minimal human intervention.

19. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 200319 GOH QA and Link System Tests - 2 The functionality tests we plan include: Temperature sensitivity: GOH, Rx, deserializer. Sensitivity to LV fluctuations/noise. Fine-tuning of the eye diagram via GOH compensation network components. Integration of the Link into the full ECAL electronics chain. The reliability tests planned for the GOH include: Vibration Mechanical shock Thermal cycling (-20°C to +40°C) Pass/fail eye diagram test Irradiation Accelerated aging Long-term stability GOH The goal of the tests is to uncover and resolve any design or manufacturing reliability issues that may exist, and to be able to sign off on the full Link system design by the October ESR. Manufacturer Data Links group

20. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 200320 Schedule Schedule concerns: May 03: If laser diodes or tested GOL are late, time for testing of Eng Run is reduced. Sep 03: If GOH design modification is necessary, little time to test it before the Pilot Run. Qualification is crucial but is the only “compressible” task.

21. Jim Grahl, University of Minnesota ECAL HEP seminar, U. Minnesota, April 29 th 200321 Summary Development of an optical link for a HEP experiment is a fascinating project, invariably involves a lot of co-development with industry. ECAL Data Link system design and project have made a lot of progress in a short time. Starting with good components helps.