1. WBS 2.6.3 Readout Infrastructure Craig Dukes University of Virginia Bill Gilbert University of Minnesota CD2/3a DOE Review of NO  A October 23-25, 2007

2. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.32 Organization 2.6.3.1 Low voltage power supplies/racks 2.6.3.2 High voltage power supplies 2.6.3.3 Power cables/cable trays 2.6.3.4 Power distribution boxes (PDBs) 2.6.3.5 Cooling 2.6.3.6 Shipping 1. Power Distribution System (PDS) (Dukes) 2. APD Thermoelectric Cooler Cooling System (Gilbert) Craig Dukes, Nicole Fields, Emmanuel Munyangabe, Yoshihiro Masui, Andrew Norman University of Virginia Bill Gilbert, Roger Rusak University of Minnesota Leon Mualem Caltech WBS Items

3. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.33 Power Distribution System Feeds electrical power to: –The Avalanche Photodiodes (APDs) –The Thermoelectric Coolers (TECs) –The Front End Boards (FEBs) –The Data Concentrator Modules (DCMs) Requirements –Low noise (3.3V + 450V) –Remote control and monitoring –Floating power supplies –Remote sensing for LV supplies –Reliable operation over 10+ years

4. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.34 Power Distribution System Layout DetectorCAENWienersPDBsFEBs FD16619812,036 ND1410497 IPND126310

5. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.35 PDS Power Requirements Power Delivered per Power Distribution Box ItemVoltageCurrentMax Channels Total Current Total Power FEB3.3 V1 A6464 A260 W TEC24 V0.15 A6410 A232 W DCM24 V1.25 A1 30 W Timing24 V1.04A1/241 A24 W APD450 V 40  A 640.003 A1.35 W FD Power DeliveredTotal 3.3 V51 kW72 kW 24 V52 kW72 kW

6. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.36 Far Detector Di-block Layout

7. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.37 Near Detector Layout

8. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.38 IPND Layout

9. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.39 Electronics Counts

10. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.310 High Voltage Supply CAEN SY1527LC mainframe CAEN A1511B card 12 ch, 0-500 V, 15 mA Individually floating channels Voltage resolution: 100 mV (~0.5%) Ripple: < 30 mV pp Each channel serves 2 PDBs Only 1 mainframe needed for entire FD Ethernet (TCP/IP) interface Widely used at Fermilab

11. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.311 Low Voltage Supply Wiener PL508 System Floating Remote sense 3 3.3V pods, 3 24V pods → each MF feeds 3 PDBs → 66 MFs needed Ripple: < 3 mV pp Ethernet (TCP/IP) interface Widely used at Fermilab

12. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.312 Fans out 3.3V, 24V, and 450V to up to 64 FEB boards, and 24V to 1 DCM 3U crate 16 x 4 + 2 = 66 FEB channels LED indicators for crate 3.3V, 24V, and 450V power LED indicators for card 3.3V and 24V power Front panel on/off switches Fused with TVS for 3.3V lines Note: FEBs have voltage regulators Reference to ground Designed and built at UVa FD: 198 ND: 10 Power Distribution Box

13. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.313 Prototype PDB Fabricated Works according to specifications Minor redesign in progress (mainly to simplify construction)

14. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.314 Mounting the PDS on the Far Detector

15. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.315 Cables Type Size (AWG)Cond. Length (m)Different Lengths # Cables Min.Max.Ave.Total 3.3V222.616.09.31,84212198 24 V1022.415.99.21,82212198 Sense2222.415.99.33,66512396 450 V2227.250.227.62,97549198 PDB-FEB1860.73.82.327,5006412,036 PDB-DCM1821.0 1981 Ground6, 1/0, 3/012.015.04232 All cables “tray cable” Oil resistant jackets Power Supply to PDB

16. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.316 Cable Safety PDB-FEB 4A max Wiener-FEB 24V: 23A max Wiener-FEB 3.3V: 110A max Power supply to PDB cables can handle full current capacity of power supplies.

17. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.317 Power Budget and Voltage Drops Detector Total Total144 kW Output103 kW

18. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.318 Cable Trays

19. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.319 Detector Cooling Overview 17 air breathing 2 ton chillers located on upper walkway Each chiller serves 2 detector blocks Started one at a time to reduce peak electrical and heat load Steady state heat load to air per chiller: Heat from APD/TE modules 3.35w x 744 = 2492w Heat from chiller operation (33% capacity)+ 2310w Net* heat load to air per chiller = 4802w Net* total heat rejected to air by 17 units= 82 kw *Ignores pipe & tube loading, gained from air, rejected to air, approx 2.3 kw per chiller 17 chillers Detector Top View

20. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.320 Water Distribution Scheme prefabricated distribution manifolds prefabricated hose manifolds

21. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.321 Hose Manifold Section 3/4”Sch 120 plastic pipe 1/8” I.D. hoses with valved disconnect With single-sided detector readout, all manifolds are identical, 32 hose style. Two piece construction eases shipping and handling overall length 350 cm Cap compression couplings distribution manifold 1.25” 175 cm

22. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.322 Prototype Hose Manifold Shown with pipe insulation, will be tested with & without insulation

23. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.323 Distribution Manifold Section 1.25” plastic pipe 262 cm (8.6 ft) +/- 3.37cm 131cm116cm 15cm 3/4” pipe taps union couplings ball valves Expansion joint, 6.75cm range, handles exp/contr and dimensional tolerances Return Pipe Section

24. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.324 Water Schematic Far Detector

25. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.325 Water Schematic IPND Each manifold serves 31 APD modules Chiller Sensor Panel 310 APD Modules Total 2 APD rows detector top 3 APD rows detector side 9.8 GPM, 1.04 kW

26. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.326 Water Schematic Near Detector

27. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.327 Water System Pressures Working pressure and worst case pressure is well under design pressure of 100 PSI

28. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.328 Chiller Control & Instrumentation DCS monitors cooling system, can adjust water temp, and start/stop chillers

29. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.329 Flow Network Modeling http://www.inres.com/Products/MacroFlow_Electronic/macroflow_electronic.html Cooling system design is being modeled using:

30. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.330 Hose/Heatsink Net Model

31. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.331 Hose Assy Diff Pressures Drilled orifices in hose barbs are the dominant restriction governing flow Changes in hose length or heatsink resistance will not be significant

32. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.332 Hose Manifold Network Model

33. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.333 Hose Manifold Temp Uniformity Total Spread = 0.15 DegC

34. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.334 Hose Manifold Flow Uniformity Excellent uniformity is due to large relative diameter (0.690 in) of manifold pipe

35. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.335 Distribution Network Model

36. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.336 Distribution Flow Balance Use of 0.040 orifices in hose barbs creates inherent balance Detector Top Detector Side Flow balance will not be disturbed by minor modifications to supply piping Flow can be adjusted using one valve near chiller without affecting balance

37. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.337 Distribution Temp Uniformity Detector Top Detector Side Total Spread = 0.06 DegC

38. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.338 Testing the Cooling Model Two key model elements will be tested: Hose/heatsink assembly flow and pressure characteristics will be measured with various orifice sizes Prototype manifold pair with hose assemblies will be tested for ambient heat gain and temperature uniformity Details: Chiller will provide 15 DegC water at 64 mL/s 4 hose/heatsink assemblies along manifold will be instrumented with thermistors Input & output water temperatures measured with thermistors Overall flow rate measured with rotameter, controlled with valve Input & output pressures measured with transducers Temp & pressure recorded using data logger and computer

39. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.339 Costs (ND+FD) ItemM&SLaborContingencyTotal Low voltage supplies$542K$6K25%$682K High voltage supplies$104K$1K15%$120K Power cables$205K$47K32%$334K PDBs$370K$68K21%$526K Cooling$1,335K$17K31%$1,778K PDS shipping$21K$3K15%$28K Total:$2,578K$142K28%$3,471K

40. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.340 Procurement Power Distribution System –Power supplies to be bought and shipped to Virginia to be tested. –Cables will be cut, terminated and tested at Virginia. –Power Distribution Boxes will be fabricated and tested at Virginia. –Cable trays and relay racks will be delivered directly to the Far Detector site and FNAL. Cooling System –Manifolds will be fabricated and shipped by vendor to the Far Detector site and FNAL on a di-block by di-block basis. –Chillers and cooling interface panels will be ordered and shipped by vendor to Far Detector and FNAL. Installation in WBS 2.9.4

41. October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.341 Schedule Lots of float: no critical-path items! Power Distribution System Fabrication of PDBs: Need 215, including spares. Make and test 1/day with 1 tech 200 days/1 tech Fabrication of cables: Dominated by 12,000 PDB-FEB cables: 1 student: cut, terminate, pack 5 cables/h: 80 days/4 undergrads Will do it in summer of 2010 with student labor UVa HEP building has enough storage for complete system → will deliver to detector in 3 shipments Cooling System Vendors can deliver as needed to keep up with outfitting schedule