FESS/Ops Shutdown Summary Nov 9 th, 2012
CUB Status – Boiler #1 ON Primary Recirculating Pump repaired Second Belly Pump to be Wired – Boiler #2 OFF – CUB Cooling Towers- Fill and Fan Sections Need Repair or Replacement, $43,000/Cell to Repair, $80,000 to Replace
CUB Status (Cont.) Cooling tower #7 Leaking at Seams, Contractor called in to assess. Cooling Tower Pump #2 (300 Hp., 8,260 GPM) – VFD rebuilt and Control card replaced, on line
CUB Cooling Outage/WH Fdr 44 Maintenance November 4 th,2012 – Installation of Barriers in the CUB Chiller 4160 Volt Starter Cabinet Completed This will now allow replacement of 4160 X 120 Volt Transformers without shutting down the entire 4160 Cabinet, Chillers and Experiments. – CUB cooling outage affected Wilson Hall, XG/LinAcc/Footprint Area, MI60, and MI65.
CUB Cooling Outage, (Cont.) – CUB Cooling Outage coordinated with the Annual Wilson Hall (FDR 44) Power Outage for Electrical Maintenance Scheduled for Nov. 4 th, 2012 Wilson Hall was without Power (Except for Emergency Generator Loads) and without Primary Cooling CD 5 th Floor and 8 th Floor computing rooms were down The XG/LinAcc/Footprint Area, MI60, and MI65 were without Primary Cooling. Mac Computer Room and XG Firus Room were provided with Spot Coolers The WH Comm. Center and Telecomm. Switch Rooms were provided Cooling from Emergency Generators and Spot Coolers
November 3 rd and 4 th 2012 FDR 40/FDR 44 and CUB Cooling Outages Feeder 40 – LINAC and X-Gallery X-Gallery Roof Drain Feeder 44 – Wilson Hall CUB Cooling Chiller Water Outage – 4160 Volt Isolation Barrier Installation
Feeder 40 November 3 rd Electrical Outage – Findings Lower LINAC Area Substation Y-L1 High Voltage Switch Failed to Operate Properly – CORRECTIVE: lubrication and exercising the switch 480 Volt Main Breaker Extremely hard to rack in/out– CORRECTIVE: lubrication and exercising the racking mechanism
Feeder 40 November 3 rd Electrical Outage – Findings Lower LINAC Area DHP-L2-2 (Asset L-128) 350 Amp Breaker – Breaker Handle Broke When Operating CORRECTIVE: Replaced handle
LINAC Lower Level 350 Amp Breaker Handle Replacement
LINAC North Stairway Lower Level Sump Emergency Pumping for Electrical Outage
X-Gallery Computer Room Roof Drain Piping ~ 40 years old, Extra Heavy Duty Cast Iron Piping, (note wall thickness)
X-Gallery Computer Room – Roof Drain Piping –Bad Lead Joint Cause of Failure
X-Gallery Computer Room Roof Drain Piping, Bad Lead Joint
CUB 4160 Started Cabinet - Exposed Vertical Bus
CUB 4160 Bus Barrier (before modification/additional Bus Protection) Initial installation design - This Portion of Bus left unprotected (Modification need)
CUB 4160 Bus w/additional Bus Protection at Top - Final Design Additional Barrier Fabricated Ventilation areas
CUB 4160 Cabinet Bus Barriers w/final modifications completed
Feeder 44 Outage - WH Basement Switchgear - High Voltage Section PM
Feeder 44 Outage – WH Basement Switchgear 480 Volt Breaker Inspections – Total of 16 Breakers Inspected / Cleaned / Tested (Replaced 3 Trip Devices – Found Defective during trip current testing)
Feeder 44 Outage – WH Basement Switchgear 480 Volt Breaker Inspections – Total of 16 Breakers Inspected / Cleaned / Tested
Feeder 44 Outage - WH Basement Switchgear WH Bsmt Xformer – 13.8 Kv 480 Volt (one of two transformers within)
Flex Bus Damaged by Transformer Failure (addition cable added for ampacity) ~1988?
Feeder 44 Outage - WH Basement Switchgear – Typical 480 Volt Cabling in back of Switchgear
Feeder 44 Outage - WH Basement MCC Thermal Imaging Report – AC1W MCC Main Breaker “A” Phase
Feeder 44 Outage - WH Basement MCC Thermal Imaging Report – Main Breaker Removed Due to “A” Phase Lug HOT
AC1 West Breaker - Broken Lug Bolt
No Head Markings on AC1 West Breaker Lug Bolt
Feeder 44 Outage – Exterior S&C 4 Bay Switch PM – DSTR-A02
Snakes Head Snakes Juice
S&C Switch Snake Phased Out!
S&C Switch Snake (under-rated conductor)
Mechanical – F3 Manhole- Sump Replacement, parts received, to be scheduled during Shutdown – PV07- MI60 Pump Vault 7, 150 Hp. Pond Pump Suction Valve won’t close, requires shutdown of both Pumps to repair Pump, replace valve, (need schedule for repair).
Mechanical (Cont.) – Site Wide Valve Exercising, V.I.P.E.R. Roll-out Annual Valve Exercising required by NFPA 25 Valve Intuitive Powered Equipped Rover (V.I.P.E.R.) Purchased, Software, Exercising Procedure and Commissioning Completed Domestic Water Valve Exercising in Village has been completed, Main Site WIP
Mechanical, (Cont.) Site-Wide ICW Flush Completed, Live Zebra Mussels Found at following locations: – IHFP-612Across from IARC/OTE – IHFP-107AP10 – IHFP-109AP30 – IHFP-110AP50 – IHFP-111AP0 – IHFP-185KRS – IHFP-203F0 – IHFP-041B0 – IHFP-113NE Wilson Hall – IHFP-106S Corner XG and TRG – IHFP-216D0 Outback
Tritium In Sewers (cont.) Tritium in the Sewers Source Identified as CUB Softener Sampling Discharges, (further testing required to verify as sole source).
Mechanical (Cont.) RHC5 XG Operators Room (Light Side) Reheat Coil #5 supplies 2 Registers to Operators Room and Abandoned ductwork above XG Computer Room, (at least 1 supply uncapped and open above dropped ceiling) RHC5 is currently held closed by having main air applied directly to the valve RHC5 could be re-enabled through installation of a pneumatic Thermostat locate in the Operators Room Since the Trunk line from AC2 is mostly blanked off above the Computer Room Ceiling, a solution to the Operators Area would be a re-route of this ductwork.
High Voltage/ Beam-line Maintenance Work – PM Lab A North Area, Labs A,B,C,D,E,F,G Affects NMS/Cryostat Bldg. Reschedule for March 2013
HVAC Site Wide Boiler Inspections, Nearing Completion CMTF HVAC Unit Warranty Repairs Completed – Gas Line relocation WIP (to allow access to Ctrl. Panel)
FESS/Ops Engineering NTSB Storm Water Pump Piping Completed CUB 95 Degree Booster LCW pump #3 Replacement, (Completed for 2 nd time) CUB 95 LCW System Filled, Circulating ~50% MI30 ICW Break at PIV44, WIP – ICW Off to MI20, 30, 31, 39, 40 AP10 ICW Break between PIV100 and 102 at MuCool, section of ICW Off, No Bldgs. affected
MI30 PIV44 Excavation
MI31 Sink Hole To Be Scheduled