Performance of RHIC Vacuum Instrumentation and Control H.C. Hseuh and L.A. Smart Collider-Accelerator Department Brookhaven National Laboratory April 11, 2005 Outline: Description of RHIC Vacuum I & C Performance and Reliability Summary RHIC Vacuum I&C Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland

RHIC Vacuum I&C Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland

BNL Collider-Accelerator Vacuum Systems 3.8 km x 2, <10-10 Torr 150m, 10-8 Torr ~500m, 10-9 Torr 800m, ~1x10-8 Torr 200m, ~2x10-11 Torr ~900m, ~10-10 Torr RHIC Vacuum I&C Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland

RHIC Vacuum Instrumentation (Excludes ~ 800 vacuum devices for the injector chains) ~ 1600 Vacuum Devices w/ remote control and monitoring Beam Vacuum Systems (Warm + Cold) 420 CCGs, 150 TCGs, 15 RGAs w/ E-M 170 SIPs, 320 TSPs 133 Sector Gate Valves Insulating Vacuum Systems 130 CCGs, 230 TCGs, 76 RGAs w/ FC ~ 42 TMPs (with isolation valves, TCG, CCG, …) Control and Monitoring 8 PLCs  4 FEC (VME Power PC)  CLC (Control Level Consoles) ~140 CCG Analog Signals to MADC (for fast logging) RHIC Vacuum I&C Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland

Control Architecture Based on Programmable Logic Controller (PLC) 8 Allen-Bradley PLC 5-40E or 5-20E processors 2 Co-processor + expander sets in each PLC OS-9 real-time operating system RS-485 multi-drop serial interface ports for GC, IPC, RGA, TMP < 5-second update time with ≤ 32 devices at each port Inputs from GC and IPC set points Outputs for valve interlock (thru 2 out of 3 set point voting scheme), Beam Permit Link (any valve closed), RF & Kicker interlocks PC GUI at two locations w/ Data Highway links to all PLCs for diagnostics and downloading programs (independent of CLC) RHIC Vacuum I&C Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland

Control Architecture –cont’d Distributed PLC Control Relieves CLC of low-level tasks - simpler software & hardware Front-end computer (FEC) 4 VME based Power PC processors 100 Mb/s Ethernet links to CLC and PLC co-processors Console Level Consoles (CLC) Control & monitoring via spreadsheets, graphics, alarms Data Logging and LogView Elog for machine operations Elog (e.g. Vacuum Elog) with each subsystem RHIC Vacuum I&C Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland

RHIC Vacuum I&C System with RS-485 Networks Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland

PLC, Co-Processors, I/O Modules GC, IPC, TSP, PLC, PC/GUI, … At service buildings PLC, Co-Processors, I/O Modules TMP w/ GC, SLC w/ RS485 serial link TMP, CCG, TCG & Valves on Cryostats RHIC Vacuum I&C Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland

LogView for diagnostics… e.g. ∆P vs. I & bunch length Most useful applications for Vacuum Spreadsheets Live monitoring and control, open architecture, no password Logging & View from minutes to 720 Hz with synchronized time stamps; compare ∆P with other machine variables with ~msec resolution Vacuum ELog Vacuum maintenance and operation details entered by technicians and staff for diagnostics and archives Graphic Display 2-D, 3-D, waterfall… Bunch Length IR10 Pressure rise IR2 Pressure rise Fill 5160 RHIC Vacuum I&C Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland

Spreadsheet for Turbopump Operation Spreadsheet for RGA Operation RHIC Vacuum I&C Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland

Performance and Reliability of Vacuum PLC and Controls Interfaces 8 PLC, Processors & BPL – very reliable for machine operation, device I/L… PLC and GC are backed by UPS, IPC backed by Generators 14 Co-processors - hung several times a week, but does not affect beam Communication errors? Resulted in no logged data, Remote soft reset (close/open ports) Some required local manual reset Must be checked daily Common software for all subsystems at control console Spreadsheet, Graphic Display, Alarm… LogView allows comparison with other machine parameters Machine ELog and Vacuum ELog – detailed archives for diagnostics RHIC Vacuum I&C Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland

Performance and Reliability of Vacuum Devices GC, CCG - < 3% failure per yr HV cards (thermistors ➱ abnormal readings) CC gauges (neon bulbs, ➱ no ion current) IPC - ~ 5% failed in 2003, less in 2004 HV cards, chassis TMP - In-house designed Station Logic Controller Possible radiation–induced errors ~ once a week (FIV, PIV closed) Some need ring access for reset RGA - Single filament, custom software, 6 masses only ~ 5% filament failure per yr Hardware and software are no longer supported by vendor TSP - In-house design to accommodate long cable run With local step-up (I) transformer, manual operation Remote capable, but not implemented SGV - ~ 5% faulted (momentarily) in 2004, resulted in BPL faults Stuck open due to corrosion in solenoid RHIC Vacuum I&C Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland

Summary Reliability of RHIC Vacuum Instrumentation are much better than other sub-systems over the last five years PLC based control has worked reliably for vacuum and machine protection Coprocessor RS485 interface is susceptible to communication errors Results in lost data Update time of > seconds per port Augmented with analog signals to MADC for fast (msec) logging Identical GC, CCG and TCG for beam and insulating vacuums CCG neon bulb disconnected during bakeout Conversion kit is available but expensive IPC works relatively well Repair delays due to custom products RHIC Vacuum I&C Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland

Considerations for the Future CCG - Buy gauges designed for operation at bakeout temperatures IPC, GC - Buy off-the-shelf controllers to avoid repair delays for custom units PLC - Include analog inputs for fast gauge monitoring TMP – Any commercial turn-key stations with integrated valves and gauges? RGA – Dual filaments Avoid custom software and interfaces TSP - Any commercial units for long cable run (with local transformer?) With remote control? SGV – Corrosion resistant solenoids? RHIC Vacuum I&C Hseuh & Smart April 11-12, 2005 OLAV I, Geneva, Switzerland