Overview of CERN BIS and Communication Hardware Raffaello Secondo On behalf of the CERN TE-MPE-MI section 3rd October 2017 CERN – ESS – ZHAW Meeting

Outline CERN BIS Concept and Specification BIS Architecture and Hardware LHC Beam Interlock System BIC Hardware Layout Optical Link modules User Interface Hardware and Communication Beam Interlock System External Review 2009 2 of 12

CERN BIS Overview and Specification Ensures beam is properly extracted if one of the connected systems detects a failure. Mean Time Between Unsafe Failures equivalent to SIL3. Reliability: < 1 missing dump over 1000 years. Availability: < 1% chance of all LHC missions prematurely aborted due to failure (1 LHC mission = 10 hours operation). LHC BIS System reaction time within 100 µs over 27 km. In operation for the LHC, SPS (ring architecture) and their injection/extraction lines (tree architecture). In operation since 2004 (in the SPS). OPERATIONAL EXPERIENCE No Missed Dumps since operation, not even on a single beam loop. Availability: ~2 false dumps triggered due to optical link degradation over ~10 years LHC BIS operation. Beam Interlock System External Review 2009 3 of 12

LHC Interlock Systems and Inputs Approx. 200 USER SYSTEMS connections LHC Devices LHC Devices LHC Devices Movable Devices BCM Beam Loss Experimental Magnets Collimator Positions Environmental parameters BTV screens Mirrors Timing SEQ via GMT SMP Software Interlocks CCC Operator Buttons Experiments Transverse Feedback Beam Aperture Kickers Collimation System FBCM Lifetime BTV MKI Beam Dumping System Beam Interlock System (distributed system over 27 km) Setup Beam Flag 32 8 12 Injection BIS PIC essential + auxiliary circuits WIC FMCM RF System BLM BPM in IR6 Access System Vacuum System Timing System (PM) Magnets Power Converters Doors EIS Vacuum Valves (~300) Access Safety Blocks RF /e-Stoppers Monitors aperture limits (some 100) Monitors in arcs (several 1000) QPS (several 10000) Power Converters ~1800 AUG UPS Cryo OK

Beam Interlock System Concept USER PERMITS BEAM PERMITS 1 User Systems 14 User Interface (CIBU) Beam Interlock Controller TARGET SYSTEM (BEAM DUMP) 14 <4m Current Loops SOFTWARE PERMIT (SIS) Single-Mode Optical Fibers <1200 m RS-485 or Fiber Optics ∑(USER PERMITS = TRUE)  BEAM PERMIT = TRUE BIC: main controller, acts as concentrator collecting USER PERMITS, generating BEAM PERMIT BICs can be linked as TREE or RING architectures

Outline CERN BIS Concept and Specification BIS Architecture and Hardware LHC Beam Interlock System BIC Hardware Layout Optical Link modules User Interface Hardware and Communication Beam Interlock System External Review 2009 6 of 12

Scalability and Architecture RING Architecture TREE Architecture Each BIC shares the redundant beam permit loop Master and Slave BICs MATRIX (AND + OR) Local BEAM PERMITs transmitted to Master via copper links Courtesy of B. Puccio and B. Todd

BIS in the LHC - I IR6 4 Fibre-Optic channels, 2 for each beam, Beam Interlock Controllers (BIC) IR6 Beam Dump Beam-1 and Beam-2 4 Fibre-Optic channels, 2 for each beam, 1 clockwise + 1 anti-clockwise for each beam. Square wave generator located at IP6 Signal can be cut by any BIC Signal is monitored by every BIC. 16 BICs per beam, 2 at each insertion point Beam 1 and Beam 2 are independent.

BIS in the LHC - II BEAM PERMIT LOOPS: redundant, “TRUE” signal is a frequency ~10 MHz with 50% duty cycle. BICs: gather USER PERMITS, generate LOCAL PERMITS. When LOCAL PERMIT is “FALSE”, BEAM PERMIT loop is interrupted and becomes “FALSE”. When BEAM PERMIT is “FALSE” the detector, located at the BEAM DUMP, triggers a dump. User Permits Local Permit BIC BIC BIC Beam Permit

BIS Hardware Simplified Layout Courtesy of B. Puccio and S. Gabourin JAVA Application CIBU User Permit Technical network (Ethernet) VME chassis User System #1 #1 FESA class User System #2 #2 copper cables or fiber optics links Rear Front User System #14 #14 BIC CIBG MenA20 CIBM CIBT CIBM: performs local AND, log history, can open the BEAM PERMIT loop. CIBT: remote testing of User Interfaces, gives Beam Info

Optical daughter cards Hardware Modules User Side Controller Side x2 Optical daughter cards CIBO Redundant Power Supply Manager CIBM Test & Monitoring CIBT CIBU variant with Fibre Optics Back Panel (Burndy connectors) Courtesy of B. Puccio

Differential I/O Interface CIBM – Manager Board Matrix A Matrix B Monitor VME BUS Differential I/O Interface JUMPERS A JUMPERS B Max3440 transceivers for each user input. Signals polarity fail-safe. 2 CPLDs: Xilinx XC95288XL for critical code. 1 FPGA: Xilinx Spartan3 for monitoring VME interface. Each channel can be disabled by hardware  2*14 jumpers Up to 7 channels can be masked by software command. Courtesy of B. Todd and S. Gabourin

CIBT – Test & Monitoring Board Differential I/O Interface Test & Monitor FPGA Display Control FPGA Max3440 transceivers for each user interface input/output. NO VME connection. 1 FPGA (Spartan 2) to test/monitor the User Interfaces. Not redundant. RS485 + Manchester encoding. 1 FPGA (Spartan 2) to control the Display.

CIBT Functions 1) Test/Monitor the User Interfaces CIBM writes a test buffer to CIBT CIBT transmits info to all Users Users sends a “monitoring” buffer to CIBT CIBT saves data and transmits it to CIBM Monitoring data stored and sent to supervision 2) Provide BEAM PERMIT INFO to Users Allow Users to know if there is beam in the machine Non critical

Optical Link Beam Permit Loops (CIBO): Fibre-Optics: Long distance link Immune to electromagnetic interference EMI. Single-mode fibre, 1310 nm window: low attenuation, zero dispersion. CIBO Beam Permit Loops (CIBO): Single-mode ELED transmitter, PIN diode receiver. Sends and receives a 10 MHz square signal. Interface signals are 5V TTL. Signal sent along 27 km LHC, 6 km SPS, PSB and transfer lines. Optical Interface for Users (CIBF): Laser transmitter, PIN diode receiver, mounted on CIBL Low Speed: 62.5 Kbps Only links > 1.2 km. CIBL  More information in the following presentation by Christophe. Courtesy of C. Martin, C. Garcia-Argos, S. Gabourin

Outline CERN BIS Concept and Specification BIS Architecture and Hardware LHC Beam Interlock System BIC Hardware Layout Optical Link modules User Interface Hardware and Communication Beam Interlock System External Review 2009

The BIS User Interface - I Installed in the User System rack. The USER_PERMIT input voltage (3V to 24V) is converted inside the CIBU in a “current loop”  reliable, EMC tolerant. Unique HW solution to connect any User System via a copper cable. The connection CIBU-BIC realised through RS485 link Courtesy of C. Martin 17

The BIS User Interface - II Front view Rear view User System (PLC based, VME based, etc…) USER PERMIT state transmitted with RS485 physical standard to BIC. USER PERMIT =“FALSE” if Input current <~10mA Courtesy of C. Martin

The BIS User Interface - III SAFETY CRITICAL USER PERMIT Fast response time (2.6 µs time) Large Input Voltage 3V-24V Hysteresis: clean signal edges RS485 output, good EMC, long distance No programmable logic in the critical path NON SAFETY CRITICAL BEAM INFO RS485 Input Manchester Encoded Courtesy of C. Martin, B. Todd

Conclusions BIS Concept, Hardware and Communication Fast, Safe, Reliable: 100 µs over 27 km Fully redundant system CRITICAL Hardware physically separated from non-critical hardware. Initially designed for LHC, highly scalable and adaptable Standardized User Interface (CIBU) Proven solution: provides safety and improves beam operation efficiency.

Thanks for the attention

BACKUP Slides

Signals with redundancy and feedback to users

LHC BIS Reaction Time

CIBIT Board The CIBIT recuperates the Local Permit of a CIBM to transmit it to the Target Systems by differential (2 wires) -> It becomes then the Beam Permit It is configured for the LEFT CIBM or Right CIBM It can send up to 4 Beam Permit to feed up to 4 Target Systems Each Beam Permit is duplicated (A and B) and send to the Target System by 1 cable Beam Permit A and Beam Permit B have each a monitoring included in the cable (2 more wires for the read back).

VME crate Extenders Signal paths between VME-P2 and Backpanels 3U, 8 layer PCB, electrically tested Avoids cabling errors No functional test, visual inspection only Excellent EMC properties Beam-1 & Beam-2 on individual Extenders Many ground planes CIBEA - 3x32pin connector CIBEB - 3x10pin connector