LHC Magnet Tests: A challenge for an OperationTeam V. Chohan G.H. Hemelsoet CernGeneva Always set the trail, never follow the path

Outline Introduction to SM18 & Magnets Tests Facility Introduction to SM18 & Magnets Tests Facility Operational Techniques Operational Techniques The Challenge The Challenge The Actors The Actors Some tools set up by operation Some tools set up by operation –Web servers –Magnet Training Criteria –Priority Handling –Online and Real Time reports Conclusions Conclusions

SM18 Tests Limited Infrastructure Limited Infrastructure Heterogeneous Team Heterogeneous Team –Staff and outside contractors –Core of test manpower was a collaboration from India Tight schedule to test 1706 magnets Tight schedule to test 1706 magnets –In fact more than 1900 tests (with spares and repeats)

Magnets tested Current / Field Required for LHC Total Tested Dipoles A/ 8.33T SSS A 233 T/M IR SSS Magnets tested on time

Cumulative tested magnets Early Performance With Operational Techniques

Staffing: Various Team of SM18 Magnet Tests Team OP Cryogenic Team Magnet Connect/Disconnect Team Magnet Transport Team Mainly associates from DAE India Industrial Consortia Equipment support

Magnet Tests.. What do we do ? New magnet arrives.. Insulation test, connection to CFB Preparatory tests (functioning of protection systems & instrumentation, insulation…) Preparatory tests (tuning DAQ..) Insulation tests Functioning of protection systems Power tests (&Training) Magnetic measurements Final insulation Cool down to 1.9K Final warm tests Electrical integrity Resistance measurements.. Final insulation tests Disconnection from CFB Warm –up

Test Facility in SM18 12 test benches in SM18 12 test benches in SM18 Roughly speaking: Roughly speaking: –3 ‘ k –3 in warm-up or cool-down phase –3 in connection or disconnection phase –3 spare &/or dedicated to special use –6 power supplies ( 1 per 2 benches) Dipoles as well as SSS ’ s Testing Dipoles as well as SSS ’ s Testing C2 C1 B2 B1 A2 A1 F2 F1 E2 E1 D2 D1

Web Server (SMTMS) Maintains electronic repository of tests results Maintains electronic repository of tests results Links tests results with management tools Links tests results with management tools Tracks test progress & test statistics Tracks test progress & test statistics Automatic generation of test reports Automatic generation of test reports Automatic filling of the tests done Automatic filling of the tests done

E-Traveller Web based tool with mobile phone interface and video display – alert teams Web based tool with mobile phone interface and video display – alert teams Keeps track of electronic, signature based information exchange Keeps track of electronic, signature based information exchange Automatic time stamped log of tests activities Automatic time stamped log of tests activities Minimizes verbal communication.. Minimizes verbal communication..

Real Time Information Real time status of the facility was displayed on the web and on TV screens around the CERN Real time status of the facility was displayed on the web and on TV screens around the CERN LHC Dashboard

Training Rules Earlier, all magnets were trained to ultimate field (12850A ~9T) Earlier, all magnets were trained to ultimate field (12850A ~9T) 3-4 hours recovery time between quenches, Time consuming… 3-4 hours recovery time between quenches, Time consuming… But… Most of the ‘good magnets’ cross nominal field (11850A ~ 8.33T) in the second training ramp.. But… Most of the ‘good magnets’ cross nominal field (11850A ~ 8.33T) in the second training ramp.. [ year 2003 Observation by OP Team] [ year 2003 Observation by OP Team] Two-Quench Rule: Accept the magnet if it crosses nominal field in Second training with small margin! Two-Quench Rule: Accept the magnet if it crosses nominal field in Second training with small margin! Three-Quench Rule: Accept the magnet if it crosses A (~8.66 T) in third training, even if it had not passed the preceding rule.. Three-Quench Rule: Accept the magnet if it crosses A (~8.66 T) in third training, even if it had not passed the preceding rule.. 80% magnets cross nominal in 2 nd ramp!

Magnet Appraisal & Performance Sheet (MAPS) Single page report on rule based magnet goodness evaluation Single page report on rule based magnet goodness evaluation OP Team empowerment for rapid, round the clock decision taking on first level of goodness evaluation OP Team empowerment for rapid, round the clock decision taking on first level of goodness evaluation Mitigates issues of magnet storage logistics Mitigates issues of magnet storage logistics

Overall & Cryo Priority Setting Ensure that magnets follow the desired cryogenic phase distribution, within the limits of possibilities due to the varying training performance ( a priori not known) Ensure that magnets follow the desired cryogenic phase distribution, within the limits of possibilities due to the varying training performance ( a priori not known)

Rapid On Bench Thermal Cycle (ROBTC) Earlier, ‘poor magnets’ were disconnected from bench, fitted with anticryostats, tested again at a later date… Earlier, ‘poor magnets’ were disconnected from bench, fitted with anticryostats, tested again at a later date… Additional disconnection, connection, preparatory tests… time consuming.. Additional disconnection, connection, preparatory tests… time consuming.. But.. Quench performance improves after thermal cycle But.. Quench performance improves after thermal cycle Rapid On-Bench Thermal Cycle (ROBTC): Rapid thermal cycle of a poor magnet without disconnection from test bench Rapid On-Bench Thermal Cycle (ROBTC): Rapid thermal cycle of a poor magnet without disconnection from test bench

Prototype vs Operation Prototype facility Prototype facility –Try to push every magnet at the limit –Repair or improve as they have time Operation not actively involved in Software and design of facility Operation not actively involved in Software and design of facility –Each group has its own idea and implements its way –Culture of keeping control and hiding things Running accelerator Running accelerator –Try to push the whole complex at the limit –Fast and coordinated repair and improvement Operation is central about design and software development Operation is central about design and software development –Advice of operation before implementation –Open environment with trust of operation

Operation know-how Before 2002 Before 2002 –All problems treated the same way No clear distinction between magnet problem and equipment problem No clear distinction between magnet problem and equipment problem –Poor global view Each magnet has its own Excel files with redundant information Each magnet has its own Excel files with redundant information After 2003 After 2003 –Separation for magnet and bench problems E-log book for Equipment Database for magnets –Global web generated view One data base for all magnet information and an automatically generated Magnet and Test Result paper logbook.

Conclusions The tools developed by Operation helped a lot to finish the magnet tests on time The tools developed by Operation helped a lot to finish the magnet tests on time –Identify bottlenecks To do list To do list Statistics Statistics –Real push from the team to gain time Change of the rules Change of the rules Tracking of down time Tracking of down time –Real time Management of the facility Priorities Priorities Web Display Web Display Whether we play a large or small role, by working together we achieve our objectives.

Foreseen Times Standard time Standard time –12 hrs connection –12 hours vacuum pumping –26 hours cooldown –36 hours cold test –12 hours warm up –12 hours disconnection Total 108 hours per magnet Total 108 hours per magnet –(4.5 days of 24 hours)

Ideal situation To ensure a maximal usage of the facility, a cog-wheeling strategy has to be set up. To ensure a maximal usage of the facility, a cog-wheeling strategy has to be set up.

In Reality …

Operational Techniques Web based database Web based database –SM18 Test Management System (SMTMS) Links electronic repository of tests results with other management tools Links electronic repository of tests results with other management tools Provide real time view of the facility to identify bottlenecks Provide real time view of the facility to identify bottlenecks –E-traveller Web based, electronic signature based information exchange tool Web based, electronic signature based information exchange tool Web Server SM 18 control Room SM 18 Hall Offices Status

Operational Techniques Improved Training rules Improved Training rules –Use the last quench of a magnet to help the warming up –Modified Magnet Training Criteria Two-Quench Rule Two-Quench Rule Three Quench Rule Three Quench Rule –Rapid On-Bench Thermal Cycle (ROBTC) Thermal cycle of a poor magnet without disconnection from test bench Thermal cycle of a poor magnet without disconnection from test bench Time & space optimization Time & space optimization –Overall & Cryo priority handling 24 hour priority decision by OP team for effective & clash-free utilization of ALL resources ( Cryo, Power, bench etc) 24 hour priority decision by OP team for effective & clash-free utilization of ALL resources ( Cryo, Power, bench etc) –MAPS Single page record of ‘rule based magnet goodness evaluation’ Single page record of ‘rule based magnet goodness evaluation’