Dry runs and cold checkout Markus Albert, Reyes Alemany, Rossano Giachino, Delphine Jacquet, Verena Kain, Belen Salvachua Ferrando, Matteo Solfaroli Camillocci

LS1 modifications Vacuum UPS 18 kV & 3.3 kV circuit breakers Cryogenics P7 enclosure Pumping stationACS transport + personnel changes, including OP/LHC !

22/09/20143 Vacuum UPS 18 kV & 3.3 kV circuit breakers Cryogenics P7 enclosure Pumping stationACS transport Introduction  Very long shut down with modifications at every level  Preparation for beam commissioning will be similar to the LHC start-up in 2008/2009  From this experience, the same strategy is adopted  Intensive individual system tests by the experts are essential  Early test campaigns of operational uses-cases by OP team from the control room (dry runs)  A dedicated machine checkout period (full integration tests) between the end of LS1 and the start of beam commissioning for a smooth transition

System tests scope  System tests from the control room by the operation team with the help of the equipement experts  Not individual equipment tests, but tests of several systems working together  Run of operational scenario  as far as possible without beam  with the systems available  Including the experiments  i.e. operational sequence, interlocks, beam mode, post mortem, logging, timing 22/09/20144

Early start in May 2014  Why so early?  Detection of issues as soon as possible  Provide still plenty of time for corrective action, even for complete review of a system, if needed.  Some stress tests or reliability tests have to be performed during several weeks. (collimators, LBDS, timing)  Drawback  Lots of systems are not stable yet, most of the time only partial tests are possible.  Part of the tests will have to be repeated when the complete systems are ready.  Restart of the injectors is the priority : experts not always available to help and solve issues immediately 22/09/20145

Operational control environment All LSA core applications and services operational. (  new 6.5 TeV hypercycle generation) Sequencer and sequences editor operational Page 1 up and running to communicate with experiments Logging service and timber application 22/09/ Basic controls environment needs to be in place before starting the tests from the control room, i.e. : DIP gateway operational

Examples of tests achieved so far 22/09/20147

Timing  Actual system tested and operational  Single events  Telegrams  Timing tables: edit, load, unload, start  No done: mastership, injection requests  End of October : new timing system will be deployed, tests will have to be repeated. 22/09/20148

RF synchro and Frequency ramp  Launch the frequency ramps with a RF beam control resynchronisation between each ramp. Proton and ions frequency settings for 6.5 TeV ramp  Frequency is properly received by the experiments  Resynchronisation of beam control successful. Frequency monitoring RF FGCs in running state LHC RF ramps down to injection LHC RF sync Next frequency ramp 22/09/20149

Handshakes and beam modes Sequence prepared to mimic the consecutive handshakes and beam mode changes of the nominal sequence. STABLE BEAM mode LHCb velo moving Injection handshake Beam dump handshake 22/09/201410

Send a post mortem even and check – Reception by the experiment via dip – Reception by the PM application, triggering of PM analysis (but not all systems send PM data yet) 22/09/ Post Mortem event This test had to be repeated several time for Alice where 2 problems were detected with database access and BCM logic. The later is solved, but the database access still need investigation Pm application, event received and analysis triggered

Beam dump reliability run  Remote reliability run:  New arming sequence (new interface between beam dump and BIS for retriggering)  With local BIS loop  Energy ramp with BETS simulator  Sequence to arm/ ramp/ dump that has been played in loop for several weeks.  Next step, new reliability run  With global BIS (bridge on all inputs but LBDS)  With a new version of TSU to be tested 22/09/201412

Beam dump reliability run Example of remote reliability run 22/09/201413

Beam dump reliability run Example of remote reliability run Energy simulated by BETS simulator Kicker voltage Dump energy from IPOC 22/09/201414

Collimators  Tests of operational functionalities with available collimators (IR3 and IR7 for the moment)  Settings generation for 6.5TeV, including new collimators  Loading functions for positions and thresholds, reset faults, disarm, start ramp with timing event  Logging functionality  Creation of a sequence for reliability run (parking/injection/ramp) that is running in loop mode. 15 TCSG.B5R3.B1 jaw positions and threshold of as logged in timber during reliability test. Collimator vistar. IR3 and IR7 collimators armed

SMP flags - energy ramp beam modes  Test of some Safe Machine Parameters behaviour during energy ramp, and with change of modes.  The Setup Beam Flag that is the key SMP parameter can not be tested before beam.  Test of the flags distribution to the experiments via DIP 6.5 TeV, beam mode ADJUST 6.5 TeV, beam mode STABLE BEAM 22/09/201416

From today to beam commissioning 22/09/201417

July August September October November December January February March April July August September October November December January February March April P P2P P4P P6P P8 LS1 Schedule V4.1 Pressure test Cool-down Flushing, nQPS, ELQA NC Cool-down ELQA PT  1 PT  2 Cool-down ELQA PT  1 PT  2 NC Flushing, nQPS, ELQA Cool-down ELQA PT  1 PT  2 NC ELQA Flushing, nQPS, ELQA Cool-down ELQA PT  1 NC ELQA Pressure test Flushing, nQPS, ELQA Cool-down ELQA PT  1 PT  2 NC ELQA CSCM PT  1 PT  2 ELQA CV maintenanc e Cool-down ELQA PT  1 NC Flushing, nQPS, ELQA ELQA CV maintenanc e Cool-down CSCM PT  1 PT  2 ELQA Pressure test Machine Check-out Beam Commissioning CSCM PT  2 CSCM PT  2 CSCM From Katy Foraz, presented in LMC the 10 th of September Sector test s23 Sector test s78 22/09/201418

July August September October November December January February March April July August September October November December January February March April P P2P P4P P6P P8 Pressure test Cool-down Flushing, nQPS, ELQA NC Cool-down ELQA PT  1 PT  2 Cool-down ELQA PT  1 PT  2 NC Flushing, nQPS, ELQA Cool-down ELQA PT  1 PT  2 NC ELQA Flushing, nQPS, ELQA Cool-down ELQA PT  1 NC ELQA Pressure test Flushing, nQPS, ELQA Cool-down ELQA PT  1 PT  2 NC ELQA CSCM PT  1 PT  2 ELQA CV maintenanc e Cool-down ELQA PT  1 NC Flushing, nQPS, ELQA ELQA CV maintenanc e Cool-down CSCM PT  1 PT  2 ELQA Pressure test Machine Check-out Beam Commissioning CSCM PT  2 CSCM PT  2 CSCM From Katy Foraz, presented in LMC the 10 th of September Sector test s23 Continue the collaboration with the equipment’s group to Organise system tests from the control room, with more and more systems available Prepare the transfer Line tests and sector tests planned now end of week 6 and end of week 8 (Reyes’ presentation) Establish a planning for the dedicated machine check-out period Sector test s78 22/09/ Before dedicated Checkout

Continuous Interlock systems tests  All BIS input will have to be tested one by one  Almost 200 BIS inputs : PIC, FMCM, vacuum, collimators, experimental magnets, BPMs, SIS…  Tests organised following the readiness of the systems  Vacuum interlock preliminary tests foreseen end of September  Huge and systematic work, that is essential to ensure the machine protection before beam can be injected.  Software Interlock system : less critical but nevertheless essential, every inputs logic need to be checked as well. 22/09/201420

 RF : SPS rephasing (LHC frequency received by the SPS RF rephasing process). ADT and RF control (CCC applications and sequencer) Collimator tests and reliability run to be continued: - Injection protection, transfer lines - New BPM collimators (TCTP) - Handling of critical parameters and settings and general settings management - Final tuning of sequences and tasks  Instrumentation : acquisition, logging, sequencer tasks (BPM, BLM, BTV etc…). Gradually coming together, the necessary tools will be ready for the transfer line tests in November. Should be fully ready beginning of  Orbit and tune feedback: new implementation to be tested 22/09/ Planned systems tests

 Kickers: MKI (pre-pulse, dynamic destination, AG keeper etc…) MKQ and AC dipole with multi-turn application.  Beam dump reliability run to be continued  New timing system (end of October) check again timing events and timing tables, mastership, injection requests dynamic destination. …… 22/09/ Planned systems tests Close collaboration between OP and equipment experts become more and more important. Regular meetings will be organised for tests preparation and their follow-up. Close collaboration between OP and equipment experts become more and more important. Regular meetings will be organised for tests preparation and their follow-up.

Tests of the access system  After LS1 modifications: commissioning and tests necessary, while the access system needs to be available non-stop  Commissioning campaign already done in June and July  5 dedicated Friday, 30 participants in total  Verification of all the I/O signals  Test of the new access powering interlocks :  Access system secured for powering phase 1 and 2 22/09/ software interlocks that prevent the powering of magnets above a current limit when access conditions are unsafe (according to the new rules)

Tests of the access system  2 week-ends are needed to complete the access system tests (machine closed)  First week-end : access system test by GS/ASE  Test the beam mode (BIW sirens, redundant cable loop)  Commissioning of the new maintenance doors (allowing access devices maintenance while in beam mode)  2 nd week-end : DSO tests  Independent verification of the access system by the DSO  Validation that the access system is operational (ensure the personal protection and safety) : mandatory to allow beam in the machine  Scheduled dates : 8-9 November and November  After these tests the access system will be ready for beam 22/09/201424

July August September October November December January February March April July August September October November December January February March April P P2P P4P P6P P8 Pressure test Cool-down Flushing, nQPS, ELQA NC Cool-down ELQA PT  1 PT  2 Cool-down ELQA PT  1 PT  2 NC Flushing, nQPS, ELQA Cool-down ELQA PT  1 PT  2 NC ELQA Flushing, nQPS, ELQA Cool-down ELQA PT  1 NC ELQA Pressure test Flushing, nQPS, ELQA Cool-down ELQA PT  1 PT  2 NC ELQA CSCM PT  1 PT  2 ELQA CV maintenanc e Cool-down ELQA PT  1 NC Flushing, nQPS, ELQA ELQA CV maintenanc e Cool-down CSCM PT  1 PT  2 ELQA Pressure test Machine Check-out Beam Commissioning CSCM PT  2 CSCM PT  2 CSCM From Katy Foraz, presented in LMC the 10 th of September Dead-line for the equipment to be ready for operation is before the first week of machine checkout (week 7) Sector test s23 Sector test s78 22/09/ End of LS1

July August September October November December January February March April July August September October November December January February March April P P2P P4P P6P P8 Pressure test Cool-down Flushing, nQPS, ELQA NC Cool-down ELQA PT  1 PT  2 Cool-down ELQA PT  1 PT  2 NC Flushing, nQPS, ELQA Cool-down ELQA PT  1 PT  2 NC ELQA Flushing, nQPS, ELQA Cool-down ELQA PT  1 NC ELQA Pressure test Flushing, nQPS, ELQA Cool-down ELQA PT  1 PT  2 NC ELQA CSCM PT  1 PT  2 ELQA CV maintenanc e Cool-down ELQA PT  1 NC Flushing, nQPS, ELQA ELQA CV maintenanc e Cool-down CSCM PT  1 PT  2 ELQA Pressure test Machine Check-out Beam Commissioning CSCM PT  2 CSCM PT  2 CSCM From Katy Foraz, presented in LMC the 10 th of September Machine checkout in parallel with HW commissioning, to be taken into account for the planning Sector test s23 Sector test s78 22/09/ Final Machine Checkout

Final machine checkout  Pre -conditions  Machine closed, access system ready for beam.  All systems are operational, mainly  All magnet circuits  PIC and QPS  Beam vacuum system  BIS  Organization  Coordinated by Rossano Giachino and Markus Albert  Daily meeting in CCC to organize access needs and adjust the test plan for the day  Operation’s team on shift day/night/week-ends 22/09/201427

 Full integration tests : first opportunity to run the entire LHC systems together and run the tests that require the machine to be fully closed. i.e :  Heat run of warm magnets (already tested and polarity checked)  Test of complete LHC cycle (pre-cycle, injection, ramp up/down) with all the operational magnets  Complete the machine protection tests (all BIS inputs)  Close beam permit loop with all BIS inputs connected  Final LBDS checks : energy tracking tests under real configuration  Final MKI tests with full operational conditions 22/09/ Final machine checkout LHC ready for beam!

Conclusion  Aside individual system tests, the operations team has already organised various tests from the CCC with the equipment experts and experiments.  Aim is to tests systems as early as possible to anticipate on software bugs, hardware issues etc…  Tight collaboration between OP and the equipment specialists for the organisation and follow-up of tests as more and more tests will be planned while approaching the end of LS1  The readiness deadline for all equipment and controls is the start of the machine checkout period, beginning in week 7.  Full operational conditions with machine closed are needed for the last checkout test period that will smoothly lead to beam commissioning. 22/09/201429