1. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire CO 2 IBL plant failures 16/06/2016 20/06/2016 O.Crespo-Lopez

2. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire Status The plant was running stable on the system B since 27 th of January Last year we did several swap and recovery completely transparent for the Detector Last year we were running @ low temp and today we are @ 5°C O.Crespo-Lopez

3. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire Why have we decided to swap O.Crespo-Lopez Since ~13 th of June the DP over the plant B filter started to increase faster than before – Related to the temperature change ??? Warning is @6bars & the alarm @ 10 bars Filter is send for particles analysis Outlet from the plant

4. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire If filter gets clogged what’s going to happen If the dP over the filter reach 10 bars then the plant B stops and we automatically swap to the plant A. – we didn’t know when it could happen – PIXEL team has been informed. We agreed it is better to provoke plant swap instead of waiting for trip. The automatic trip could possibly happen in the middle of the data taking during the night. The consequences would be more severe than controlled swap with experts supervision. O.Crespo-Lopez

5. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire Failure of the 16 th of June @ 10h00 after a call to the detector expert we have triggered the manual plant swap – System follows automatic swap procedure – Plant B in operation goes to STOP (means not ready for operation) – Plant A in stand-by goes to OPERATION @ 10h01 the chiller A started and after some seconds it stopped O.Crespo-Lopez 46s 1 2 3 4 1/ start of the system A 2/ swap from water to air cool condenser 3/ compressor speed increase 4/ trip due to high pressure alarm

6. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire Alarms The chiller fails on high pressure alarm – How it works O.Crespo-Lopez Chiller start on chilled water If PT402 > 19 bar then swap on air cool condenser Manual reset If PT402 > 20 bar then stop of the chiller If chiller stops then plant stops As the second plant was in stop we stopped the whole system

7. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire Chiller HP alarm during start-up Chiller high pressure alarm during start-up – Several times we have seen that when we start a chiller after a long period of stop we get higher pressure drop between the outlet of compressor and the outlet of the condenser but never so important comparing to the last events O.Crespo-Lopez 4.4 bar during startup Less than 1 bar in Run

8. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire How to avoid Proposals: – 1 st increase the alarms threshold Current values: – HP Warning: 18 bara – swap from water to air: 19 bara – HP Alarm, Stop of the system: 20 bara – HP switch (hardware): ~24 bara Proposal values: – HP warning: 18 bara – swap from water to air: 20 bara – HP Alarm, Stop of the system: 21 bara – HP switch (hardware): ~24 bara O.Crespo-Lopez Need to be tested during the next TS, but no PLC modifications needed

9. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire How to avoid Proposals: – 2 nd change sensor for HP alarm Currently we are looking on PT402 for all the warning and alarms The idea is to move the alarm for the swap and the one for the full stop to PT408 O.Crespo-Lopez Due to the significant pressure drop we have during chiller start-up it could solve the problem Request PLC modifications, could be tested during TS2

10. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire How to avoid Proposals: – 3 rd start with hot gas bypass open We think it will reduce the pressure drop during start-up (to be confirm) O.Crespo-Lopez Request PLC modifications, it has to be tested manually maybe during TS2 before to modify the PLC program

11. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire Failure of the 16 th of June Due to HP alarm system A tripped – Stop of the detector HV&LV due to DSS Alarms @10:05:10 system A restarted by expert @10:05:45 system A stopped again on (no flow alarm) O.Crespo-Lopez If dP over the detector (measured inside the junction box sector 5) less than 1.5 bara during more than 5 seconds => STOP the system

12. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire Failure of the 16 th of June @ 10:07 the plant has been restarted After some second we got again no flow alarm @10h08 the plant has been restarted with no flow alarm blocked by expert for few seconds O.Crespo-Lopez

13. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire Why Before to start plant & chiller A this one was cooled by chiller B through little HX – Chiller B was running with 2 bar suction pressure means evaporation temp @ -30.5 – The liquid CO 2 before the pump was @ -28.2° In the PLC after a stop of the plant we go through the stepper and there is one condition If this one is not fulfilled then the valve PV110 still close O.Crespo-Lopez

14. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire Why In this case the pressure is the same that the accumulator in all the lines and we start to evaporate the CO 2 – Means when we re-open the valve we need time to retrieve the flow After 5s if the DP over the detector is less than 1.5bar we consider there is not enough flow for the detector Cause – Chiller setpoint @ 2bar (- 30.5°C) then not enough subcooling before the pump and system blocked in a transition T3 O.Crespo-Lopez

15. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire How to avoid Keep the chiller setpoint @ 1.2 bara suction pressure setpoint (already done) As an improvement we can modify the logic of the process: – Currently if after 5s the DP over the detector is < @1.5 bar then we stop the plant It’s the good way when the second plant is available But when only one plant is available we have to let this one running in any case instead of stop this one on no flow alarm (detector protected by DSS) O.Crespo-Lopez These modifications will request consequent PLC modifications and it need to be tested carefully if you are agree I would propose to do these tests on the dummy load during the EYETS

16. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire Conclusion If the plant would swap automatically the consequences would be more severe. the no flow error was due to the high setpoint operation, which has not been tested thoroughly for all procedures. The CO2 flow into the detector has been stopped for less than 10 minutes. We were 4 people in front the computers when the problem happened. O.Crespo-Lopez

17. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire System overview O.Crespo-Lopez

18. European Organization for Nuclear Research - Organisation européenne pour la recherche nucléaire Chiller O.Crespo-Lopez