2. POPS Strategy for Restarting & Repairing
TE-TM 17 May 2016
Fulvio Boattini, Jean-Paul Burnet, Karsten Kahle

3. Contents
Analysis of the Fault in the POPS DSP1 storage capacitor bank;
PS operation in present conditions;
Confrontation with “the supplier”;
POPS recovery plan:
POPS operation in degraded mode;
Repairing of DSP1 capacitor bank with improved protections;
Upgrade of remaining 5 +1 spare capacitor banks;
Planning and Costs
Conclusions

4. Ante fact: POPS recommissioning of the faulted DC1 converter
Since February 2016 POPS runs in degraded mode
Re-commissioning was foreseen on Wednesday the 27th of April at the presence of a technician from “the supplier”.
A number of tests have been performed:
Visual inspection;
Insulation at 5kV;
Current and Voltage sensors injection;
IGBT commutation at reduced DC1 voltage;
Pre-charge at nominal DC voltage.
2 x 1500 V
440 Apk

5. Fault of the DSP1 Short circuited capacitors
POPS was restarted in normal mode but at the DSP1 capacitor bank failed in short circuit.
Short circuited capacitors

6. Fault Analysis DCDC1 + AFE1 + DSP1
The fault arrived when the pre-charge of all DSPx was nearly completed:
One or two capacitor units failed;
The complete capacitor bank (3MJ) discharged;
DCDC1 + AFE1 + DSP1

7. Fault Analysis DCDC1 + AFE1 + DSP1
The fault arrived when the pre-charge of all DSPx was nearly completed:
Udc1
Udc6
Udc2÷4
Udcp; Udcn
DC fuse melted
AFE charges DC caps
IEGTs are stopped here
DC1 converter post mortem 0V
3200V
One or two capacitor units failed;
The complete capacitor bank (3MJ) discharged;
Filtering caps discharged;
DC fuses open;
AFE rectifier filled the DC filtering caps to 3000V;
Input CB open.
AFE1 converter post mortem
IEGTs are stopped here
UdcpA; UdcnA
IafeR,S,T
Input CB is open much later 0V
3200V
DC fuse melted
AFE charges DC caps
DCDC1 + AFE1 + DSP1

8. Preliminary conclusions
The fault has been initiated by a short circuit in one or two capacitor units;
Short circuited capacitors
Technical explanation not found yet. Difficult to think it is a coincidence;
No IGBT fault nor short circuit detected;
DC1 repairing seems ok;
AFE diodes had 6kA pk current for 10ms (max is around 20kA pk);

9. Present status
The PS is presently running with the rotating machine (PS stop was limited to 24h)
Last maintenance of the rotating machine-February 2014 (generator was built in 1968)
The rotating machine is now operating correctly but suffers of a series of drawbacks:
No spare rotors
Electrical insulation reached end of life
Thyristor rectifiers are obsolete
Transformers reached end of life
Excitation and speed control are obsolete
Oil retention pits are not conform (five transformers plus generator)
Generator cooling is done with tap water (90 m3/h)

10. Confrontation with the supplier
A weakness in the design of the capacitors was identified during LS1.
CERN tried to obtain the replacement of all capacitors, but the supplier denied any problem.
CERN has now additional arguments to reiterate the request.
Bailiff on the CERN site to certify the damage in DSP1
“the supplier” has been invited
Real causes of the short circuit are not known yet. In depth analysis is required.

11. POPS recovery plan
Prepare POPS for running in degraded mode (without DC1);
All storage capacitors (630 units) will be measured;
Tests on dummy load;
DM will be ready for the next TS in June.
Repairing of DSP1; Target Q1 2017;
Improvement of capacitor bank and power converters protection;
New capacitors / shelfs to be installed in the container;
Upgrade of remaining capacitor banks; Q4 2017;
Upgrade of remaining 5 containers + 1 as spare;
Baseline: the project will be financed and executed by CERN;
CERN dispute with “the supplier” will continue to obtain a contribution.

12. POPS recovery plan: operation in DM
DSP HV cables tested against insulation OK
DC1 IEGT test with externally applied voltage OK
500V
100A
DC1 positive
DC1 negative

13. POPS recovery plan: operation in DM
DSP HV cables tested against insulation OK
DC1 IEGT test with externally applied voltage OK
Container areas to be enclosed by gates
Septembre 2016

14. POPS recovery plan: operation in DM
DSP HV cables tested against insulation OK
DC1 IEGT test with externally applied voltage OK
Container areas to be enclosed by gates
Septembre 2016
All storage capacitors (630) are being measured
Ongoing
AFE1 Pre-charge with externally applied voltage
W21
Restarting POPS on test magnets (Standby operation)
W23

15. POPS recovery plan: repairing DSP1
Protection improvement of storage capacitor is under study
Total capacitors divided into four blocks
Each block individually protected and segregated (less energy per bank)
Distribution of pressure reliefs around the container (to be checked with HSE)

16. POPS recovery plan: repairing DSP1
New DSP1 container to be built with new capacitors
IT 3990 for POPS-B
Rapeseed oil capacitors
Capacitors are under accelerated ageing test at CERN since 1400h
(T=60ºC; Upk=1.3Un; fout=9.6fn)
Test are planned to be finished end July 2016
2 x
Still 1250h to finish

17. POPS recovery plan: upgrade of all capacitor banks
The remaining 5 container shall be retrofitted to the new solution
An additional container will be purchased as spare

18. POPS recovery plan: upgrade of all capacitor banks
The remaining 5 container shall be retrofitted to the new solution
An additional container will be purchased as spare
Some ideas for additional protections
Fast DC circuit breaker
Voltage Crowbar
X 8
X 6

19. POPS recovery plan: upgrade of all capacitor banks
The remaining 5 container shall be retrofitted to the new solution
An additional container will be purchased as spare
Some ideas for additional protections
Fast DC circuit breaker
Voltage Crowbar
X 8
X 6
IEGT stack testing platform in B287

20. POPS recovery plan: Planning & Costs
Total cost estimation for repairing DSP1:
New container: kCHF
Capacitors: kCHF
Installation : kCHF
Additional protection kCHF
___________________________
Total reparation of DSP kCHF
Cost estimation for remaining containers:
Additional Container: kCHF
Capacitors: x250 kCHF
Container upgrade: x50 kCHF
Additional Protection: x50 kCHF
___________________________
Total kCHF

21. Summary POPS back in DM planned for TS in June 2016
New capacitor bank protection improvements currently under study
Upgrade of DSP1 container and POPS back in NM planned for Q1 2017
Upgrade of all remaining capacitor banks for Q4 2017
Global budget estimation for the project is 2.55 MCHF

22. Thank you for your attention
Questions ?