Presentation is loading. Please wait.

Presentation is loading. Please wait.

LMC Update - R2E Failures December 5 th 2012 (R2E) Mitigation Project: www.cern.ch/r2e LMC Update December 5 th 2012 R2E SEE Update M. Brugger on behalf.

Published byPeter Boyd Modified over 9 years ago

Similar presentations

Presentation on theme: "LMC Update - R2E Failures December 5 th 2012 (R2E) Mitigation Project: www.cern.ch/r2e LMC Update December 5 th 2012 R2E SEE Update M. Brugger on behalf."— Presentation transcript:

1 LMC Update - R2E Failures December 5 th 2012 (R2E) Mitigation Project: www.cern.ch/r2e LMC Update December 5 th 2012 R2E SEE Update M. Brugger on behalf of the R2E Project Review follow-up: LinkLink !!! Many Thanks To Everybody !!!

2 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 2 Power-Converter Riddle AC/DC Problem understood Full Mitigation (LS1) ‘New’ Failures Vacuum PLC in P7 Increase in Rad-levels ‘New’ Critical Area Cryo Mecos in UX45 P4 10x more radiation in 2012 ‘Heaviest’ Radiation Test & Biggest ‘Burn’ H4Irrad last run (11 parallel User) ‘Spectacular’ final failure Introduction

3 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 3 Critical areas well within predictions Operational parameters are important Other areas not to be forgottenn e.g, UX45: 2011: 2.6x10 6 2012: almost 2x10 7 cm -2 Radiation Levels Predictions versus Observations (scaled to exp. lumi) 1.1x10 8 1.8x10 7 1.2x10 8 1.8x10 7 5.5x10 7 3.0x10 7 2.6x10 8 6.5x10 7 …and Today Decrease: TCL closed Increase: Tight Collimators

4 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 xMasBreak Actions very efficient Mecos AMB Events (P4/8) 4 © A. Suraci et al. TE-CRG-OA Better cleaning/control of the beam CV891/fb-1 2.3 1.9 0.7 4.3 3.2 - 1.5 ? Cryogenics TS2/3 Period interesting 2011 2012

5 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 Difference in the periods between Technical Stops? Clear correlation – Reason? (Vacuum [Scrubbing, Solenoids], Satellite Bunches, … ?) Interesting behaviour also in P4 5 Cryogenics

6 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 6 H4irrad tests confirmed expected sensitivity Cryogenics UPS FIRE No Beam (4h40) Beam without Target (7h40) Destructive SEU AMB Mecos Linde Type (P4/P8) S2M X X X X X X X 2X X X XXXX Vacon X No Beam (12h10) SEU Similar to P4 events 3h502h40 © B. Ivens, F. Morant, M. Pezzetti et al.

7 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 7 Locations: UJ56, UJ76, UX45, US85 Failure types: Mecos AMB Profibus communications Mitigations: AMB case (UX45, UX65, US85): Relocation under study [Mecos, S2M] Profibus: Relocation from UJs during LS1 Main Impact: Important contribution to R2E downtime Cryogenics DumpDump/TBCNo DumpNo Dump/TBC 4210 Status November 2012

8 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 8 QPS – SEE Related Failures DumpDump/TBCNo DumpNo Dump/TBC 312221 Locations: UJs, RRs, DS Failure types: Digital DAQ systems for nQPS 600A protection uFIP communication Mitigations: New DAQ system under design (FPGA) Firmware upgrades BricoFip (mid-term)/NanoFip (long-term) Main Impact: caused most of the R2E dumps Status November 2012

9 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 9 QPS – SEE Related Failures R2E consolidation 2012: 27 R2E trips during stable beam >LS1: with all foreseen measures in place: 1 © R. Denz, J. Steckert 2011: 3.5 dumps per fb -1 2012: 1.4 dumps per fb -1 1/3 of errors in UJ14/16/56

10 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 10 Locations: UJs, RRs, ARC Failure types (only confirmed ones kept in follow-up): 600A: Aux Power Supply [14] FGC [10] 120A failure [1] Mitigations: AC/DC power supply problem understood New FGClite design (on-going) Radiation tolerant power converters (started) Main Impact: Many destructive failures Power Converters DumpDump/TBCNo DumpNo Dump/ TBC 140110 Status November 2012

11 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 11 Failure Overview Dump ConfirmedDump TBCNo DumpNo Dump TBC 5710366 Destructive failures Dump ConfirmedDump TBCNo DumpNo Dump TBC 16140 Preliminary! Status November 2012 (all failures)

12 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 12 Total R2E related downtime ~250 (confirmed) - ~300h (incl. TBC) Mainly based on dump cases (others difficult to track) Downtime Analysis Preliminary! Status November 2012

13 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 What Changes After LS1 13 Main impacted systems – observed failure signatures: Equipment Upgrades New developments Relocation (UJs, US85, UX45 partly) Shielding (RRs) (Future) Commercial systems: Full relocation (except REs) No new installation in critical areas without radiation tests & developments >LS1 R2E related failures: Compatible with nominal/ultimate goal Enormous Effort during LS1

14 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 14 Tunnel Equipment: DS/ARC will get critical Long-term damage to be considered (cells 8-13) Impact of 25ns operation? Future upgrades/new installations RRs & Power Converters FGClite prototype in time and to be deployed Power-part R&D started & component tests on-going Areas to be watched UX45!, (UX25), UX65, UJ/UA23, UJ/UA87 (+REs) What Remains After LS1

15 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 15 Implementation has started General layout soon to be frozen Optimization (RP, CV, Tests) on-going Solutions for rail system investigated 1 st Planning exists DIRAC dismantling critical to be ready in time Aiming for 2014 Operation PS-EAirrad

16 LMC Update - R2E Failures December 5 th 2012 R2E: Past/Present/Future ~3 dumps per fb -1 2012 ~12 dumps per fb -1 2011 2011/12 xMasBreak ‘Early’ Relocation + Additional Shielding + Equipment Upgrades Several shielding campaigns prior 2011 + Relocations ‘on the fly’ + Equipment Upgrades < 0.5 dump per fb -1 R2E-Project aiming for … >LS1 (nominal -> ultimate) 2012 SEE Failure Analysis -Equipment relocations @ 4+ LHC Points (>100 Racks, >60 weeks of work) -Additional shielding -Critical system upgrades (QPS, FGC) 16 Status November 2012

17 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 17 2011, xMasBreak and previous actions very efficient 2012 Failures: Within Chamonix predictions … and the winner is: QPS + EPC (+Cryo) LS1 Mitigation Enormous relocation campaign (4+ Points) Remaining shielding Numerous system upgrades (QPS, EPC) R2E test facility in preparation Long-Term Damage & 25ns Operation -> impact on radiation levels in ARC/P4/P6 Conclusions

18 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 18 BACKUP

19 LMC Update - R2E Failures December 5 th 2012 UJ overview High Energy Hadron Fluence [  /cm -2 ] Area201120122011/fb -1 2012/fb -1 Comments UJ142.8*10 8 1.4*10 8 5.1x10 7 7.9x10 6 Physics deb. UJ162.2*10 8 1.1*10 8 4.2x10 7 6.2x10 6 Physics deb. UJ231.5*10 6 2.6*10 6 2.8x10 5 (*)1.5x10 5 (*)Injection UJ563.9*10 7 1.3*10 8 7.2x10 6 7.3x10 6 Physics deb. UJ766.4*10 6 6.0*10 7 1.2x10 6 (*)3.4x10 6 (*)Collimation UJ873.0*10 6 2.6*10 6 5.7x10 5 (*)1.5x10 5 (*)Injection P1: decrease by factor of 6x, thanks to additional shielding P7: increase due to tight collimator settings P2/8: reduction because of cleaner injections in the LHC (*) = ATLAS/CMS luminosity 19

20 LMC Update - R2E Failures December 5 th 2012 RR overview P1/5: decrease up to a factor of 2x (1.5-2x) due to TCL.4 closure P7: asymmetry L/R (also observed in operation) High Energy Hadron Fluence [  /cm -2 ] Area201120122011/fb -1 2012/fb -1 Comments RR137.2*10 6 1.4x10 7 1.4x10 6 7.9x10 5 Physics deb. RR178.4*10 6 2.0x10 7 1.6x10 6 1.1x10 6 Physics deb. RR531.2*10 7 2.0x10 7 2.3x10 6 1.1x10 6 Physics deb. RR571.0*10 7 2.0x10 7 1.9x10 6 1.1x10 6 Physics deb. RR737.7*10 6 4.2x10 7 1.5x10 6 (*)2.4x10 6 (*)Collimation RR771.1*10 7 2.1x10 7 2.1x10 6 (*)1.2x10 6 (*)Collimation (*) = ATLAS/CMS luminosity 20

21 LMC Update - R2E Failures December 5 th 2012 UX overview (4/6/8) P4: increase, beam-gas coupled with higher cumulative current in the machine How much can/will it go up in view of 25ns operation? High Energy Hadron Fluence [  /cm -2 ] Area201120122011/fb -1 2012/fb -1 Comments UX452.6*10 6 1.4x10 7 4.3x10 5 (*)8.2x10 5 (*)Beam-gas UX65N/A~1.x10 6 N/A Beam-gas UX852.1*10 8 2.8x10 8 1.7x10 8 (**)1.6x10 8 (**)Physics deb. US854.4*10 7 7.0x10 7 3.7x10 7 (**)4.1x10 7 (**)Physics deb. (*) = ATLAS/CMS luminosity (**) = LHCb luminosity 21

22 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 22 Leakage into DS getting important Cells 7-10: ~10Gy <Cell17: ~1Gy >Cell17: ~0.1Gy Currently relatively low vacuum activity ?How will this possibly change with 25ns? ARC & DS

23 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 23 Beam Loss Analysis (B1) Injection: ~7-8x10 16 Lost protons: Total: ~1x10 16 Experiments: ~3.4x10 15 Rest: ~6.6x10 15 Collimation: IR3: ~6x10 14 IR7: ~6x10 15

24 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 24 Beam Loss Analysis (B1) Injection: ~7-8x10 16 Lost protons: Total: ~1x10 16 Experiments: ~3.4x10 15 Rest: ~7.4x10 15 Collimation: IR3: ~7x10 14 IR7: ~6.7x10 15

25 LMC Update - R2E Failures December 5 th 2012 Cryo Actions Taken CV891: automatic reset implemented into PLC (action: switch off 24 Vdc) TT89x: modification of electronics card PLC: moved into better protected zone (P4-P6-P8 caverns) 24 Vdc: longer term, to be moved into a safer protected zone and better distribution of power consumption to be done. AMB (Magnetic bearing controllers): relocated partially in P8, (2 nd unit will be done at LS1, additional shielding foreseen; at P4, will be done during LS1

26 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 26 UJs: relocated DSP: To be replaced by rad-tol version + Firmware upgrade of the uC based boards oQPS, nQPS: Firmware upgrade for DAQ systems, dipole splice supervision boards, current lead protection systems (RR) Detectors with rad-tol DAQ in cells 8-12 (nDQQDL) IPQ/IPD/IT: Rad-tol DQQDI in all RRs (200 boards already produced) 600A: Rad-tol quench detection board (prototype developed) Optional: Special rad-tol splice superivsion and BricoFIP TM QPS R2E Upgrades

27 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 27 Vacuum DumpDump/TBCNo DumpNo Dump/ TBC 4300 Locations: UJ76 (all confirmed), Point 4 (TBC), Point6 (TBC) Failure types: Loss of the PLC control (soft) Power supply (destructive failures) Mitigations: Relocation from UJ76 Status November 2012

28 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012 28 Failure Statistics

29 LMC Update - R2E Failures December 5 th 2012

30 LMC Update - R2E Failures December 5 th 2012

31 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012

32 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012

33 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012

34 LMC Update - R2E Failures December 5 th 2012 LMC Update - R2E Failures December 5 th 2012

Download ppt "LMC Update - R2E Failures December 5 th 2012 (R2E) Mitigation Project: www.cern.ch/r2e LMC Update December 5 th 2012 R2E SEE Update M. Brugger on behalf."

Similar presentations

Discussion on TE/CRG equipment at H4IRRAD Marco Pezzetti (A. Tovar – B. Ivens / Support CE for cryo LHC) & TE-CRG-OA – TE-CRG-CI 1TECRGCE M.PEZZETTI.

Discussion on TE/CRG equipment at H4IRRAD Marco Pezzetti (A. Tovar – B. Ivens / Support CE for cryo LHC) & TE-CRG-OA – TE-CRG-CI 1TECRGCE M.PEZZETTI.
Will we still see SEEs? (a word trick game) R2E Project Meeting - draft presentation for Chamonix 2012 – 19 th January 2012 M. Calviani (EN/STI) for the.

Will we still see SEEs? (a word trick game) R2E Project Meeting - draft presentation for Chamonix 2012 – 19 th January 2012 M. Calviani (EN/STI) for the.
K. Potter RADWG & RADMON Workshop 1 Dec. 20041 WELCOME TO THE 4th RADWG & RADMON WORKSHOP 1 December 2004.

K. Potter RADWG & RADMON Workshop 1 Dec WELCOME TO THE 4th RADWG & RADMON WORKSHOP 1 December 2004.
LHC UPS Systems and Configurations: Changes during the LS1 V. Chareyre / EN-EL LHC Beam Operation Committee 11 February 2014 EDMS No. 1354977111/02/2014.

LHC UPS Systems and Configurations: Changes during the LS1 V. Chareyre / EN-EL LHC Beam Operation Committee 11 February 2014 EDMS No /02/2014.
Beam commissioning strategy 09-12-09 Global machine checkout Essential 450 GeV commissioning System/beam commissioning Machine protection commissioning.

Beam commissioning strategy Global machine checkout Essential 450 GeV commissioning System/beam commissioning Machine protection commissioning.
Interlocks for Magnet Protection System1 Beam dumps 4 th 11h33 and 5 th 03.01 Both beam dumps caused by a stop of a Powering Interlock PLC,

Interlocks for Magnet Protection System1 Beam dumps 4 th 11h33 and 5 th Both beam dumps caused by a stop of a Powering Interlock PLC,
PROFIBUS controller problem (SEU) on cryo valve on DFBMH (600 A circuit RQ6.L7B1/2). Loss of cryo maintain ==> Beam dump. End of fill #1975 after ~2.5.

PROFIBUS controller problem (SEU) on cryo valve on DFBMH (600 A circuit RQ6.L7B1/2). Loss of cryo maintain ==> Beam dump. End of fill #1975 after ~2.5.
LHC Beam Dumping system

LHC Beam Dumping system
Crab Cavities in IR1 and IR5 Some considerations on tunnel integration What will be the situation in the tunnel after the LHC IR Phase-1 Upgrade. What.

Crab Cavities in IR1 and IR5 Some considerations on tunnel integration What will be the situation in the tunnel after the LHC IR Phase-1 Upgrade. What.
TS-LEA, CERN, 1211 Geneva 23 1 T. Wijnands TS/LEA, C. Pignard AB-CORADWG-RADMON day – 1 December 2004 LHC Power Converters & Radiation T. Wijnands TS/LEA.

TS-LEA, CERN, 1211 Geneva 23 1 T. Wijnands TS/LEA, C. Pignard AB-CORADWG-RADMON day – 1 December 2004 LHC Power Converters & Radiation T. Wijnands TS/LEA.
R2E Weekly Report status AT/MB/MC – 4 th R2E Project Meeting – 7 th April 2011.

R2E Weekly Report status AT/MB/MC – 4 th R2E Project Meeting – 7 th April 2011.
Oliver Bitterling  Introduction to the QPS  Radiation damage in electronic systems  Construction of radiation tolerant systems  Radiation test and.

Oliver Bitterling  Introduction to the QPS  Radiation damage in electronic systems  Construction of radiation tolerant systems  Radiation test and.
J. MEKKI (1/14) 05/12/12 – RadWG meeting Summary of the radiation levels of the equipments at CNRAD RF MosFETs Cryogenics TE/EPC LED warning system BPM.

J. MEKKI (1/14) 05/12/12 – RadWG meeting Summary of the radiation levels of the equipments at CNRAD RF MosFETs Cryogenics TE/EPC LED warning system BPM.
LHC Beam Screen Heaters: System overview, consolidation needs and project planning 4 February 2013 - TE-CRG/JCC (Beam Screen Heater Review) 1.

LHC Beam Screen Heaters: System overview, consolidation needs and project planning 4 February TE-CRG/JCC (Beam Screen Heater Review) 1.
Radiation test - PSI Date Equipment Owner Test groupDUT May 20th EN/STI OSL, TLD BE/BI Detector June 1stTE/EPCEN/STI Mosfet Voltage regulators Voltage.

Radiation test - PSI Date Equipment Owner Test groupDUT May 20th EN/STI OSL, TLD BE/BI Detector June 1stTE/EPCEN/STI Mosfet Voltage regulators Voltage.
RADWG 6 TH - 2012 16 TH OCTOBER 2012 G. Spiezia (EN/STI/ECE)

RADWG 6 TH TH OCTOBER 2012 G. Spiezia (EN/STI/ECE)
MP3 report : W49 (6 slides, 6’) J.Ph Tock for the MP3 team Since oral report last week (24 th of November) 1JPhT – LMC 1st of December 2010.

MP3 report : W49 (6 slides, 6’) J.Ph Tock for the MP3 team Since oral report last week (24 th of November) 1JPhT – LMC 1st of December 2010.
Safe-Room Status Intermediate ‘Conclusions’ M. Brugger (based on HSE discussion)

Safe-Room Status Intermediate ‘Conclusions’ M. Brugger (based on HSE discussion)
TE-MPE-CP, RD, 23-Nov-2010 1 Summary of Radiation Induced QPS Events in LHC 2010 R. Denz TE-MPE-CP.

TE-MPE-CP, RD, 23-Nov Summary of Radiation Induced QPS Events in LHC 2010 R. Denz TE-MPE-CP.
1 Second LHC Splice Review Copper Stabilizer Continuity Measurement possible QC tool for consolidated splices H. Thiesen 28 November 2011 K. Brodzinski,

1 Second LHC Splice Review Copper Stabilizer Continuity Measurement possible QC tool for consolidated splices H. Thiesen 28 November 2011 K. Brodzinski,

Similar presentations

Ads by Google