Yesterday’s morning’s controls issues

Slides:



Advertisements
Similar presentations
8:16 SB 25ns dumped by RF; integrated lumi 0.6 nb-1. 9:14 BIC problem in TI8 and CMS recovering their tracker 10:09 Abort gap cleaning commissioning. 16:29.
Advertisements

Friday, h10: Stable beams at 1.38 TeV. 09h03: Beam dump.  One bunch dropped too much in intensity and gave BPMS interlock (dump protection). 10h36:
Sat/Sun 26/27 th March Sat 18h20: Stable beams. Fill #1653. Sun 10h02: Stable beams continued. vdM scans. Length calibrations. 16h59: Beam dump. Delivered.
5/11/2012LHC 8:30 meeting Summary of Week 44 Coordination: Mike Lamont & Jan Uythoven.
LHC Wed 16 Sept 2015 Struggling with cryo o-night Tuesday - Wednesday 07:11 Injecting again 09:00 Dump on injection of train of 144 bunches, losses IP7.
:00: Prepared new fill B1: Q X =0.293, Q Y =0.269; lifetime = 25h ✔ B2: Q X =0.297, Q Y =0.267; lifetime = 5h ✗ Strong 50Hz and 100hz lines.
Monday h00: End of fill #1640: 3.7 pb h14: RB S12 and S23 tripped before loading rampdown table, active filter. MPE piquet. Fip_Com_Lost.
LHC progress Report LHCC Wednesday 23 rd March 2011.
00:15: Stable beams fill 1883, 1.1E33 cm-2s-1.  Seems the 144 bunches beam 1 doe not fit properly on the injection kicker waveform. Systematically the.
04:01 Injection kicker timing problems B2  Injection pre-pulse shifted, causing BETS to inhibit beam injection. 144 bunches on TDI.  Run without ROF.A81B2.
Thursday 20 th April 01:53 RCBV17.R7B2 tripped as happened few days ago: SIS intercepted. 05:54 Stable beams, 1380b, still RF problems B1, abort gap cleaning.
Tue 23/8 – Wed 24/8 09:00 Start 90 m optics run – Preparation and verification of the collimation functions – Re-generation of functions for Transverse.
LHC Status Fri Morning 3-June
Wednesday 8.9 External crossing angles:
Wednesday :06 Dump fill 3533: interlock BPMS in IP6
Machine Coordinators: G. Arduini, J. Wenninger
Thursday 25/11/10 & Night Work on: 16:15 Inject fill #1520
Friday 30th September 06:00 Operator dump fill #2171, int. lumi about 75 pb-1 Previous operator dump from physics was on 14 September ! 07:00 Start of.
Summary Friday h38: Ramp down and pre-cycle.
Tuesday Fill lost #1855 at 7:28 due to the trip of RQX.R1. This was traced to a software bug in the FGC… The Class 51 (LHC_PC) FGC software has been upgraded.
Monday 11/07: Recovery Plenty of accesses.
Saturday 30th June - Sunday: Rocky !
Saturday 09:40 Stable beams #1369 (56 bunch scheme)
Week 46 Week 46: Machine coordinators: Roger Bailey – Gianluigi Arduini Main aims of the week: Stable beams with ions Scheduled stop for ion source refill.
Fill 1410 revisited Peak luminosity 1.4e32 Beam current 2.68/2.65 e13
Tue :25 Beams dumped due to RF vacuum interlock
Wednesday /Thursday 09-11:00 Verification of the LSS6 interlocked BPMs: took longer to fill due to some problems with RF cavities in the PS. In the mean.
LHC Morning Meeting - G. Arduini
Summary of Week 11: 14 – 21 March
Thursday 17th May 01:11 Fill #2633 dump in squeeze arc circulator load channel Ad1B 03:16 Fill #2644 Stable beams, lumi 5.2e33 cm-2s-1 04:38 Dumped by.
Wednesday 31 October: Morning
Lost at 12:05 to power converter (RTQX2.R1) problem
Note: squeeze in Atlas, CMS, LHCb and Alice on going
Mon 22/8 – Tue 23/8 00:07 stable beams #2040. Initial luminosities: 2.1x1033 cm-2s-1 18:00 Programmed dump. End of physics #2040. ~83 pb-1 in ~18 hours.
LHC Morning Meeting - G. Arduini
Friday 31st August & night
Friday day 00:51 Fill #1862 Beams lost to inner triplet skew quad power converter fault. Delivered lumi 33 pb-1 Access required for repair RQSX3.L1 converter.
Saturday 2/4/ :00-09:00 Problems with SPS:
Tuesday 28th September Fill 1375 ongoing 09:00 Move in Roman Pots
Tuesday Loss maps in the night:
04-05/08/2010 End of physics at ~11:00 after 18.5 hours ~120 nb-1.
Friday morning 10:30 Optics measurements and corrections.
Wire almost entirely through beam before BLM triggers dump
Finally.... We had to wait a long time.... We had many problems....
Yesterday morning Held 1647 for a while – SPS kicker problem
Summary Thursday h21: Stable beams fill #1303.
Monday – Morning Fill 3265 stable beams since 1:31
Mon Access for switching temperature probe DFB L7.
Thursday Fill 1364 (24 bunch) dumped at 07h
Fill 1743 Peak luminosity ~6.7 e32 Integratated lumi ~20 pb-1
Monday 6th 07:39 Dump fill #1854, RQ6.L2 tripped, followed by access
Friday 8/10/10 - Morning Planned to keep fill until the afternoon
Monday morning 09:40 Dump fill 2838, integrated ~130 pb-1.
LHC Status Tue Morning 22-Aug
Wednesday h10: Programmed dump. 07h54: Injection. 200b.
Friday 14th September 03:15 Beta* at 1 km. Optics measurements, see yesterday. 07:00 Dump. 08:00 Refill with 3 nominal bunches for high beta. 11:15 Beams.
Monday :15 fill 3523 dumped by BPMs IP6
Summary Tuesday h23: Machine closed. Precycle started.
Wednesday 16th May 03:05 beam dump, RF fault crowbar problem on module M1B2 06:09 Another crowbar at injection. Access for RF; problem Beam Imminent Warning.
Wednesday 23/2 Thanks CRYO!!!.
Saturday 06/11 Triplet R8 tripped during ramp for collision setup.
Wednesday h18: 228b fill lost just before start of ramp.
Another Immortal Fill….
Tuesday 24/5 Fill # :00 Stable beam fill # bunches
Saturday 26th March 9:15 Problem with RB78 temporarily solved. It's running on one Thyristor bridge only and without the active filter LHCb polarity change.
Saturday 28th 08:17 Fill #1812: Stable beams, peak luminosity ~1.1x1033 cm-2s-1 19:00 Go to adjust for RF-checks; Record fill lumi of 31.7 pb-1 for CMS.
Tuesday End of fill 1393 Movement of suspicious Roman pots OK
Wed 12/10 09:30 – 10:30 Ramp down of ALICE solenoid to 50% nominal field on request of ALICE. Increase of the vacuum (e-cloud in this area that has not.
Monday 152 bunch operation summary
Presentation transcript:

Yesterday’s morning’s controls issues The JMS brokers went gradually out of memory due to bad clients (slow or too demanding clients). Around midnight the brokers were not able to answer to all requests, ending up in a congestion of the message circulation. The reboots during the night were not successful as there were still some clients 'avalanching' the brokers. This morning we searched for some clients on the CCC consoles and killed a few of them. Then the clean restart of the JMS brokers went ok and since 10h30 this morning the situation is ok. The BPM issue was a side-effect as the client concentrator was in a bad state due to the JMS service not answering correctly and therefore ended up into opening too many subscriptions to the BPM crates, hence crashing the server there. We continue now to monitor the traffic and the memory usage until tomorrow morning. We will also (1) make sure the notification system from DIAMON informs the JMS team well in advance when the memory consumption rises and (2) to detect bad clients. Pierre Charrue 23/03/2011 LHC 8:30 meeting

Yesterday 11:00 Beam back 12:10 Stable beams Fill 1644 136 bunches Initial luminosity 1.6e32 Afternoon: POPS down…, PS switches back to rotating machine… Beam quality good from the injectors, even though they had to check all settings for all beams after the switch back to rotating machine in PS. 18:00 beams dumped 19:12 Starting filling for 200 b just in time injection schedule just in time approval from rMPP Brennan checked the BPMD interlock Over interjected pilot OK - just 23/03/2011 LHC 8:30 meeting

Yesterday evening 21:15 Stable beams fill 1645 200 bunch initial lumi ~2.5 e32 (emittance ~2.4 micron) 23/03/2011 LHC 8:30 meeting

Night shift summary Held on to the stable beams until the programmed dump at 6:10. Performed only small optimizations in Alice and LHCb. To be noted: - a couple of RF LL line trips (4B2 and 3B2); - RB.A45 has lost a quench heater power supply; - this shift crew gave no access during this full MAN. Giulia & Lasse 23/03/2011 LHC 8:30 meeting

Feed-forward in ramp 23/03/2011 LHC 8:30 meeting

EOF 06:00 this morning 23/03/2011 LHC 8:30 meeting

Totals Total 1645 Thanks Atlas 23/03/2011 LHC 8:30 meeting

Incoming Tues PM 200b Weds AM Weds PM 1.38 TeV commissioning Thurs AM 1.38 TeV stable beams Thurs inter-fill ADT/AGC Thurs PM/night 1.38 TeV Fri AM CV intervention (tbc) 4 h Fri Energy gaps interlock test 1 h Fri PM/Night 1.38 TeV (plus extra ramp for quench test) Sat/Sun Back to 3.5 TeV, test ramp, fill #3 Transfer lines tests 3 h 96*75ns at injection 8 h First look at 50 ns 200 b continued Mon 06:00 Dump beams for TS 23/03/2011 LHC 8:30 meeting

Outstanding Injection of 96 bunches ( weekend) Timing distribution ( technical stop) FGC firmware update Collimation system controls fix 1.38 TeV setting-up ( Wednesday onwards) Crossing angle all zero: OK TCTs to set up & loss maps Interlock tests for the roman pot ( parasitic or TS) RF voltage limit interlock Longitudinal blow-up ( if needed) Wire scanners Abort gap cleaning ( with high intensity injection) Injection gap cleaning ( with high intensity injection) 23/03/2011 LHC 8:30 meeting