1. LHC commissioning experience at 6.5 TeV
Eucard Workshop - Firenze
November 2015
Jörg Wenninger
CERN Beams Department
Operation group / LHC
For the LHC commissioning and operation teams

2. Outline Introduction Re-commissioning of LHC
E-cloud control & diagnostics
Diagnostics examples
Summary
Eucard WS - Firenze

3. The Large Hadron Collider LHC
Installed in 26.7 km LEP tunnel
Depth of m
Lake of Geneva
LHC ring
Eucard WS - Firenze
Control Room
SPS ring

4. The Large Hadron Collider LHC
Installed in 26.7 km LEP tunnel
Depth of m
Lake of Geneva
LHC ring
LHCb
CMS
LHC ring
Eucard WS - Firenze
Control Room
SPS ring
ALICE
ATLAS

5. The LHC can be operated with protons and ions (so far Pb208).
LHC ring layout
Total length km, in the former LEP tunnel.
8 arcs (sectors), ~3 km each.
8 straight sections of 700 m.
beams cross in 4 points.
2-in-1 magnet design with separate vacuum chambers.
2 COUPLED rings.
Eucard WS - Firenze
The LHC can be operated with protons and ions (so far Pb208).

6. LHC energy evolution Long Shutdown 1 (LS1) Energy (TeV) 6.5 TeV
Consolidation of all interconnections
7 TeV
Design
5 TeV
Magnet de-training after installation
Energy increase 
no quench at 3.5 TeV
Eucard WS - Firenze
4 TeV
3.5 TeV
Issues on Interconnection,
incident
3.5 TeV
Long Shutdown 1 (LS1)
1.18 TeV
Consolidation delays
2007
2008
2009
2010
2011
2012
2013
2014
2015

7. Outline Introduction Re-commissioning of LHC
E-cloud control & diagnostics
Diagnostics examples
Summary
Eucard WS - Firenze

8. LHC Run 2 goals (2015 - 2018) Operate the LHC at 6.5 TeV (or higher).
Operate with 25 ns bunch spacing.
For Run 1 we operated with 50 ns spacing (e-cloud).
Maximize the integrated luminosity & collect ≥ 100 fb-1.
Eucard WS - Firenze
Objectives for 2015:
Learning year of Run 2 (6.5 TeV, 25 ns spacing),
Achieving reliable operation with 25 ns spacing is top priority.
b* at the IPs were relaxed to ease operation: b* = 80 cm was selected while cm was in reach. We plan to move to cm in 2016.

9. 2015 operation October 28, Record no. bunches June 24,
50 ns e-cloud scrubbing run
April 3,
End of powering tests – 150 quenches
April 10
First beam at
April
May
June
July
August
Sept
Oct
Easter,
Beam circulating
June 3,
Start of physics operation for run 2
August 7
End of 25 ns e-cloud scrubbing run
September 16,
Chili 8.3M earthquake shakes LHC rings

10.  R. Jones on Thursday morning
Machine status in 2015
We could rely on a solid experience from run 1 ( ) to startup the LHC – back in business in 2 months, followed by intensity ramp up since early summer.
Status:
Excellent magnetic reproducibility,
Optics well corrected, 5-10% b-beating,
Aperture is good and compatible a further reduction of b*,
Magnets behaving well at 6.5 TeV (just 3 additional quenches since beam operation started),
Good & improved instrumentation,
Excellent operation control:
Injection, ramp, squeeze etc.
 R. Tomas on Thursday m.
Eucard WS - Firenze
 R. Jones on Thursday morning
The main point of concern is called e-clouds.

11. LHC operation evolution
‘Serious’ operation started with 50 ns beams (~ no e-cloud), this period was limited to 2 weeks due to start-up delays.
The intensity ramp up stopped at 480 bunches (max. 1380).
Operation with 25 ns beam spacing started mid-August, we have now reached 2244 bunches per beam (nominal ~2750 bunches – 82% full).
Operating at the cryogenics cooling power limit (e-cloud heat load).
E-clouds are the main limitation for operation, intensity follows conditioning.
Eucard WS - Firenze

12. Integrated luminosity
The initial projections of integrated luminosity for 2015 were ~5-15 fb-1.
We finally achieved > 4 fb-1.
The main reasons for the lower value:
Start-up delays (~6 weeks),
Availability issues (radiation failures on the quench protection tunnel electronics – solved now),
Difficulties to master electron clouds.
Eucard WS - Firenze
The 2015 proton run is finished now, this year will close with a 4 week lead ion run.

13. Performance
The start-up of Run 2 was relatively fast, we are ~40% below the Run 1 luminosity performance. We are still in the learning & ramp up phase.
Operation with 50ns beams – no e-clouds – was a lot easier !
Peak luminosity:
Run 1: 7.61033 cm-2s-1
Run 2: 5.11033 cm-2s-1
Eucard WS - Firenze
4 TeV
Design:
11034 cm-2s-1
6.5 TeV
3.5 TeV

14. Outline Introduction Re-commissioning of LHC
E-cloud control & diagnostics
Diagnostics examples
Summary
Eucard WS - Firenze

15. Instrumentation challenges for LHC
After LHC Run 1 most of the instrumentation was in good shape, with improvements in the pipeline for Run 2. Some improvements:
Lower BPM systematics (crate temperature),
Gated tune system for compatibility with transverse feedback system,
New synchrotron radiation monitor mirrors (heating by beam) and optics,
Local very high resolution BPMs, …
For Run 2 some of the main challenges are:
(Almost) Everything bunch-by-bunch (up to 2800),
E-cloud diagnostics,
Instability diagnostics.
Eucard WS - Firenze
I will concentrate here mainly on diagnostics around electron clouds since this is our main issue at the moment.

16. Electron cloud challenge
The LHC with its tightly packet bunches spaced by 25ns is subject to a very strong electron clouds.
The key parameter for e-clouds is the Secondary Emission Yield (SEY) of electrons from the vacuum chamber surface.
N+1 e-
Bunch N+1 accelerates e-,
multiplication at impact
++++++
N+2 e-
Process repeats for Bunch N+2 …
++++++ N e-
Bunch N liberates e-
++++++
Eucard WS - Firenze
The e-cloud triggers vacuum pressure increases and beam instabilities. It may deposit excessive heat on the vacuum chamber walls  cryogenic cooling capacity and stability.

17. Electron cloud mitigation
There is a strong dependence of e-clouds on bunch spacing (at fixed SEY).
Vacuum chamber in run
With 50 ns spacing e-clouds are much weaker than for the design spacing of 25 ns.
 To ease life during Run 1, the bunch spacing was reduced to 50 ns
Eucard WS - Firenze
Conditioning of the vacuum chamber by beam-induced electron bombardment (“scrubbing”) leads to a progressive reduction of the SEY:
e-clouds are produced deliberately with the beams to bombard the surface of the chamber to reduce the SEY until the cloud ‘disappears’ (self-destruction).
Conditioning is performed at 450 GeV where fresh beams can be injected easily.
Conditioning requires with a beam that is more powerful ( more electron could generation) that the beam one plans to use for operation !

18. Scrubbing strategy
50 ns: scrubbing with 25 ns, then revert to 50 ns for operation.
25 ns: try the same strategy  we have invented a new doublet beam to enhance the e-cloud further.
The doublet beam violates all LHC beam instrumentation specs !
…but in the end it could be accommodated
Eucard WS - Firenze
Cryogenics limit 
Doublet beam
20 ns
5 ns 

19. E-cloud diagnostics: heat load
Cryogenic cell
Eucard WS - Firenze
Cooling channels
The heat load QBS deposited in the vacuum chamber beam screen (cooled to 4-30 K with He gas) is the KEY observable for e-cloud.
The heat load reflects the strength of the e-cloud.
QBS can be reconstructed from cryogenic temperature and pressure measurements (+ some modelling as missing some observables).
Online heat-load measurements that became available in 2015 were essential to monitor the evolution of e-cloud during scrubbing and operation.

20. E-cloud diagnostics: heat load
Energy
Eucard WS - Firenze
Example of heat load evolution during filling and ramp. During transient phases the heat load is not accurate (~10 mins delays).
The large spread of heat load from one arc to the next is real and not understood ! If all arcs would be at the level of the lowest one, we would have already reached design intensity.

21. E-cloud diagnostics - RF
The amount of e-cloud affecting a given bunch is reflected in its energy loss on each turn observable on the RF stable phase.
Stable phase information is used to monitor the e-cloud on a bunch-by- bunch level – very powerful diagnostics.
RF phase shift
Eucard WS - Firenze
25 ns beam – strong electron cloud
100 ns beam – no electron cloud B1 B1
Bunch no.

22. Scrubbing runs Two scrubbing run for 50 ns and later 25 ns operation.
Eucard WS - Firenze
Two scrubbing run for 50 ns and later 25 ns operation.
SEY inferred from heat load measurements, confirmed by beam stability.
Doublet beam could not be used, too unstable beam (!!) – SEY too high.
We came out of the scrubbing runs with an important residual e-cloud activity.

23. Heat load evolution
Courtesy K. Brodzinski
QBS [W/half cell]
Number of injected bunches
Installed capacity limit for K including existing margin recovered from lower K
Margin from lower K
Limit for s2-3
116
Installed capacity limit for K
Eucard WS - Firenze
As the number of bunches was increased the LHC was operated closer and closer to the heat load capacity of the cryogenic system.
The intensity in the LHC is currently limited by the cryogenics, we can only step up intensity when we gain on the e-cloud front.

24. Emittances
The wire scanners, our reference instrument, are limited to ~12 bunches at top energy – unusable for any physics operation.
Limited by magnet quenches & damage to the wire.
Our emittance workhorse is therefore the synchrotron light monitor that also delivers bunch by bunch emittances (30 b / sec).
Absolute calibration is delicate, relative calibration is good.
Eucard WS - Firenze
Essential instrument to track emittance blow up from beam- beam, e-cloud or instabilities
Difficult to compare injection and flat top measurements (source change)

25. Instability observation
Diagnostics of instabilities proved to be difficult at LHC during Run 1.
Improved diagnostics (head-tail monitors, bunch-by-bunch transverse feedback data, etc) is in place or will be in place soon.
Instability recorded by head-tail monitor
A triggering network put in place to be able to trigger all instruments of interest from one or multiple source devices.
Eucard WS - Firenze
 presentation by T. Levens on Thursday afternoon

26. Outline Introduction Re-commissioning of LHC
E-cloud control & diagnostics
Diagnostics examples
Summary
Eucard WS - Firenze

27. Emittance cross-calibration
With colliding beams the convoluted emittances can be reconstructed from the absolute luminosity or from beam separation scans.
With one experiment (CMS) providing real-time bunch- by-bunch luminosity one can perform bunch-by-bunch cross-calibrations – agreement at the level of  5%.
Eucard WS - Firenze
BSRT = synchrotron light mon.

28. Synchrotron radiation damping
Bunch length
Synchrotron radiation damping at 6.5 TeV is sufficient to overcome collective effects (IBS) and noise and damp the proton emittances !
After ~22 hours the bunches become longitudinally (slightly) unstable  growth
tSR ~ 18 hours
teff ~ 85 hours
Eucard WS - Firenze
In collision the horizontal emittance is ≈ stable, the vertical emittance shrinks !
BSRT = synchrotron light mon.

29. Luminosity imbalance
As soon as the luminosity increases the 2 main LHC experiments watch each others luminosity… In 2015 we observed an imbalance ranging from ~10% (start of fill) and ~4% (end of fill).
Roughly ~3% of the change is explained by the emittance evolution (due to the crossing angles at the IPs), we are missing 5-6% that is not understood.
Challenging to dis-entangle – precision knowledge of the parameters at the IP in collision is missing (b*, dispersion, crossing angles, errors on L measurements…).
Hot off the press: the exp. with the lower luminosity fixed a firmware problem – the luminosity imbalance is now smaller and inverted !
Eucard WS - Firenze
Luminosity
Time

30. Aperture restriction
A position with anomalous beam losses was located on beam 2 in the arc between LHCb and ATLAS only few days after commissioning.
An aperture restriction due to an was found by scanning the beam position.
Shifted beam orbit
Clear for beam
The object outline could be mapped accurately with our BLMs and BPMs.
The reference beam orbit was shifted upward and sideways to avoid the object.
So far operation – even at high intensity – does not suffer from this object.
Opening the magnet to remove this object would take 2-3 months !
Vacuum chamber
Eucard WS - Firenze
Edge of the object

31. Unidentified Falling Objects - UFOs
Dust particles falling into the beam – ‘UFOs’ – have interfered with operation since Run 1.
If the induced losses are too high, the beams are dumped to avoid a magnet quench (20 times / year in Run 1).
UFOs have also present at 6.5 TeV – 17 beams were dumped by UFOs and 2 magnets were quenched.
Vacuum chamber
Eucard WS - Firenze
2015
25ns scrubbing
Fortunately the rate decreases with time – significant condition is observed (also in Run 1).
Further fine tuning of the beam loss monitor thresholds for such short losses (millisecond scale) is possible.
 number of quenches

32. Chile earthquake September 16th
In the night of September 16th an earthquake of magnitude 8.3 in Chile struck the LHC for ~1 hour.
The machine was at injection, and the main effect was a significant radius change (2x largest tides).
The radial amplitude was sufficient to make a very good chromaticity measurement. The OP crews through the radial feedback had gone crazy…
Eucard WS - Firenze
T ~ 20 s

33. Thank you for your attention!
Outlook
After a few month of commissioning and after curing some initial teething problems, operation at 6.5 TeV is now stable and quite robust.
The LHC is currently operating with high intensity beams of 25 ns spacing, limited by the available cryogenics power due to strong electron cloud activity.
On the instrumentation side bunch-by-bunch information is essential and is steadily improving. A related issue is the collection, filtering and storage of high frequency bbb data. Our controls group, despite pre-warnings, is lagging behind…
With the improvements that are anticipated for 2016 we should reach the required performance for Run 2 in the coming year.
Eucard WS - Firenze
Thank you for your attention!

34. Eucard WS - Firenze

35. Magnetic model
Like most super-conducting machines the LHC is subject to decay and snapback of persistent currents at injection / start of the ramp.
Q excursions up to 0.06, Q’ excursions of ±20 units.
Decay time constant ~30 mins, snapback over 1 min.
Both decay and snapback are parametrized with a model that is played back during injection and fed into the start of the ramp.
Quality of the model: ±0.01 (Q), ±2-4 (Q’).
For Q’ we rely ~ entirely on reproducibility (one check at start of injection)
Eucard WS - Firenze
Tune snapback
Run1
Tune decay model

36. Tune and transverse feedbacks
At high intensity the LHC can only be operated with a transverse feedback (damper). In some phases the damper gain is so high that it is impossible to measure a tune…
‘Poisoned’ LHC run1 – fortunately the tune is reproducible!
Since 2015, 450 out of 35’640 RF buckets are dedicated to tune measurements: the damper gain can be set independently to lower value – accept some emittance blow up but get a better (gated) tune signal.
Eucard WS - Firenze

37. Dipole training campaign
The 1232 main dipole magnets had to be trained for 6.5 TeV operation training quenches could be performed for each sector in 24 hours, limited by the recovery time of the cryogenic system.
Just over150 training quenches were required to bring the LHC to e TeV.
The large spread in number of quenches between the eight sectors (arcs) is due to the mixture of magnets from the 3 producers.
3 spontaneous (no beam loss!) quenches occurred in operation at 6.5 TeV.
Eucard WS - Firenze