1. ITER-CERN Workshop CERN approach to the large-scale installation of Fibre Optics & Cabling 17 Jan 2017
Daniel RICCI
On behalf of the Electrical Engineering Group (EN-EL) at CERN

2. CERN-ITER Workshop – Daniel RICCI (CERN)
Introduction
Fibre Optics and Cabling is a Section part of
Electrical Engineering Group (EN-EL)
Engineering Department (EN)
EN-EL
From 400kV to 400/230V
~ 105 people (75 staff/fell associates)
FC: ~ 28 people (18 staff/fell associates)
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

3. Introduction: EN-EL domain of activity
CERN accelerator complex + Experiments + Data centres
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

4. Cabling = Fibre Optics & Copper Cables (Ctrl & DC)
Definitions
Cabling = Fibre Optics & Copper Cables (Ctrl & DC)
For all CERN accelerators, experiments and general infrastructure
User: a CERN person or a group requesting cabling or optical fibre connection
Not a client!
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

5. CERN-ITER Workshop – Daniel RICCI (CERN)
Type of activity (1/2)
Engineering Support to the CERN users
Specification of cable cross-sections, optical fibres and connectors
In collaboration with the users
Responsible for two large infrastructures
Optical Fibres ( km of fibres)
Ctrl Cable Trays (350 km)
Procurement and standardisation of cables, fibres, connectors and racks
For all EN-EL (including power distr.)
Average investment: 3.5 MCHF/year
Managing cable park: ~4200 drums for LS1
Monitoring aging and steering cable/fibre replacement campaigns
~ km cables in radiation areas
~1MCHF/machine point
The picture shows the complexity of the network with thick optical links crossing over the local region (Pays de Gex).
The network is multi-users, which is much more complicated to manage than point-to-point.
The big circle represent LHC. The nodes represent the main star points (fibre distribution points) on the CERN site.
Procurement and installation of standard racks (e.g for LHC)
12000 cables (rounded value) in radiation areas = copper km optical cables
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

6. Type of activity (2/2) Project Management for Cabling Installations
Users’ requests
Find solutions with design offices
~200
Management of contracts and contractors
Schedule installations with technical coordination
Link FC
Cabling has an active role as a link among the main actors of a project.
Those are:
the users (issuing cabling requests). We have about 200 with whom we are in regular contact.
the large procurement and installation contracts (for which cabling ensures the thorough follow up: from the specification to the final installation)
The integration office and the machine technical coordination (with whom cabling participates in the design and planning of the installation)
All what concerns the financial and budgetary aspects
CF is part of CERN. We are simply a support group centralising certain aspect of the job, namely the interaction with the contractors. In this respect, CERN users are not clients but colleague with whom we work together.
In this dynamics, cabling act literally like a glue. This means however that we are subject to a constant pressure in all the phases.
Adapt procedures with safety coord. and RP
Users’ budget and financial aspects
Very large volumes  strong impact on machines
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

7. CERN-ITER Workshop – Daniel RICCI (CERN)
Outline
Compilation and check of needs
Definition of cabling campaigns, overall coordination
QA of cable pulling and assembly of connectors
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

8. CERN-ITER Workshop – Daniel RICCI (CERN)
Outline
Compilation and check of needs
Definition of cabling campaigns, overall coordination
QA of cable pulling and assembly of connectors
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

9. Collection and treatment of the User requests
Set up of standardised request forms:
“Demande d’Installation” of Fibres (DIF), Cables (DIC), Racks (DIR),...
Requests have a unique identifier and are traced all along the process
Forms are conceived to be easily uploaded in the cabling database
In case of modification/update, the forms are exchanged with the users following a “versioning” process
Future evolution: web page, ticketing system
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

10. Iterative process with the Users
System specification/design
Single user specifies his system/cables
EN-EL participates in:
The definition of the optical fibre and connector types
The correction of the (copper) cable cross-sections according to the installation constraints
EN-EL brings overall view of installations with respect to multiple users
Definition and deployment of cable ladders and racks
Optimisation of the lengths of the cables
Overall cable laying/routing
Standardisation of components
Integration meetings are organised and centrally coordinated at CERN
Validations of layouts
Engineering Change Requests (ECRs)
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

11. Iterative process with the Users
Planning and machine coordination
Coordination meetings centrally managed at CERN
OBJECTIVE for FC: anticipate as much as possible the user requests
~1 year before the installation: deadlines for collection are set, monitored, and followed up in coordination meetings
Late requests are piled up and treated only if time/resources available
Draft of local planning for detailed FC follow up and monitoring
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

12. Procurement of cables/connectors (1/2)
EN-EL set up a specific unit for procurement and logistics of material
Standard cables/connectors are centrally bought by CERN store and internally transferred to a cable park
Special cables bought directly by EN-EL (~10% of the total)
Cable park is a storage area from which cables/connectors are dispatched to the various worksites
Logistics managed through “GESMAR” software
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

13. Procurement of cables/connectors (2/2)
Procurement unit works like an internal support for the project managers:
DAM
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

14. CERN-ITER Workshop – Daniel RICCI (CERN)
Outline
Compilation and check of needs
Definition of cabling campaigns, overall coordination
QA of cable pulling and assembly of connectors
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

15. Work by geographical campaigns
Optimise work organisation and resource distribution
PROJ 1
Various works for TE/ABT
PROJ 2
LIU (top priorities)
PROJ n
Various works for BE/BI
We de-compose and re-compose the users’ requests in a way to fit the maximum of works in the same geographical area.
2000 block on the left were compacted to 218 on the right.
The campaigns are then planned and scheduled through multiple interactions with the machine technical coordination.
This organisation also allows optimizing the cost, when the contracts foreseen advantageous discounts for regrouping the activities (discounts “par lot”).
Cost optimisation: profits from contractual discounts for large volumes
~2000 cabling requests for LS1  218 campaigns
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

16. Work preparation Execution LS1 PreparationLS2 Execution LS2 t
Execution
LS1
PreparationLS2
Execution
LS2
Technical stops t
Preparation: a continuous process spanning up to 70% of the total time
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

17. CERN-ITER Workshop – Daniel RICCI (CERN)
Forecasting
Estimation of volumes and resources with increasing accuracy
Tools developed
“PLAN” by Technical Coordination (EN-MEF)
Project Database & Resource Estimation Tool by Cabling
Used by LS1 management for project arbitration
Forecasting was an absolute necessity during this period.
In green the workload estimation expressed in number of staff needed.
In purple is the staff that was necessary during LS1 to accomplish the job.
2013
2014
Volume/Staff-number law
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

18. Monitoring of the activities
Development of a progress monitoring tool
Helps Proj. Managers + Contractors to organize present work and to plan future tasks
Warn timely on activity delay or anticipation
Inform CERN management on detailed project status
98% completion
A similar tool was implemented for dose monitoring
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

19. CERN-ITER Workshop – Daniel RICCI (CERN)
Work Organization (1/3)
Principle for requests processing
Grouping of cabling activities in the same area during the same period brings advantages to CERN (preparation, planning, supervision, reception and price reduction)
REQUEST GROUPING
SMALL REQUESTS
1 Job
1 Job
1 Job
1 Job
1 Worksite
Cabling Campaign
1 Worksite
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

20. CERN-ITER Workshop – Daniel RICCI (CERN)
Work Organization (2/3)
Work Phases
QAP (quality assurance plan)
USERS
EN-EL-FC
CONTRACTOR
1. Preparation
2. Execution
3. Reception
4. Completion
Requests
Studies and administrative documents
Job recording
Warning Notes and Schedule follow up
Worksite supervision and administrative follow up
Work Execution
Reception Sheets
Control and reception of works
Re-works
(if needed)
Acceptance
Control and documentation
Invoicing
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

21. CERN-ITER Workshop – Daniel RICCI (CERN)
Work Organization (3/3)
Cable campaigns generally take place when general services, cable ladders and piping are installed. The user equipment is not yet in place except the racks.
Tunnel Linac4 – end of the cable campaigns
Pulling of RF cables in the Linac4 Hall
The late cable requests will be processed after the installation of the user equipment.
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

22. CERN-ITER Workshop – Daniel RICCI (CERN)
Outline
Compilation and check of needs
Definition of cabling campaigns, overall coordination
QA of cable pulling and assembly of connectors
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

23. Quality Assurance/Control on supplies
Detailed in backup slides
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

24. Training classes and examination
Training classes on connector assembly organized by the contractor
Exams for skill certification organized by CERN
Because skill is a contract requirement … but quality is a CERN obligation w.r.t. users
Practical tests organized at CERN with CERN assessors
1st exam results
Exam class
Simulation of cable pulling for RP area organized by the contractor
In safe area and in similar conditions
Optimized procedures (i.e. time) per cable type
Simulation
Real environment
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

25. CERN-ITER Workshop – Daniel RICCI (CERN)
Pilot projects
Two large pilot projects were organized before LS1 to make the new contractors practicing with installations and specific CERN environment
PS-PSS (~ 3 MCHF)
̴ 160 km copper cables ( ̴ 600 cables)
̴ 456 km optical fibres ( ̴ 400 connectors)
̴ 2 km new cable trays
2) LINAC4 (~ 1.4 MCHF)
̴ 85 km copper cables ( ̴ 2200 cables)
̴ 80 km optical fibres ( ̴ 1000 connectors)
̴ 1 km new cable trays
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

26. CERN-ITER Workshop – Daniel RICCI (CERN)
Work supervision
Project Manager
Respect of safety
Preparation before
the work execution
Work Supervisor
Work progression
of the contractor
Work reception
Implementation of
the modifications
Respect of the work
proceedures
Control of the
quantities installed
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

27. Contractor Performance (1/2)
EXAMPLE
Reception Sheet Analysis for Copper Cabling
Type of non-conformities
S1: Minor non-conformities
S2: Impact on machine running
S3: Impact on running & safety
181 non-conformities in total
(3.6% of total installed components)
Very acceptable
43% non-conformities S2 and S3 happened during connector fitting and test
NOTE: These analysis doesn’t take into consideration the non-conformities addressed by CERN during the work supervision.
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

28. Contractor Performance (2/2)
Non-conformities examples
LHC Underground Consolidation in ARC34: Microduct Cable kink
CCC Surface links in 874: Microduct wrong colour and size
R2E P5: Wrong earth connection
R2E P7: Connector not conform, pin position incorrect
S3 TYPE
S2 TYPE
S3 TYPE
S2 TYPE 1) 2) 3) 4)
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

29. Safety has absolute priority at CERN reponsible for the safety
Technical activities do not start/continue if safety conditions are not respected
Respect of the CERN
equipment present in
the area.
Ensures working in
a safe environment
The Project Manager is
reponsible for the safety
of his actitity.
Chooses/adapts the technical solutions
taking into account the safety constraints
Takes all measures to enforce
safety rules during construction.
The roles are codified by specific safety-instructions (eg.: GSI-WS-1)
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

30. Anticipation is the key
Conclusions
Invest massive efforts in the preparation
Anticipate qualification and procurement of material
Anticipate users’ needs and installations
Consolidate the working method
Review work procedures
Work by geographical campaigns
Boost the contracts
Increase number of contractors
Promote training and organize tests
Assure and Control the Quality
Development of progress monitoring tools
Increase the work supervision on-site
Anticipation is the key
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

31. Thank you

32. Standard implementation of a project Principle
Study of the users’ needs, power database and DC layout
Estimated quantity
of cables
per system
(DIC provisional)
Supply of equipment
Layouts
Infrastructure
Cabling
Calculation of
quantities
Rack list
per user
Size of the cable ladders
Civil Engeneering passages
Cable Installation
Requests
(DIC)
Study phase
Financial
estimation
3D Implantation layout of
racks and Functional Positions
3D Integration of
cable ladders
Definition of the
installation sequences
X- 9 months
Purchase of cable
rack, connector
Cable organization on
cables ladders
Sharing the cabling
activity in campaign
X- 6 months
Cables requests
treatment
Preparation phase
Delivary of
material
Preparation of the
technical instructions
X Installation
Cable ladder
installation
Rack
installation
Installation
phase t
Cabling in
cable campaigns
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

33. The preparation: areas of intervention
Workforce
Section team
Structure, Training
Contractors
New/additional contracts
Technical
Project studies & supply of material
Work anticipation during technical stops
Adaptation of the working method
Cable campaigns, shifts
Planning
Forecasting, Scheduling, Monitoring
For both works and resources
Working with external subjects
Users, Management
Machine coordination, Integration
Safety and Radio Protection
Very broad spectrum
The following focuses on:
Procurement of material
Cabling campaigns
Workload & planning
Monitoring
The preparation is a continuous process which intervened both at the beginning and during the LS1.
This talk focuses of the overall preparation. More specific preparations (contract, execution, ALARA, etc.) are included in the next talks.
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

34. Procurement of material
Anticipate qualification and supply
Early processing of user needs, deadline: Jun 2012  orders placed
All cables and connectors specified: Oct 2012  last orders finalised
Certain types of cables/fibres require up to 6 months to be procured
18+ months for water-cooled cables and special fibres
Example: qualification of
Radiation Resistant Optical Fibres
2 years qualification ( )
Over 20 irradiation tests
1-year production ( )
65 quality assurance tests
1 MCHF procurement contract
2400 km
900km installed during LS1 in LHC-SPS
Concerning the procurement of material we aimed at anticipating the qualification and the supply by early processing the users needs.
In June 2012 all orders could be placed and finalised in October 2012.
An example of long-term qualification relates to the procurement of special radiation resistant optical fibres.
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

35. Contractors’ load curve
Workload & Planning
Anticipate contractor workload and optimize planning time-slots
Optimal shaping of workload curves w.r.t. co-activity constraints
Negotiation of work time-slots with machine coordinators
Contractors’ load curve
Planning « agility »
Prepare to “jump-in”: increase parallelism when time-slots open in the planning
Push back campaigns in radiation areas to benefit of decay (reduce people turnover)
Mini-campaigns in LHC to fit short planning time-slots after the passage of other major projects
The red curve represent the “resource loaded” planning with respect to the March forecast.
Jump-in was prepared by organising “buffer activities” (activitées tampon).
Mini-campaigns were prepared by organising “extra activities” with respect to what original approved. This is an example of how the favourable boundary condition helped to do +10% of works (which have of course to be well prepared in advance by CF!).
Curves do not include contractors’ supervision and coordination staff
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

36. The organisation: staffing
BE5
TE3
EN1
IT1
Staff IC
Staff LD
Fellows
VIAs
COAS
PJAS
EN-EL21
Industrial support contracts 4 2x
Vital requirement:
10 people with sufficient knowledge of the machines
ref. EDMS:
Thanks!
FC section doubled in just 1 year + high turnover (45% departure/arrival rate in )
Challenging technical training and transfer know-how
Management/supervision became critical (esp. second level coordination)
The block diagram shows how the section staff was composed:
EL provided the big chunk with people hired through a full variety of contracts
The leftover was complemented with resources provided through the industrial support contracts
A special mention is for the other groups in various departments who granted a vital requirement
Turnover: we had to replace one person each two during to cover the same function.
The worry is that we have a similar turnover also after LS1 (the total people in the section is lower but their contracts are still too unstable)  we are loosing about half the knowledge and experience acquired in LS1
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

37. CERN-ITER Workshop – Daniel RICCI (CERN)
Contractors
Technical dossiers
PAQ procedures
Tests on prototype installations and pilot projects
Weekly follow up meetings
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

38. Definition of the training
Simulations in like-real environment
Training on CERN specific connectors
Test classes
Training
Addressing for external training
Steering training activities and actions from contractors
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

39. CERN-ITER Workshop – Daniel RICCI (CERN)
Quality Assurance
Test on procurement (fibre optics)
Field tests
Follow up of non conformities
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

40. Quality assurance on supplies
Process intended to ensure that supplies are complying with CERN requirements
The quality of the products installed decisively influence the cabling installation performance
QA developed for CF supplies
Enhanced in case of need
Based on risk analysis
Fully monitored
FC suppliers
One frame contractor
Optical
Fibres
Copper
Cabling
CERN store
Two radiation resistance fibre suppliers
Two Water cooled cables suppliers
External suppliers
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

41. QC on optical fibre supplies
Quality control process highly developed for the optical fibre supplies:
Big variety of products (~2700 different products in the contract) constantly evolving
Complex technology products
CF section responsible of the quality of products delivered to CERN
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

42. QA on fibre supplies: working method
“Feedback loop“ to factory.
CERN-ITER Workshop – Daniel RICCI (CERN)
17 Jan 2017

43. CERN-ITER Workshop – Daniel RICCI (CERN)
QC activities
Tests in the laboratory with special test equipment
microscopes, interferometers, insertion loss/return loss meters, OTDR
Inspections in the field
Inspections at supplier’s premises
stock, factory
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)

44. QA on copper cabling supplies
90% of the products supplied by the CERN store
CERN store acting as interface between CERN and suppliers
Definition of new technical specifications for cable procurement (RIACWG)
Close collaboration with CERN store to implement them
Quality control performed in the field
Enhanced with 1 dedicated Support Function + all work supervisors
Visual and electrical inspections after connector fitting
Product quality and installation quality validated at the same time
Monitoring the cable aging
Keep checking the product quality over the years
Planning in advance replacement campaigns
17 Jan 2017
CERN-ITER Workshop – Daniel RICCI (CERN)