1. Specification committees for supplies, and quality of the procurement processes in the ATS Bertrand Nicquevert, EN-MEF-QOP EN Dept. Seminar 17 th Septembre 2015 Indico number: 1520670

2. EN buys about 40% of CERN supplies! 20150917B. Nicquevert, EN Dept. Seminar2 ATS GS Rch EN

3. Challenge 20150917B. Nicquevert, EN Dept. Seminar3 How to assure the quality of the supplies procurement and manage the risk Getting what is wanted… (quality) … for the best price… (cost) … in due time! (schedule) How to ensure conformity of the procurement process with the financial, legal and safety rules

4. Overview of process 20150917B. Nicquevert, EN Dept. Seminar4

5. What is this seminar about, and not 20150917B. Nicquevert, EN Dept. Seminar5 There is a free one day training course « Purchasing supplies at CERN » – register! Course code 14TM008PCS01IX Today’s seminar will not cover in depth the topics tackled by this training course It will focus on the way the phase « technical auditing » is organised for the Accelerators & Technology Sector (ATS)

6. Requirements engineering 20150917B. Nicquevert, EN Dept. Seminar6 Don’t be like those blind (wo)men Think it also as an elephant!

7. 20150917B. Nicquevert, EN Dept. Seminar7 What was explained by the user What was understood by the technical officer What was promised by the bidder Tendering phase

8. 20150917B. Nicquevert, EN Dept. Seminar8 What was designed by the contractor Execution of the contract What was agreed at the design review What was tested at the contractor’s premises What was accepted by CERN

9. 20150917B. Nicquevert, EN Dept. Seminar9 Support by the contractor Delivery and installation of the supply Associated documentation After commissioning and safety inspection

10. 20150917B. Nicquevert, EN Dept. Seminar10 Evaluation post-mortem of the contract Cost What was charged to CERN Finance Schedule When was it delivered Quality After real use cases by the final user (one year later)

11. Aim of the technical auditing 20150917B. Nicquevert, EN Dept. Seminar11 Reduce the discrepancy along the lifecycle, from the real need to the delivered supply passing by the expressed need, the required supply, the designed supply, the manufactured supply, the installed supply Review the tendering documents so that this real need is correctly expressed, and the way to get it is sufficiently defined Ensure that the procurement rules are complied with

12. Where is it defined: CERN financial rules 20150917B. Nicquevert, EN Dept. Seminar12 Annex 1 to the Regulations for the Implementation of the CERN Financial Rules « Procurement Rules » §6.4 Technical auditing The purpose of technical auditing is to review carefully the invitation to tender, so as to ensure that the Procurement Rules are complied with.

13. Scope of application 20150917B. Nicquevert, EN Dept. Seminar13 Technical auditing shall be carried out by a specification committee in the framework of technical specification meetings, which shall be held in the following cases: - Invitations to tender of an estimated value above 750 000 CHF in the case of competitive tendering; - Invitations to tender of an estimated value above 200 000 CHF in the case of non-competitive tendering as defined in section 9; - In cases where market surveys, price enquiries and invitations to tender include Research and Development and prototype work for amounts above 50 000 CHF; - If considered necessary by the Head of the Procurement Service or the Head of the user Department, in the case of competitive tendering whose estimated value exceeds 200 000 CHF.

14. Specification committee composition 20150917B. Nicquevert, EN Dept. Seminar14 The specification committee shall be composed of: The Head of the user Department concerned or his designated representative, who shall, except in the case of a service contract, chair and convene the meeting; In the case of a service contract, the Industrial Services Manager or his designated representative, who shall take over the role of chairman and convene the meeting; The Head of the Procurement Service or his designated representative, as well as the representatives of the Procurement Service in charge of the invitation to tender; A representative of the Occupational Health & Safety and Environmental Protection (HSE) Unit; A technical referee with proven competence in the field concerned appointed by the chairman, who shall attend the meeting as an expert from another CERN Department; If considered necessary by the chairman, another CERN representative appointed by the chairman, such as a representative of the Legal Service.

15. ATS implementation 20150917B. Nicquevert, EN Dept. Seminar15 Established by a Memo from the DAT, dated Dec. 2014 Clarifies the mission, scope (e.g. any case >200 kCHF) and organization

16. Specification Committees for ATS 20150917B. Nicquevert, EN Dept. Seminar16 Chairperson: by delegation of DH, nominated by the lead chairman (currently BN) Procurement officer in charge, represents also FP Head Technical officer(s) in charge Depending on context: GL / SL or PL or TC may also attend or comment HSE representative (50% by commenting) Technical referee from another dept Sometimes, legal service of legal adviser from FP, or KT For info, DPO

17. Process model of technical auditing 20150917B. Nicquevert, EN Dept. Seminar17 Roles identified and clarified Four phases Preparation: from start-up meeting with FP to the OK for circulation, with « good enough » documents deemed final Verification: until the Spec Committee meeting, where both content and form are discussed and agreed upon by all stakeholders Validation: check that all changes agreed are correctly implemented, until green light Dispatch: until dispatch to potential bidders

18. Sub-phases 20150917B. Nicquevert, EN Dept. Seminar18

19. Roles and facebook in EN-MEF-QOP 20150917B. Nicquevert, EN Dept. Seminar19 Executive Secretary (S. Chérault): all along the process, support to formatting, coding and circulation of documents, and record Preparation officer (M T Santos Calleja): gives OK for circulation when docs are « good enough » Lead chairman (B. Nicquevert): chairs most SCs or delegates, gives green light for dispatch, overviews all processes Other chairpersons (P. Bonnal, G. Perinić, more may join soon)

20. Detailed model (BPMN) of the process 20150917B. Nicquevert, EN Dept. Seminar20

21. Continuous improvement 20150917B. Nicquevert, EN Dept. Seminar21 Double feedback loop –> thinking further [adapted from Argyris and Schön 1978, Organizational learning]

22. ProLiQS 20150917B. Nicquevert, EN Dept. Seminar22 Procurement Lifecycle Quality Subcommittee (to the Quality Assurance Committee) Mandate: Coordinate the forum of representatives from the various stakeholders of the procurement lifecycle […] in matter of procurement quality, requirements engineering, technical auditing and procurement organizational processes Define and validate processes, role definitions, conventions, standards and templates Identify trends in the field, and promote good practices

23. Lifecycle? 20150917B. Nicquevert, EN Dept. Seminar23

24. 2014-05-15ATS Seminar on Quality — Pierre Bonnal24 Quality? at a glance Quality Planning Quality Assurance Quality Control “ ” I say what I will do Someone checks that it is appropriate I do what I have said I provide evidence of compliance ” “ I also identify defects in the processes and seize the opportunity to improve them Courtesy from Quality Planning Quality Assurance Quality Control Quality Management

25. ProLiQS composition and deliverables 20150917B. Nicquevert, EN Dept. Seminar25 Representatives of all stakeholders of the procurement lifecycle: users from ATS (+GS), FP-PI-AT, DG-LS, HSE, and EN- MEF-QOP Main deliverables for all CERN WoCS: in charge of the content definition, edition and circulation Templates for supplies And also thinking further and catch the trends in the field of requirements engineering

26. 20150917B. Nicquevert, EN Dept. Seminar26 WoCS « Working on the CERN site » Dynamic document Current released 6.0 Current revision under validation 6.6 (7.0 to be released by end of 2015)

27. Purpose of this document Any organization shall set up working rules Contractors and contractors’ personnel shall apply these rules where applicable This document (“Working on the CERN Site”) collects only that rules that may have a cost for contractors = =

28. Table of Contents “Working on the CERN Site” ( Ref. WoCS/Rev.6.0 ) https://edms.cern.ch/file/1155899/last_released/Prestations SiteCERN_WorkingCERNSite.pdf 1. CERN’s “Code of Conduct” 2. Hours, Access, Modular Huts… 3. Safety and Safety Coordination 4. Storage Transport and Handling 5. Utilities, Energy Work Site Services 6. Miscellaneous…

29. Templates 20150917B. Nicquevert, EN Dept. Seminar29 Series of documents to support the preparation by the Technical Officer Available on line for CERN staff from FP site https://procurement.web.ce rn.ch/procurement- templates

30. Useful links 20150917B. Nicquevert, EN Dept. Seminar30 Site of the QAChttp://quality.web.cern.ch/procurementhttp://quality.web.cern.ch/procurement Site of FP- PIhttp://procurement.web.cern.ch/http://procurement.web.cern.ch/

31. Enhancing the Procurement Process A few years agoNow: e-TenderingTomorrow: Requirements Engineering ReqIFSTEP 2014-05-15ATS Seminar on Quality — Pierre Bonnal31 Courtesy from

32. Requirements Engineering 2014-05-15ATS Seminar on Quality — Pierre Bonnal32 Tomorrow: Requirements Engineering ReqIFSTEP Bidder Prepare the bid Analyze the bids Ensure requirements are met Perform the contract Courtesy from

33. Requirements Engineering 2014-05-15ATS Seminar on Quality — Pierre Bonnal33 ~ RMV (Requirements Management & Verification) Structured list of requirements (specification items): Req. ID + category Req. statement (a “shall statement”) Bidder compliance { C, PC, NC, TBD } Req. status { Planned, Qualif. Stage, Accept. Stage } Deviation request(s) and decisions(s) Verification method { T, A, I, ER } Verification level Verification execution document Verification reporting document Verification status { Planned, C, NC } Courtesy from

34. Requirements Engineering 2014-05-15ATS Seminar on Quality — Pierre Bonnal34 Off-the-shelf Requirements Authoring Tool Courtesy from

35. Requirement Engineering ATS Seminar on Quality — Pierre Bonnal35 Guide to the Systems Engineering Body of Knowledge (SEBoK), version 1.2, 2013 (816 pages) http://www.sebokwiki.org/pdf/SEBoKv1.2_full.pdf ISO/IEC 15288:2008, Systems and software engineering – System life cycle processes Get It Right The First Time: Writing Better Requirements, IBM Rational DOORS, 2011 (68 pages), http://publib.boulder.ibm.com/infocenter/rsdp/v1r0m0/topic/com.ibm.help.download.doors.doc/pdf92/get_it_right_the_first_time.pdf http://publib.boulder.ibm.com/infocenter/rsdp/v1r0m0/topic/com.ibm.help.download.doors.doc/pdf92/get_it_right_the_first_time.pdf INCOSE Systems Engineering Handbook - a guide for system life cycle processes and activities, version 3.2.1 January 2011, INCOSE-TP-2003-002-03.2.1 (374 pages) www.incose.org www.incose.org A Glossary of Requirements Engineering Terminology, IREB, v1.5, May 2013 (116 pages) www.ireb.org NASA Systems Engineering Handbook, NASA/SP-2007-6105 Rev1, December 2007 (360 pages) http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20080008301_2008008500.pdf Space Engineering – Verification, European Cooperation for Space Standardization ECSS-E-ST-10-02C, March 2009 (45 pages) www.ecss.nlwww.ecss.nl Courtesy from

36. Practical Need #3 – Rationale 2014-05-15ATS Seminar on Quality — Pierre Bonnal36 High/good quality products/services Low/affordable cost products/services Competitive advantage They can choose their customers Well organized They prefer well organized customers Have adopted or adopting requirements engineering tools Longer term Courtesy from

37. Acknowledgments 20150917B. Nicquevert, EN Dept. Seminar37 EN-MEF-QOP colleagues ProLiQS members All procurement officers with whom we work on almost daily basis, and colleagues from FP-PI EN and ATS management for their support to the technical auditing activities All technical officers (you!) for whom all these efforts are actually made, so that you can really get what you wished at best cost

38. 20150917B. Nicquevert, EN Dept. Seminar38 What was actually wished by the user By the way, would really in-sourcing solve the situation? What was explained by the user