Towards an Enhanced Quality Management Framework for the A&T Sector Pierre Bonnal EN-MEF

ATS Seminar on Quality — Pierre Bonnal2 Agenda What is quality? The needs in matter of quality The responses: The QAC (Quality Assurance Committee) The EN-MEF- QOP section The QMF (Quality Management Framework) editorial project Some key features of the QMF Conclusions

ATS Seminar on Quality — Pierre Bonnal3 Quality Two meanings: 1.“Quality” means those features of products which meet customer needs and thereby provide customer satisfaction; 2.“Quality” means freedom from deficiencies. Joseph M. Juran Quality is the degree to which a set of inherent (existing) characteristics fulfills requirements. ISO 9000:2000 The difficulty in defining quality is to translate future needs of the user into measurable characteristics, so that a product can be designed and turned out to give satisfaction at a price that the user will pay. W. Edwards Deming Efficiency vs. Effectiveness

E.g. Quality at CERN ATS Seminar on Quality — Pierre Bonnal4 Quality of our facilities and equipment Quality of our managerial and operational processes Freedom of deficiencies Stakeholder satisfaction Effectiveness

Quality at a glance ATS Seminar on Quality — Pierre Bonnal5 Quality Planning Quality Assurance Quality Control Quality Management

ATS Seminar on Quality — Pierre Bonnal6 Quality at a glance Quality Planning Quality Assurance Quality Control “ ” I say what I will do Someone checks that it is appropriate I do what I have said I provide evidence of compliance ” “ I also identify defects in the processes and seize the opportunity to improve them

Quality at a glance ATS Seminar on Quality — Pierre Bonnal Policy = strategic level Standards & Processes = tactical level Procedures Records = operational level

ATS Seminar on Quality — Pierre Bonnal8 Quality at a glance Quality Planning vs. Quality Control

ATS Seminar on Quality — Pierre Bonnal9 Roles to assign Tools to use (IT tools, templates, forms…) Processes to follow Quality Planning Standards to apply

ATS Seminar on Quality — Pierre Bonnal10 Quality Planning For releasing documents For managing configuration For managing projects For managing operations For ensuring safety For handling non conformities For managing risks, crises For acquiring More broadly: for decision making Processes to follow

ATS Seminar on Quality — Pierre Bonnal11 Quality Planning Managerial process Engineering process Product-related Process-related Standards to apply

ATS Seminar on Quality — Pierre Bonnal12 V & V (verification & validation) Quality Control Traceability of performed tasks and deliverables Change requests, decisions and implementation Non conforming products and deliverables

The QA Trade-off ATS Seminar on Quality — Pierre Bonnal13 Resource s dedicated to QA Value created by QA

Agenda ATS Seminar on Quality — Pierre Bonnal14 What is quality? The needs in matter of quality The responses: The QAC (Quality Assurance Committee) The EN-MEF- QOP section The QMF (Quality Management Framework) editorial project Some key features of the QMF Conclusions

The Needs – Need # ATS Seminar on Quality — Pierre Bonnal15

The Needs – Need # ATS Seminar on Quality — Pierre Bonnal16 Great IT tools but insufficiently mature processes

Agenda ATS Seminar on Quality — Pierre Bonnal17 What is quality? The needs in matter of quality The responses: The QAC (Quality Assurance Committee) The EN-MEF- QOP section The QMF (Quality Management Framework) editorial project Some key features of the QMF Conclusions

ATS Seminar on Quality — Pierre Bonnal18 The Response from the Management a Body QAC Needs Strategy Steering Validation Trends a Section EN-MEF- QOP Implementation Support Dissemination Verification

QAC  Continuation of QAWG ATS Seminar on Quality — Pierre Bonnal19 1 st meeting in March 1998 Chair: P. Faugeras Sci. Sec.: M. Mottier Marcel Mottier period Elena Manola period Tadeusz Kurtyka period

ATS Seminar on Quality — Pierre Bonnal20 19 th February 2014

Agenda ATS Seminar on Quality — Pierre Bonnal21 What is quality? The needs in matter of quality The responses: The QAC (Quality Assurance Committee) The EN-MEF- QOP section The QMF (Quality Management Framework) editorial project Some key features of the QMF Conclusions

Key Deliverable ATS Seminar on Quality — Pierre Bonnal22 CDR by end of 2014 Modular and Scalable Quality Management and Systems Engineering Framework “Quality Management Framework”

Key Deliverable  QMF ATS Seminar on Quality — Pierre Bonnal Policy = strategic level Standards & Processes = tactical level Procedures Records = operational level

ATS Seminar on Quality — Pierre Bonnal24 Key Deliverable  QMF cern.ch/quality

QMF Editorial Project ATS Seminar on Quality — Pierre Bonnal25 Project Management Methodology Lean

QMF Editorial Project ATS Seminar on Quality — Pierre Bonnal26 QOP drafts documents Documents are discussed in relevant forums + with relevant stakeholders Check Approve QOP & others (via QOP) QAC + additional committees + A&T Mngt (via QOP) Project Roadmap approved by QAC

Agenda ATS Seminar on Quality — Pierre Bonnal27 What is quality? The needs in matter of quality The responses: The QAC (Quality Assurance Committee) The EN-MEF- QOP section The QMF (Quality Management Framework) editorial project Some key features of the QMF Conclusions

ATS Seminar on Quality — Pierre Bonnal28 Practical Need #1 How to enhance the document handling process?

Enhancing the Document Handling ATS Seminar on Quality — Pierre Bonnal29 Authoring Templates Generic documents Circulating Verification = check Validation = approval Identifying Coding/numbering + Versioning Archiving + Retrieving

ATS Seminar on Quality — Pierre Bonnal30 Doct. ID  Should not be problem Legal peopleOne-digit/letter approach:approach: First release: DRAFT1234 – A DRAFT DRAFT n1234 – B DRAFT – C First revision: 1234/DRAFT1234 – D DRAFT 1234/DRAFT – E DRAFT 1234/DRAFT n1234 – F DRAFT 1234/Rev – G Next revisions: 1234/Rev./DRAFT1234 – H DRAFT 1234/Rev./DRAFT – J DRAFT 1234/Rev./DRAFT n1234 – K DRAFT 1234/Rev.2 ··· n1234 – L ··· Z, AA

ATS Seminar on Quality — Pierre Bonnal31 M = major m = minor b = built Doct. ID  Should not be a problem CM M.mCM M.m.b approach:approach: First release: 1234 v. 0.1 DRAFT 1234 v DRAFT 1234 v. 0.2 DRAFT 1234 v DRAFT 1234 v. 1.0 RELEASED 1234 v. 0.1 DRAFT 1234 v. 0.2 DRAFT First revision:1234 v. 1.0 RELEASED 1234 v. 1.1 DRAFT 1234 v. 1.2 DRAFT 1234 v DRAFT 1234 v. 1.3 DRAFT 1234 v DRAFT 1234 v. 2.0 RELEASED 1234 v. 1.1 DRAFT 1234 v. 1.2 DRAFT Next revision:1234 v. 1.3 DRAFT 1234 v. 2.1 DRAFT 1234 v. 2.0 RELEASED 1234 v. 2.2 DRAFT 1234 v. 2.3 DRAFT 1234 v. 3.0 RELEASED

CM M.m ATS Seminar on Quality — Pierre Bonnal32

Enhancing the Document Handling ATS Seminar on Quality — Pierre Bonnal33 Archiving + Retrieving

ATS Seminar on Quality — Pierre Bonnal34 Two Families of Documents Engineering docts. Safety docts. + O&M docts. Scientific publications Documents of personal interest vs. Documents of general interest

The “5 Roles” Author auteur Proofreader correcteur Editor resp. de collection Publisher éditeur Librarian bibliothécaire Authors “Prepared by” Checkers “Checked by” (~ “Verified by”) Approvers “Approved by” (~ “Validated by”) Releaser “Released by” (the Appro Leader) e-Librarian / Cataloguist “Cataloged by” ATS Seminar on Quality — Pierre Bonnal35

Verification vs. Validation from Software Engineering but also applied to documents from Barry W. Boehm (1981) Verification : Are we building the product right? Validation : Are we building the right product? ATS Seminar on Quality — Pierre Bonnal36

The “5 roles” ATS Seminar on Quality — Pierre Bonnal37 RolesAuthorCheckerApproverEditore-Librarian Eng. Docts. Safety Docts. O&M Docts Sci. Pubs.

Better Use of the EDMS ATS Seminar on Quality — Pierre Bonnal40 A&T Technical Documentation Technical Management & Support Studies (on-going + completed) Projects (on-going + completed) Facilities (under-construction + in-operation + dismantled) Items & Assets (under-devt. + in-use + out-of-service) Depts. (department  group  section  person breakdown)

Practical Need #1 – Implementation ATS Seminar on Quality — Pierre Bonnal41 1.Upgrading the “EDMS Starter Pack” to align it with these principles  part of the QMF 2.Refurbishing the EDMS accordingly Some short term results, some medium term ones

ATS Seminar on Quality — Pierre Bonnal42 Practical Need #2 How to enhance the management of projects? Which PM standard to implement? Effectiveness

Which PM Standard or Methodology? ATS Seminar on Quality — Pierre Bonnal43

Which PM Standard or Methodology? ATS Seminar on Quality — Pierre Bonnal44 new product training / HR events PROJECTS computing construction industrial plant organizational new service

ATS Seminar on Quality — Pierre Bonnal45 Which Systems Engineering Standard? The research leading to this framework has received funding from the European Commission under the FP7 ITN project PURESAFE, grant agreement no

= a lifecycle + key deliverables ATS Seminar on Quality — Pierre Bonnal46

Which Projects*? ATS Seminar on Quality — Pierre Bonnal47 New facility-related projects* CERN-wide, sectorial or departmental projects Equipment- and system-related projects* Group-level projects Upgrade- and repair-related projects* incl. upgrade and repair projects submitted to ALARA Organizational process-related projects* * Projects = { studies + development projects + programs }

Practical Need #2 – Implementation ATS Seminar on Quality — Pierre Bonnal48 Four guidelines: Governing principles Facility-related projects* Equipment- and system-related projects* Organizational process-related projects* with insights on: roles, scope, cost, time, quality, risk, procurement Medium term results

ATS Seminar on Quality — Pierre Bonnal49 Practical Need #3 How to enhance the procurement process?

Enhancing the Procurement Process A few years agoNow: e-TenderingTomorrow: Requirements Engineering ReqIFSTEP ATS Seminar on Quality — Pierre Bonnal50

Now: e-Tendering Bidder Prepare the bid Analyze the bids Ensure requirements are met Perform the contract ATS Seminar on Quality — Pierre Bonnal51

Requirements Engineering ATS Seminar on Quality — Pierre Bonnal52 Tomorrow: Requirements Engineering ReqIFSTEP Bidder Prepare the bid Analyze the bids Ensure requirements are met Perform the contract

Requirements Engineering ATS Seminar on Quality — Pierre Bonnal53 ~ RMV (Requirements Management & Verification) Structured list of requirements (specification items): Req. ID + category Req. statement (a “shall statement”) Bidder compliance { C, PC, NC, TBD } Req. status { Planned, Qualif. Stage, Accept. Stage } Deviation request(s) and decisions(s) Verification method { T, A, I, ER } Verification level Verification execution document Verification reporting document Verification status { Planned, C, NC }

Requirements Engineering ATS Seminar on Quality — Pierre Bonnal54 Off-the-shelf Requirements Authoring Tool

Requirement Engineering ATS Seminar on Quality — Pierre Bonnal55 Guide to the Systems Engineering Body of Knowledge (SEBoK), version 1.2, 2013 (816 pages) ISO/IEC 15288:2008, Systems and software engineering – System life cycle processes Get It Right The First Time: Writing Better Requirements, IBM Rational DOORS, 2011 (68 pages), INCOSE Systems Engineering Handbook - a guide for system life cycle processes and activities, version January 2011, INCOSE-TP (374 pages) A Glossary of Requirements Engineering Terminology, IREB, v1.5, May 2013 (116 pages) NASA Systems Engineering Handbook, NASA/SP Rev1, December 2007 (360 pages) Space Engineering – Verification, European Cooperation for Space Standardization ECSS-E-ST-10-02C, March 2009 (45 pages)

Practical Need #3 – Rationale ATS Seminar on Quality — Pierre Bonnal56 High/good quality products/services Low/affordable cost products/services Competitive advantage They can choose their customers Well organized They prefer well organized customers Have adopted or adopting requirements engineering tools Longer term

Agenda ATS Seminar on Quality — Pierre Bonnal57 What is quality? The needs in matter of quality The responses: The QAC (Quality Assurance Committee) The EN-MEF- QOP section The QMF (Quality Management Framework) editorial project Some key features of the QMF Conclusions

Many other quality-related topics! ATS Seminar on Quality — Pierre Bonnal58 Enhancing the management of maintenance Enhancing the management of configuration (incl. items, assets, ECRs, non conforming products…) Enhancing risk-based decision processes Enhancing the use of ERP-PLM tools …

Conclusion (1/3) ATS Seminar on Quality — Pierre Bonnal59 High-end research results Research results at affordable costs Research Funding Agencies Satisfaction = ——— Value Price Lean Quality Management Framework

Conclusion (2/3) ATS Seminar on Quality — Pierre Bonnal60 Quality Management refers to many processes A few addressed in a lean perspective (e.g. Lean Project Management, Lean Systems Engineering) Other not yet proposed in a lean perspective Challenge: creating these lean processes! (e.g. Lean Process Diagramming, Lean Requirements Engineering)

Conclusion (3/3) ATS Seminar on Quality — Pierre Bonnal61 Quality Management implies the use of many IT Tools They shall grow coherently and be correctly interfaced The “plaster strategy” be avoided Regular reengineering of the processes

Thank you for your attention

= a 80-page booklet + a few guidelines ATS Seminar on Quality — Pierre Bonnal64

Topics to be addressed ATS Seminar on Quality — Pierre Bonnal65 Applicable Standards (engineering + Safety) Configuration Management (incl. engineering change) Design, PLM, Item and Asset Management Documentation Management (engineering + Safety) Maintenance Management, Issue Management Operations-related (incl. intervention planning…) Procurement (incl. partnerships and agreements…) Project Management, Program Management Risk and Crisis Management (incl. ALARA) Scientific Publications… Systems Engineering Practices and Management

Quality at a glance ATS Seminar on Quality — Pierre Bonnal :2008  6 documented processes required: Control of documents§ Control of records§ Internal audit§ Control of non conforming products§ 8.3 Corrective actions§ Preventive actions§ 8.5.3