Selecting and Conducting Pilot Projects in a Very Small Entity (VSE) 2018-11-10 Department of Software and IT Engineering Selecting and Conducting Pilot Projects in a Very Small Entity (VSE) Prepared by Claude Y Laporte, Eng., Ph.D. Editor – ISO/IEC JTC 1 SC7 - Working Group 24 École de Technologie Supérieure, Canada

Overview of the Course This course will use the ISO/IEC 29110 International Standards, Technical Reports and associated Deployment Packages to conduct a pilot project in a VSE. An introduction to the set of ISO/IEC 29110 documents. An introduction to the set of Deployment Packages A detailed description of the Deployment Package titled ‘Select and Conduct Pilot Projects’: Task 1 - Assess the opportunity to conduct a pilot project Task 2 - Plan the Pilot Project Task 3 - Conduct the Pilot Project Task 4 - Evaluate the Results of the Pilot Project Propose improvements to development process of a VSE Propose improvements to the Pilot Project Deployment Package Propose improvements to 29110 Standards and Technical Reports A Very Small Entity (VSE) is defined as an entity (enterprise, organization, department or project) having up to 25 people. 2018-11-10

Content Introduction Set of ISO/IEC 29110 Standards and Technical Reports for the Generic Profile Group. Overview of the set of Deployment Packages Pilot Projects Deployment Package Propose improvements Network of Support Centers 2018-11-10

The Innovation Process and the Development of the International Standard Phase 1 – Recognition of Needs and Problems. Began at a 2004 meeting in Australia of an ISO meeting Phase 2 – Basic and Applied Research Survey of Process Improvement Initiatives for VSEs Survey be conducted of VSEs worldwide Phase 3 - Development The Development of International Standards for VSEs. Phase 4 - Commercialization and phase 5 - Diffusion and Adoption Development of the Means to Accelerate the Adoption and Utilization of International Standards by VSEs ( e.g. Pilot Projects) Phase 6 - on Consequences Future Work and Conclusion. Source: Rogers 2003 2018-11-10

Pilot Projects Definition A method for exploring the value of a new technological concept via an objective study conducted in a somewhat realistic setting (adapted from Glass 1997). Successful pilot project is also an effective means of building adoption of new practices by members of a VSE * To be credible, the pilot projects should satisfy the following requirements (Fenton 1994): The pilot project experiment has to be designed correctly, The pilot project has to be performed in a real situation. It is not a toy project, i.e. an artificial problem in an artificial situation, The measurements have to be appropriate to the goals of the experiment, The experiment has to be run for long enough. * To develop a solid business case to promote the adoption of ISO 29110 by VSEs internationally 2018-11-10

Strategy of WG 24 To develop standards and guidelines for VSEs Use the notion of ‘Profile’ to develop a roadmap and standards to meet the needs of VSEs. A profile is an ‘assemblage’ from one or more base standards to accomplish a particular function. Compliance with ISO/IEC 12207 and/or ISO 9001 and assessment should be possible with a minimum of redesign of the VSE’s processes A Profile Group (PG) A collection of profiles which are related either by composition of processes (i.e. activities, tasks), or by capability level, or both Focus first on VSEs developing generic software * 2018-11-10

The "Generic" Profile Group Applicable to a vast majority of VSEs that do not develop critical software, commercial off the shelf software (COTS) products, and have typical situational factors. Does not imply any specific application domain In the future new domain-specific profiles may be developed Critical software is defined as software whose failure could have an impact on safety or could cause large financial or social losses (IEEE 610.12). 2018-11-10

Set of Documents Targeted by Audience 29110 Overview (TR 29110-1) 29110 ISPs Framework and Taxonomy (ISP 29110-2) Specifications of VSE Profiles (ISP 29110-4) Specification - Nnnn VSE Profile (ISP 29110-4-x) 29110 Guides (TR) Assessment Guide (TR 29110-3) Management and Engineering Guide (TR 29110-5) Management and Engineering Guide – Nnnn VSE Profile (TR 29110-5-x) 2018-11-10 Source: ISO/IEC 29110

Set of Documents Targeted by Audience General Documents (applicable to all Profiles) Part 1- Overview (Technical Report) Introduces the major concepts required to understand and use the suite of documents Part 2- Framework and Profile Taxonomy (Standard) Specifies the elements common to all profiles (structure, conformance, assessment) and introduces the taxonomy (catalogue) of 29110 profiles. Part 3 -Assessment Guide (Technical Report) Describes the process to follow to perform an assessment to determinate the process capabilities and the organizational process maturity Documents for the first Profile (specific to one Profile) Part 4-1-x -Specifications (Standard) Provides the composition of a profile, provide normative links to the normative subset of standards Part 5-1-x Management and Engineering Guide (Technical Report) Provide guidance on its implementation and use of a profile Deployment Packages (DP) 2018-11-10 Adapted from ISO/IEC 29110

ISO/IEC 29110 - Part 5-1 Part 5-1-x Management and Engineering Guide (Technical Report) Provides an implementation management and engineering guide for the Nnnn VSE Profile described in ISO/IEC ISP 29110 Part 4-x. Annex A (informative) Provides the description of the concept of the Deployment Package and a typical table of content Provides the list of Deployment Packages for the Basic Profile 2018-11-10 Adapted from ISO/IEC 29110

ISO/IEC 29110 - Part 5 – Table of Contents Foreword Introduction 1. Scope 2. Normative references 3. Terms and definitions 4. Basic VSE profile management and engineering guide 4.1 Introduction 4.2 Project Management (PM) process 4.3 Software Implementation (SI) process 4.4 Roles 4.5 Product description 4.6 Software tools requirements Annex A (informative)– Deployment Package Bibliography Source: ISO/IEC 29110 2018-11-10

Part 5 - Project Management Process – 5 Activities Stage 1 Stage 2 Stage 3 Stage 4 2018-11-10 Source: ISO/IEC 29110

Part 5 - Software Implementation – 6 Activities Stage 1 Stage 2 Stage 3 Stage 4 2018-11-10 Source: ISO/IEC 29110

Annex A – Informative Deployment Packages In order to facilitate the implementation, by VSEs, of a Profile, a set of Deployment Packages are available. Definition of deployment package Table of content of a deployment package List of deployment packages for the Basic Profile Detailed Description of Deployment Packages is Next Topic Source: ISO/IEC 29110 2018-11-10

Content Introduction Set of ISO/IEC 29110 Standards and Technical Reports for the Generic Profile Group. Deployment Packages Pilot Projects Deployment Package Propose improvements Network of Support Centers 2018-11-10

Deployment Packages (DPs) A deployment package is a set of artifacts developed to facilitate the implementation of a set of practices, of the selected framework, in a VSE. A deployment package is not a complete process reference model. Deployment packages are not intended to preclude or discourage the use of additional guidelines that VSEs find useful. By deploying and implementing a Deployment Package, a VSE can see its concrete step to achieve or demonstrate coverage to Part 5 *. Deployment Packages are designed such that a VSE can implement its content, without having to implement the complete framework at the same time. Each DP is reviewed and edited by 2 persons Ana Vasquez (Mexico) Claude Y Laporte (Canada) * Reference to other standards (e.g. ISO 9001) and Models (e.g. CMMI) Adapted from ISO/IEC 29110 2018-11-10

Deployment Packages for the Basic Profile Verification and Validation Tests Construction Architecture and Detailed Design Product Delivery Project Management Version Control Requirements Analysis Self-Assessment Deployment Packages are free ! 2018-11-10

Content of Deployment Packages* 1. Technical Description Purpose of this document Why this topic is Important ? 2. Definitions Generic Terms Specific Terms 3. Relationships with ISO/IEC 29110 Part 5-1 4. Description of Processes, Activities, Tasks, Steps, Roles and Products 5. Template 6. Example 7. Checklist 8. Tool 9. Reference to Standards and Models: ISO 9001, ISO/IEC 12207, CMMI 10. References 11. Evaluation Form * As defined in Annex A of ISO/IEC 29110 - Part 5-1 http://profs.logti.etsmtl.ca/claporte/ 2018-11-10

Additional Deployment Packages for the Basic Profile Tool-Based Deployment Packages Description of a process and a step-by-step description of installation and utilisation of a tool Version Control Version Control with CVS Version Control with SVN Project Management Project Management with GForge Issue Tracking with GForge 2018-11-10

Content Introduction Set of ISO/IEC 29110 Standards and Technical Reports for the Generic Profile Group. Deployment Packages Pilot Projects Deployment Package Propose improvements Network of Support Centers 2018-11-10

Pilot Project - Roles Pilot Project Coordinator Management of the VSE The individual who will contact a VSE, explore the possibility of conducting a pilot project with its management, plan and conduct a pilot project and, produce a report of the pilot project. Management of the VSE A person, within a VSE, who has the authority to approve and allocate the internal resources the conduct the pilot project and the authority to deploy the technology if the pilot project is successful. Participant (s) to the Pilot Project Person (s), under the authority of the management of the VSE, that will be trained and who will participate to the pilot project. 2018-11-10

Pilot Project Deployment Package – Cover Page Title of Deployment Package Select and Conduct Pilot Projects Notes: This document is the intellectual propriety of its author’s organization. However, information contained in this document is free of use. The distribution of all or parts of this document is authorized for non commercial use as long as the following legal notice is mentioned: © École de technologie supérieure and Universidad de la República (Uruguay) Commercial use of this document is strictly forbidden. This document is distributed in order to enhance exchange of technical and scientific information. This material is furnished on an “as-is” basis. The author(s) make(s) no warranties of any kind, either expressed or implied, as to any matter including, but not limited to, warranty of fitness for purpose or merchantability, exclusivity, or results obtained from use of the material. The processes described in this Deployment Package are not intended to preclude or discourage the use of additional processes that a Very Small Entity (VSE) may find useful. 2018-11-10

Pilot Project Deployment Package Purpose To provide tailorable and usable guidelines and materials in order to select and conduct pilot projects in VSEs. Overview of the Tasks Task 1 - Assess the opportunity to conduct a pilot project Task 2 - Plan the Pilot Project Task 3 - Conduct the Pilot Project Task 4 - Evaluate the Results of the Pilot Project 2018-11-10

Task 1 - Assess the Opportunity to Conduct a Pilot Project Goals The pilot project coordinator and the management of the VSE assess the opportunity to conduct a pilot project. If the conclusion of the assessment is positive, the commitment of VSE’s management to conduct the pilot project is obtained. Steps Step 1: Sign the Confidentiality Agreement (optional) * Step 2: Define the characteristics and context of the VSE Step 3: Define the problem(s) to be addressed e.g. Perform an assessment using the Self-Assessment DP and tool * Step 4: Select the technology to pilot Step 5: Select the project which will pilot the technology Step 6: Examine the cost and benefits of the pilot project Step 7: Obtain the commitment of VSE’s management 2018-11-10

Confidentiality Agreement (optional) 2018-11-10

Task 1 - Assess the Opportunity to Conduct a Pilot Project Self-Assessment Deployment Package (DP) This Deployment Package (DP) supports self-assessment based on the Basic Profile processes as defined in ISO/IEC 29110 Part 5-1 Activities related to formal assessments are described in ISO/IEC 29110-3 Assessment Guide Assessment criteria Process attribute rating values (from ISO/IEC 15504-2) Not achieved/Partially achieved/Largely achieved/Fully achieved As an example, the result of rating this task could be: Task Input Output NPLF Notes SI.2.1 Assign tasks to the work team members in accordance with their role, based on the current Project Plan. Project Plan [reviewed] P Project plan not updated (weakness); Define roles for project personnel (improvement). Adapted from Self-Assessment DP 2018-11-10

Task 1 - Assess the Opportunity to Conduct a Pilot Project A spreadsheet has been developed to help perform an assessment. 2018-11-10

Task 2 - Plan the Pilot Project Goals Goals for the pilot project are finalized Evaluation guidelines are identified A pilot project plan is developed. The commitments of VSE’s management and the participants to the pilot project to the plan are obtained. Steps Step 1: Define success criteria Step 2: Identify variables to be measured Step 3: Analyze threats to validity of pilot results and identify variables to control Step 4: Define the mechanism for doing the evaluation of the pilot Step 5: Define the stakeholders inputs needed and the means of obtaining them Step 6: Define ways of gathering data during the pilot. Step 7: Develop Pilot Project Plan using the Template * Step 8: Obtain commitment to the plan 2018-11-10

Pilot Project Plan Template 1. Purpose 2. Acronyms 3. Reference Documents 4. Analysis of opportunity to conduct a pilot project 4.1. Characteristics of the VSE 4.2. Pilot definition 4.3. Context of the VSE (i.e. readiness to conduct pilot project) 5. Plan the Evaluation of the Pilot Project 5.1. Success criteria 5.2. Variables to be measured 5.3. Threats to validity of pilot results 5.4. Other variables to control 5.5. Define the mechanism for doing the evaluation of the pilot 5.6. Procedures for gathering during project execution of data needed for pilot evaluation 6. Project Plan 6.1. Overview 6.2. Description of tasks, dates, effort and artifacts 6.3. Resources 6.4. Interfaces/Dependencies 6.5. Risks 2018-11-10

Task 3 - Conduct the Pilot Project Goals To conduct the pilot according to the plan To manage risks and adjust the plan if unexpected events happened Steps Step 1: Conduct the pilot according to the plan Collect data ( e.g. effort) * Step 2: Identify and record problems/issues Step 3: Solve problems/issues and record solutions Step 4: Manage pilot project risks Step 5: Control availability and quality of data Step 6: Take corrective actions needed to assure availability and quality of data Step 7: Save data 2018-11-10

Effort Spreadsheet 2018-11-10

Task 4 – Evaluate the Results of the Pilot Project Goals To evaluate the results of the pilot project To improve future pilots and the Deployment Packages.* Steps Step 1: Identify the benefits of the pilot project Step 2: Measure costs of conducting the pilot project Step 3: Evaluate threats to validity of pilot project results Step 4: Perform a cost/benefit analysis Step 5: Produce the Pilot Project Report using the template * Step 6: Store information * And provide inputs to improve ISO/IEC 29110 set of Standards and Technical Reports 2018-11-10

Pilot Project Report Template 1. Purpose 2. Acronyms 3. Reference Documents 4. Overview 5. Description of tasks, effort and schedule Deviation 6. Resources 7. Interfaces/Dependencies 8. Risks 9. Lessons Learned Things That Went Well Things That Could Have Gone Better Things That Surprised Us 2018-11-10

Pilot Project Completed 1 Pilot Project in a Computer Aided Design (CAD) Software Support Organisation Distributes and supports three types of software products: Computer Aided Design, Computer Aided Manufacturing and Computer Aided Engineering. Products serve mainly the aerospace and the automobile industries. Process improvement project lasted about 4 months. Defined the tasks of 4 developers and undertook to improve the following processes: Project management Software configuration management Issue tracking Requirements management Development of a set of step-by-step implementation guides: Version control practices using an Open source tool (SVN), Project management practices using GForge, Requirements management practices using XMLbasedsrs, Configuration management practices using SVN. 2018-11-10

Pilot Project Completed 2 Project conducted at a School Board of the Montréal Area Mission To provide a stimulating environment for student learning. It represents 54 primary schools, 14 secondary schools, 2 general training centers and 4 vocational training centers. Over 8,000 employees. It uses IT to help staff achieve their mandate. IT department with a staff of 4: 1 analyst and 3 developers. Project duration: between 2 weeks and 6 months Methodology Studied 3 DPs Software Requirements Version Control Project Management Analysed the DPs with regards to the context of the School Board Compared the DPs with current way of doing business Identified improvements needed by the organisation Added components related OpenUp to DPs selected Identified gaps from current processes Prioritized the proposed changes Developed and presented a set of recommendations 2018-11-10

Pilot Projects Being Conducted - Canada Pilot Project 1 - Centre of Excellence in Neuromics of Université de Montréal Graduate student in Software Engineering Working for a consulting firm Development of a software process for a research laboratory Pilot Project 2 - Gemalto Company Security and cryptography Working for the Organisation as Manager of Embedded System Development Pilot Project 3 (Graduate student in Project Management) Mise à l'essaie de la future norme ISO/CEI 29110 dans des très petites organisations en technologie de l'information, 2018-11-10

Content Introduction Set of ISO/IEC 29110 Standards and Technical Reports for the Generic Profile Group. Deployment Packages Pilot Projects Deployment Package Propose improvements Network of Support Centers 2018-11-10

Propose Improvements Propose improvements to development process of a VSE Propose improvements to the Pilot Project Deployment Packages Propose improvements to the set of ISO/IEC 29110 Standards and Technical Reports 2018-11-10

Content Introduction Set of ISO/IEC 29110 Standards and Technical Reports for the Generic Profile Group. Deployment Packages Pilot Projects Deployment Package Propose improvements Network of Support Centers 2018-11-10

Network of VSE Support Centers Objectives Help accelerate the development of ISO standard for VSEs Accelerate deployment of VSE Standard Accelerate the development and application of Deployment Packages Belgium (Cetic) Canada (ÉTS) Colombia (Parquesoft) Finland France (UBO) Ireland (LERO) Luxembourg (Tudor Research Center) Mexico Thailand (Institute of Software Promotion for Industries) Collaboration Agreement signed between ETS and Members of Network 2018-11-10

Deployment Packages for ISO/IEC 20000 This standard defines the requirements for an IT service provider to deliver managed services of an acceptable quality for its customer. Service Management Processes Source: ISO/IEC 20000 - Information Technology — Service Management 2018-11-10

Deployment Packages for ISO/IEC 20000 http://profs.logti.etsmtl.ca/claporte/ISO20000/index.html Source: Kabli 2018-11-10

Content Introduction Set of ISO/IEC 29110 Standards and Technical Reports for the Generic Profile Group. Deployment Packages Pilot Projects Deployment Package Propose improvements Network of Support Centers 2018-11-10

Bibliography Fenton, N., Pfleeger, S., Glass, R., Science and Substance: A Challenge to Software Engineers, IEEE Software, Vol. 11, No. 4 July 1994, pp. 86-95. Glass, R., L., Pilot Studies: What, Why, and How, J. Systems and Software, 1997; 36: 85-97 Rogers, Everett M., Diffusion of Innovations, fifth edition, Free Press, New York, 2003. ISO/IEC 29110 - Lifecycle Profiles for Very Small Entities (VSEs) – Part 1: Overview. International Organization for Standardization/International Electrotechnical Commission: Geneva, Switzerland. ISO/IEC 15289 Systems and software engineering - Content of systems and software life cycle process information products (Documentation) ISO/IEC 20000- Information Technology — Service Management, International Organization for Standardization/International Electrotechnical Commission: Geneva Switzerland. Kabli, Samia, Conception, réalisation et mise à l'essai de trousses de déploiement pour faciliter et accélérer l'implémentation de la norme ISO/CEI 20000 par les très petites structures, École de technologie supérieure, 2009. Laporte, C.Y., Alexandre, S., O’Connor, R., A Software Engineering Lifecycle Standard for Very Small Enterprises, in R.V. O’Connor et al. (Eds.): EuroSPI 2008, CCIS 16, pp. 129–141. 2018-11-10