1. Selecting and Conducting Pilot Projects in a Very Small Entity (VSE)
Department of Software and IT Engineering
Selecting and Conducting Pilot Projects in a Very Small Entity (VSE)
Prepared by Claude Y Laporte, Eng., Ph.D.
Editor – ISO/IEC JTC 1 SC7 - Working Group 24
École de Technologie Supérieure, Canada

2. Overview of the Course
This course will use the ISO/IEC International Standards, Technical Reports and associated Deployment Packages to conduct a pilot project in a VSE.
An introduction to the set of ISO/IEC documents.
An introduction to the set of Deployment Packages
A detailed description of the Deployment Package titled ‘Select and Conduct Pilot Projects’:
Task 1 - Assess the opportunity to conduct a pilot project
Task 2 - Plan the Pilot Project
Task 3 - Conduct the Pilot Project
Task 4 - Evaluate the Results of the Pilot Project
Propose improvements to development process of a VSE
Propose improvements to the Pilot Project Deployment Package
Propose improvements to Standards and Technical Reports
A Very Small Entity (VSE) is defined as an entity (enterprise, organization, department or project) having up to 25 people.

3. Content
Introduction
Set of ISO/IEC Standards and Technical Reports for the Generic Profile Group.
Overview of the set of Deployment Packages
Pilot Projects Deployment Package
Propose improvements
Network of Support Centers

4. The Innovation Process and the Development of the International Standard
Phase 1 – Recognition of Needs and Problems.
Began at a 2004 meeting in Australia of an ISO meeting
Phase 2 – Basic and Applied Research
Survey of Process Improvement Initiatives for VSEs
Survey be conducted of VSEs worldwide
Phase 3 - Development
The Development of International Standards for VSEs.
Phase 4 - Commercialization and phase 5 - Diffusion and Adoption
Development of the Means to Accelerate the Adoption and Utilization of International Standards by VSEs ( e.g. Pilot Projects)
Phase 6 - on Consequences
Future Work and Conclusion.
Source: Rogers 2003

5. Pilot Projects
Definition
A method for exploring the value of a new technological concept via an objective study conducted in a somewhat realistic setting (adapted from Glass 1997).
Successful pilot project is also an effective means of building adoption of new practices by members of a VSE *
To be credible, the pilot projects should satisfy the following requirements (Fenton 1994):
The pilot project experiment has to be designed correctly,
The pilot project has to be performed in a real situation.
It is not a toy project, i.e. an artificial problem in an artificial situation,
The measurements have to be appropriate to the goals of the experiment,
The experiment has to be run for long enough.
* To develop a solid business case to promote the adoption of ISO by VSEs internationally

6. Strategy of WG 24 To develop standards and guidelines for VSEs
Use the notion of ‘Profile’ to develop a roadmap and standards to meet the needs of VSEs.
A profile is an ‘assemblage’ from one or more base standards to accomplish a particular function.
Compliance with ISO/IEC and/or ISO 9001 and assessment should be possible with a minimum of redesign of the VSE’s processes
A Profile Group (PG)
A collection of profiles which are related either by composition of processes (i.e. activities, tasks), or by capability level, or both
Focus first on VSEs developing generic software *

7. The "Generic" Profile Group
Applicable to a vast majority of VSEs that do not develop critical software, commercial off the shelf software (COTS) products, and have typical situational factors.
Does not imply any specific application domain
In the future new domain-specific profiles may be developed
Critical software is defined as software whose failure could have an impact on safety or could cause large financial or social losses (IEEE ).

8. Set of Documents Targeted by Audience
29110 Overview (TR )
29110 ISPs
Framework and Taxonomy (ISP )
Specifications of VSE Profiles (ISP )
Specification - Nnnn VSE Profile
(ISP x)
29110 Guides (TR)
Assessment Guide (TR )
Management and Engineering Guide (TR )
Management and Engineering Guide – Nnnn
VSE Profile
(TR x)
Source: ISO/IEC 29110

9. Set of Documents Targeted by Audience
General Documents (applicable to all Profiles)
Part 1- Overview (Technical Report)
Introduces the major concepts required to understand and use the suite of documents
Part 2- Framework and Profile Taxonomy (Standard)
Specifies the elements common to all profiles (structure, conformance, assessment) and introduces the taxonomy (catalogue) of profiles.
Part 3 -Assessment Guide (Technical Report)
Describes the process to follow to perform an assessment to determinate the process capabilities and the organizational process maturity
Documents for the first Profile (specific to one Profile)
Part 4-1-x -Specifications (Standard)
Provides the composition of a profile, provide normative links to the normative subset of standards
Part 5-1-x Management and Engineering Guide (Technical Report)
Provide guidance on its implementation and use of a profile
Deployment Packages (DP)
Adapted from ISO/IEC 29110

10. ISO/IEC Part 5-1
Part 5-1-x Management and Engineering Guide (Technical Report)
Provides an implementation management and engineering guide for the Nnnn VSE Profile described in ISO/IEC ISP Part 4-x.
Annex A (informative)
Provides the description of the concept of the Deployment Package and a typical table of content
Provides the list of Deployment Packages for the Basic Profile
Adapted from ISO/IEC 29110

11. ISO/IEC 29110 - Part 5 – Table of Contents
Foreword
Introduction
1. Scope
2. Normative references
3. Terms and definitions
4. Basic VSE profile management and engineering guide
4.1 Introduction
4.2 Project Management (PM) process
4.3 Software Implementation (SI) process
4.4 Roles
4.5 Product description
4.6 Software tools requirements
Annex A (informative)– Deployment Package
Bibliography
Source: ISO/IEC 29110

12. Part 5 - Project Management Process – 5 Activities
Stage 1
Stage 2
Stage 3
Stage 4
Source: ISO/IEC 29110

13. Part 5 - Software Implementation – 6 Activities
Stage 1
Stage 2
Stage 3
Stage 4
Source: ISO/IEC 29110

14. Annex A – Informative Deployment Packages
In order to facilitate the implementation, by VSEs, of a Profile, a set of Deployment Packages are available.
Definition of deployment package
Table of content of a deployment package
List of deployment packages for the Basic Profile
Detailed Description of Deployment Packages is Next Topic
Source: ISO/IEC 29110

15. Content
Introduction
Set of ISO/IEC Standards and Technical Reports for the Generic Profile Group.
Deployment Packages
Pilot Projects Deployment Package
Propose improvements
Network of Support Centers

16. Deployment Packages (DPs)
A deployment package is a set of artifacts developed to facilitate the implementation of a set of practices, of the selected framework, in a VSE.
A deployment package is not a complete process reference model. Deployment packages are not intended to preclude or discourage the use of additional guidelines that VSEs find useful.
By deploying and implementing a Deployment Package, a VSE can see its concrete step to achieve or demonstrate coverage to Part 5 *.
Deployment Packages are designed such that a VSE can implement its content, without having to implement the complete framework at the same time.
Each DP is reviewed and edited by 2 persons
Ana Vasquez (Mexico)
Claude Y Laporte (Canada)
* Reference to other standards (e.g. ISO 9001) and Models (e.g. CMMI)
Adapted from ISO/IEC 29110

17. Deployment Packages for the Basic Profile
Verification
and
Validation
Tests
Construction
Architecture
and
Detailed Design
Product
Delivery
Project
Management
Version
Control
Requirements
Analysis
Self-Assessment
Deployment Packages are free !

18. Content of Deployment Packages*
1. Technical Description
Purpose of this document
Why this topic is Important ?
2. Definitions
Generic Terms
Specific Terms
3. Relationships with ISO/IEC Part 5-1
4. Description of Processes, Activities, Tasks, Steps, Roles and Products
5. Template
6. Example
7. Checklist
8. Tool
9. Reference to Standards and Models: ISO 9001, ISO/IEC 12207, CMMI
10. References
11. Evaluation Form
* As defined in Annex A of ISO/IEC Part 5-1

19. Additional Deployment Packages for the Basic Profile
Tool-Based Deployment Packages
Description of a process and a step-by-step description of installation and utilisation of a tool
Version Control
Version Control with CVS
Version Control with SVN
Project Management
Project Management with GForge
Issue Tracking with GForge

20. Content
Introduction
Set of ISO/IEC Standards and Technical Reports for the Generic Profile Group.
Deployment Packages
Pilot Projects Deployment Package
Propose improvements
Network of Support Centers

21. Pilot Project - Roles Pilot Project Coordinator Management of the VSE
The individual who will contact a VSE, explore the possibility of conducting a pilot project with its management, plan and conduct a pilot project and, produce a report of the pilot project.
Management of the VSE
A person, within a VSE, who has the authority to approve and allocate the internal resources the conduct the pilot project and the authority to deploy the technology if the pilot project is successful.
Participant (s) to the Pilot Project
Person (s), under the authority of the management of the VSE, that will be trained and who will participate to the pilot project.

22. Pilot Project Deployment Package – Cover Page
Title of Deployment Package
Select and Conduct Pilot Projects
Notes:
This document is the intellectual propriety of its author’s organization. However, information contained in this document is free of use. The distribution of all or parts of this document is authorized for non commercial use as long as the following legal notice is mentioned:
© École de technologie supérieure and Universidad de la República (Uruguay)
Commercial use of this document is strictly forbidden. This document is distributed in order to enhance exchange of technical and scientific information.
This material is furnished on an “as-is” basis. The author(s) make(s) no warranties of any kind, either expressed or implied, as to any matter including, but not limited to, warranty of fitness for purpose or merchantability, exclusivity, or results obtained from use of the material.
The processes described in this Deployment Package are not intended to preclude or discourage the use of additional processes that a Very Small Entity (VSE) may find useful.

23. Pilot Project Deployment Package
Purpose
To provide tailorable and usable guidelines and materials in order to select and conduct pilot projects in VSEs.
Overview of the Tasks
Task 1 - Assess the opportunity to conduct a pilot project
Task 2 - Plan the Pilot Project
Task 3 - Conduct the Pilot Project
Task 4 - Evaluate the Results of the Pilot Project

24. Task 1 - Assess the Opportunity to Conduct a Pilot Project
Goals
The pilot project coordinator and the management of the VSE assess the opportunity to conduct a pilot project.
If the conclusion of the assessment is positive, the commitment of VSE’s management to conduct the pilot project is obtained.
Steps
Step 1: Sign the Confidentiality Agreement (optional) *
Step 2: Define the characteristics and context of the VSE
Step 3: Define the problem(s) to be addressed
e.g. Perform an assessment using the Self-Assessment DP and tool *
Step 4: Select the technology to pilot
Step 5: Select the project which will pilot the technology
Step 6: Examine the cost and benefits of the pilot project
Step 7: Obtain the commitment of VSE’s management

25. Confidentiality Agreement (optional)

26. Task 1 - Assess the Opportunity to Conduct a Pilot Project
Self-Assessment Deployment Package (DP)
This Deployment Package (DP) supports self-assessment based on the Basic Profile processes as defined in ISO/IEC Part 5-1
Activities related to formal assessments are described in ISO/IEC Assessment Guide
Assessment criteria
Process attribute rating values (from ISO/IEC )
Not achieved/Partially achieved/Largely achieved/Fully achieved
As an example, the result of rating this task could be:
Task
Input
Output
NPLF
Notes
SI.2.1 Assign tasks to the work team members in accordance with their role, based on the current Project Plan.
Project Plan [reviewed] P
Project plan not updated (weakness);
Define roles for project personnel (improvement).
Adapted from Self-Assessment DP

27. Task 1 - Assess the Opportunity to Conduct a Pilot Project
A spreadsheet has been developed to help perform an assessment.

28. Task 2 - Plan the Pilot Project
Goals
Goals for the pilot project are finalized
Evaluation guidelines are identified
A pilot project plan is developed.
The commitments of VSE’s management and the participants to the pilot project to the plan are obtained.
Steps
Step 1: Define success criteria
Step 2: Identify variables to be measured
Step 3: Analyze threats to validity of pilot results and identify variables to control
Step 4: Define the mechanism for doing the evaluation of the pilot
Step 5: Define the stakeholders inputs needed and the means of obtaining them
Step 6: Define ways of gathering data during the pilot.
Step 7: Develop Pilot Project Plan using the Template *
Step 8: Obtain commitment to the plan

29. Pilot Project Plan Template
1. Purpose
2. Acronyms
3. Reference Documents
4. Analysis of opportunity to conduct a pilot project
4.1. Characteristics of the VSE
4.2. Pilot definition
4.3. Context of the VSE (i.e. readiness to conduct pilot project)
5. Plan the Evaluation of the Pilot Project
5.1. Success criteria
5.2. Variables to be measured
5.3. Threats to validity of pilot results
5.4. Other variables to control
5.5. Define the mechanism for doing the evaluation of the pilot
5.6. Procedures for gathering during project execution of data needed for pilot evaluation
6. Project Plan
6.1. Overview
6.2. Description of tasks, dates, effort and artifacts
6.3. Resources
6.4. Interfaces/Dependencies
6.5. Risks

30. Task 3 - Conduct the Pilot Project
Goals
To conduct the pilot according to the plan
To manage risks and adjust the plan if unexpected events happened
Steps
Step 1: Conduct the pilot according to the plan
Collect data ( e.g. effort) *
Step 2: Identify and record problems/issues
Step 3: Solve problems/issues and record solutions
Step 4: Manage pilot project risks
Step 5: Control availability and quality of data
Step 6: Take corrective actions needed to assure availability and quality of data
Step 7: Save data

31. Effort Spreadsheet

32. Task 4 – Evaluate the Results of the Pilot Project
Goals
To evaluate the results of the pilot project
To improve future pilots and the Deployment Packages.*
Steps
Step 1: Identify the benefits of the pilot project
Step 2: Measure costs of conducting the pilot project
Step 3: Evaluate threats to validity of pilot project results
Step 4: Perform a cost/benefit analysis
Step 5: Produce the Pilot Project Report using the template *
Step 6: Store information
* And provide inputs to improve ISO/IEC set of Standards and Technical Reports

33. Pilot Project Report Template
1. Purpose
2. Acronyms
3. Reference Documents
4. Overview
5. Description of tasks, effort and schedule Deviation
6. Resources
7. Interfaces/Dependencies
8. Risks
9. Lessons Learned
Things That Went Well
Things That Could Have Gone Better
Things That Surprised Us

34. Pilot Project Completed 1
Pilot Project in a Computer Aided Design (CAD) Software Support Organisation
Distributes and supports three types of software products:
Computer Aided Design, Computer Aided Manufacturing and Computer Aided Engineering.
Products serve mainly the aerospace and the automobile industries.
Process improvement project lasted about 4 months.
Defined the tasks of 4 developers and undertook to improve the following processes:
Project management
Software configuration management
Issue tracking
Requirements management
Development of a set of step-by-step implementation guides:
Version control practices using an Open source tool (SVN),
Project management practices using GForge,
Requirements management practices using XMLbasedsrs,
Configuration management practices using SVN.

35. Pilot Project Completed 2
Project conducted at a School Board of the Montréal Area
Mission
To provide a stimulating environment for student learning.
It represents 54 primary schools, 14 secondary schools, 2 general training centers and 4 vocational training centers.
Over 8,000 employees. It uses IT to help staff achieve their mandate.
IT department with a staff of 4: 1 analyst and 3 developers.
Project duration: between 2 weeks and 6 months
Methodology
Studied 3 DPs
Software Requirements
Version Control
Project Management
Analysed the DPs with regards to the context of the School Board
Compared the DPs with current way of doing business
Identified improvements needed by the organisation
Added components related OpenUp to DPs selected
Identified gaps from current processes
Prioritized the proposed changes
Developed and presented a set of recommendations

36. Pilot Projects Being Conducted - Canada
Pilot Project 1 - Centre of Excellence in Neuromics of Université de Montréal
Graduate student in Software Engineering
Working for a consulting firm
Development of a software process for a research laboratory
Pilot Project 2 - Gemalto Company
Security and cryptography
Working for the Organisation as Manager of Embedded System Development
Pilot Project 3 (Graduate student in Project Management)
Mise à l'essaie de la future norme ISO/CEI dans des très petites organisations en technologie de l'information,

37. Content
Introduction
Set of ISO/IEC Standards and Technical Reports for the Generic Profile Group.
Deployment Packages
Pilot Projects Deployment Package
Propose improvements
Network of Support Centers

38. Propose Improvements
Propose improvements to development process of a VSE
Propose improvements to the Pilot Project Deployment Packages
Propose improvements to the set of ISO/IEC Standards and Technical Reports

39. Content
Introduction
Set of ISO/IEC Standards and Technical Reports for the Generic Profile Group.
Deployment Packages
Pilot Projects Deployment Package
Propose improvements
Network of Support Centers

40. Network of VSE Support Centers
Objectives
Help accelerate the development of ISO standard for VSEs
Accelerate deployment of VSE Standard
Accelerate the development and application of Deployment Packages
Belgium (Cetic)
Canada (ÉTS)
Colombia (Parquesoft)
Finland
France (UBO)
Ireland (LERO)
Luxembourg (Tudor Research Center)
Mexico
Thailand (Institute of Software Promotion for Industries)
Collaboration Agreement signed between ETS and Members of Network

41. Deployment Packages for ISO/IEC 20000
This standard defines the requirements for an IT service provider to deliver managed services of an acceptable quality for its customer.
Service Management Processes
Source: ISO/IEC Information Technology — Service Management

42. Deployment Packages for ISO/IEC 20000
Source: Kabli

43. Content
Introduction
Set of ISO/IEC Standards and Technical Reports for the Generic Profile Group.
Deployment Packages
Pilot Projects Deployment Package
Propose improvements
Network of Support Centers

44. Bibliography
Fenton, N., Pfleeger, S., Glass, R., Science and Substance: A Challenge to Software Engineers, IEEE Software, Vol. 11, No. 4 July 1994, pp
Glass, R., L., Pilot Studies: What, Why, and How, J. Systems and Software, 1997; 36: 85-97
Rogers, Everett M., Diffusion of Innovations, fifth edition, Free Press, New York, 2003.
ISO/IEC Lifecycle Profiles for Very Small Entities (VSEs) – Part 1: Overview. International Organization for Standardization/International Electrotechnical Commission: Geneva, Switzerland.
ISO/IEC Systems and software engineering - Content of systems and software life cycle process information products (Documentation)
ISO/IEC Information Technology — Service Management, International Organization for Standardization/International Electrotechnical Commission: Geneva Switzerland.
Kabli, Samia, Conception, réalisation et mise à l'essai de trousses de déploiement pour faciliter et accélérer l'implémentation de la norme ISO/CEI par les très petites structures, École de technologie supérieure, 2009.
Laporte, C.Y., Alexandre, S., O’Connor, R., A Software Engineering Lifecycle Standard for Very Small Enterprises, in R.V. O’Connor et al. (Eds.): EuroSPI 2008, CCIS 16, pp. 129–141.