1. Software quality and management 5. Process oriented approaches towards software quality. SPICE model / ISO 15504 standard. Dr. Balla Katalin

2. Software quality and management - 5. 2 Contents 4 Continuous maturity models –What is SPICE? –History of SPICE –Elements of SPICE model –Dimensions of SPICE model Process dimension –Software life cycle processes: ISO 12207 –V-model Maturity dimension – SPICE audit

3. Dr. Balla KatalinSoftware quality and management - 5. 3 Maturity models  Staged models  Look to the entire organisation  In their wiev there is one process with different aspects, characteristics  Deal with managerial and technical processes, the technology used, the organisation itself … 4Continuous models Define maturity levels for certain processes (not for the entire organisation), according to different characteristicsDefine maturity levels for certain processes (not for the entire organisation), according to different characteristics The user of the model can decide about the process for which maturity will be checkedThe user of the model can decide about the process for which maturity will be checked 4„Combined”, integrated models Combine the two approaches, making use of the most useful elements

4. Dr. Balla KatalinSoftware quality and management - 5. 4 What is SPICE? SP ICE 4 S oftware P rocess I mprovement and C apability d E termination 4 http://www.sei.cmu.edu/iso-15504/ http://www.sei.cmu.edu/iso-15504/ 4 http://www.sqi.gu.edu.au/spice/ 4 An international collaborative effort to develop a standard 4 It has been underway (unofficially) since 1990 and officially since June of 1993 4 The (prospective) standard is intended to, among other things, establish a common framework –for expressing the process capability ratings resulting from a 15504- conformant assessment and – to provide a migration path for existing assessment models and methods wishing to become 15504-conformant.

5. Dr. Balla KatalinSoftware quality and management - 5. 5 History of SPICE 4 Approximately 20 countries are actively contributing resources to this effort. The effort is being managed by a working group ISO/IEC JTC1/SC7/WG10 of the JTC1 subcommittee responsible for software engineering standards, ISO/IEC JTC1/SC7. ISO/IEC JTC1/SC7/WG10ISO/IEC JTC1/SC7 4 JTC1 ISO/IEC JTC1 is a joint ISO and IEC technical committee which deals with information technology.ISO/IEC JTC1

6. Dr. Balla KatalinSoftware quality and management - 5. 6 History of SPICE 4 Interest in developing an international standard on software process assessment was sparked by an investigative study sponsored by the U.K. Ministry of Defense (MOD) into methods for assessing the development capability of software suppliers.

7. Dr. Balla KatalinSoftware quality and management - 5. 7 History of SPICE 4 The study identified and reviewed two dozen existing methods already in use and put forth these findings: –There is a general need to supplement reliance of software procurers on ISO 9001. –There is wide support for a software assessment scheme which is in the public domain, widely recognised, and preferably backed by an international standard. –Some organisations using or developing their own assessment schemes have registered interest in supporting a public domain, standardised scheme, in preference to their own schemes. –An initiative to develop such a scheme would be directed toward continuous process and quality improvement matched to business needs. The initiative would be focused around an international standard on process management which would itself provide a framework for a capability assessment scheme supporting both self-improvement for software suppliers and capability determination as a means of evaluating contract risk.

8. Dr. Balla KatalinSoftware quality and management - 5. 8 The SPICE project 4 January 1993: standard development has been approved 4 June 1993: SPICE project was started 4 SPICE goal and scope: –The SPICE (Software Process Improvement Capability dEtermination) project is an ancillary effort staffed primarily by volunteers from around the world. The SPICE project has three goals: Assist the standardization effort in its preparatory stage to develop initial working drafts. Undertake user trials in order to gain early experience data that will form a basis for revision of the published technical reports prior to review as full international standards. Create market awareness and take-up of the evolving standard.

9. Dr. Balla KatalinSoftware quality and management - 5. 9 Actual state of SPICE 4 All parts of the 15504 document set have been approved for publication as Technical Reports; 15504-5 will lag behind the other 8 parts by approximately 6 months. Availability can be determined by contacting the authorized standards agent in one’s country (ANSI for the United States) or ISO. The work of WG10 will now focus on progressing the document set to full international standard status; this work will likely take approximately 24 months. 4 SPICE Trials Around the World –Phase 2 trials have now closed and phase 3 is being planned. –U.S. SPICE Trials –There have been no additional trials reported in the United States since the first one was reported in early 1998. 4 Expression of interest :

10. Dr. Balla KatalinSoftware quality and management - 5. 10 ISO 15504 elements 4 ISO/IEC TR 15504-1:1998 Information technology -- Software process assessment -- Part 1: Concepts and introductory guide ISO/IEC TR 15504-1:1998 4 ISO/IEC 15504-2:2003 Information technology -- Process assessment -- Part 2: Performing an assessment ISO/IEC 15504-2:2003 4 ISO/IEC 15504-2:2003/Cor 1:2004 ISO/IEC 15504-2:2003/Cor 1:2004 4 ISO/IEC 15504-3:2004 Information technology -- Process assessment -- Part 3: Guidance on performing an assessment ISO/IEC 15504-3:2004 4 ISO/IEC 15504-4:2004 Information technology -- Process assessment -- Part 4: Guidance on use for process improvement and process capability determination ISO/IEC 15504-4:2004 4 ISO/IEC TR 15504-5:1999 Information technology -- Software Process Assessment -- Part 5: An assessment model and indicator guidance ISO/IEC TR 15504-5:1999 4 ISO/IEC TR 15504-9:1998 Information technology -- Software process assessment -- Part 9: Vocabulary ISO/IEC TR 15504-9:1998

11. Dr. Balla KatalinSoftware quality and management - 5. 11 What is SPICE/ ISO 15504 good for? 4 It is a framework related to processes and process maturity, summarising the experience of small, medium and large companies worldwide 4 A framework for –Understanding strengths and weaknesses of processes –SPI and measurement of SPI –Helping the customers understand the maturity of their providers 4 Helps in harmonising process auditing models

12. Dr. Balla KatalinSoftware quality and management - 5. 12 Elements of SPICE model 4 Concentrates on individual processes – Input and output –Goal / scope of the process –Activities of the process –Roles and responsibilities –Checkpoints –Process measures –Performance requirements –Competencies needed Process

13. Dr. Balla KatalinSoftware quality and management - 5. 13 SPICE process management 4 Are the activities connected to individual processes performed (at least informally)? 4 Is the process properly planned? 4 Are the activities executed conform to the plan? 4 Is process execution controlled? 4 Is process execution monitored, are corrective actions being taken? 4 Is the process standardised? 4 Is the process understood quantitatively also? 4 Is the process continuously improved?

14. Dr. Balla KatalinSoftware quality and management - 5. 14 SPICE maturity levels 0 1 2 3 5. Optimising 4. Predictable 3. Established 2. Managed 1. Performed 0. Not performed 4 5

15. Dr. Balla KatalinSoftware quality and management - 5. 15 Process maturity Processes Process maturity levels 0. Not performed 1. Performed 2. Managed 3. Established 4. Predictable 5. Optimising Primary Customer related Engineering Supporting Managerial Organisational

16. Dr. Balla KatalinSoftware quality and management - 5. 16 Dimensions of SPICE model

17. Dr. Balla KatalinSoftware quality and management - 5. 17 Process dimension 4 Processes = Software life cycle processes !!!

18. Dr. Balla KatalinSoftware quality and management - 5. 18 Process dimension: Software life cycle processes 4 ISO /IEC 12207: Information Technology - Software life cycle processes –The international standard ISO 12207 is a globally accepted standard for software lifecycle processes. Though not suited for the direct application in a concrete project, it offers a frame that national standards or corresponding process details to be integrated in order to achieve a standard that can be used in an actual case. Furthermore, the ISO 12207 standard includes definitions that can be applied as a basis for a common terminology, even in national standards. 4 „This is the Process for Building Software In the 21st Century! „ http://www.12207.com/

19. Dr. Balla KatalinSoftware quality and management - 5. 19 Process dimension: Software life cycle processes 4 ISO/IEC 12207:1995 Information technology -- Software life cycle processes ISO/IEC 12207:1995 4 ISO/IEC 12207:1995/Amd 1:2002 ISO/IEC 12207:1995/Amd 1:2002 4 ISO/IEC TR 15271:1998 Information technology -- Guide for ISO/IEC 12207 (Software Life Cycle Processes) ISO/IEC TR 15271:1998 4 ISO/IEC TR 16326:1999 Software engineering -- Guide for the application of ISO/IEC 12207 to project management ISO/IEC TR 16326:1999

20. Dr. Balla KatalinSoftware quality and management - 5. 20 Process dimension: Software life cycle processes 4 According to ISO 12207, the following processes are possible to execute during software life cycle :

21. Dr. Balla KatalinSoftware quality and management - 5. 21 Process dimension: V-model 4 /Myers, 1979/ The Art of Software Testing /Myers, 1979/ –The testing cycle has been structured to model the development cycle. 4 /Boehm, 1979/ Guidelines for Verifying and Validating Software Requirements and Design Specifications /Boehm, 1979/ – "V-chart" which shows the context of verification and validation activities throughout the software lifecycle 4 /VM 1997/ V-Model 97, Lifecycle Process Model /VM 1997/ –Lifecycle Process Model -Developing Standard for IT Systems of the Federal Republic of Germany. General Directive No. 250. June 9 –V-model: Regulations setting up all activities, products, and their logical interdependencies during the development and maintenance/modification of systems, realizing the system tasks predominantly by using IT, within the scope o the federal administration. 4 /Sommerville, 1999/ Software Engineering /Sommerville, 1999/ –V&V Process: is a whole life-cycle process. V&V must be applied at each stage in the software process.

22. Dr. Balla KatalinSoftware quality and management - 5. 22 Process dimension: V-model Boehm, 1979/Boehm, 1979/ Guidelines for Verifying and Validating Software Requirements and Design Specifications

23. Dr. Balla KatalinSoftware quality and management - 5. 23 Process dimension: V-model 4 V&V Process: is a whole life-cycle process. V&V must be applied at each stage in the software process. Has two principal objectives: - The discovery of defects in a system. - The assessment of whether or not the system is usable in an operational situation. (Sommerville, 1999/ )Sommerville, 1999/

24. Dr. Balla KatalinSoftware quality and management - 5. 24 Process dimension: Conformity between ISO 12207 and V-model Development Process Activities according to ISO 12207 Activities according to the V- Model Process Implementation PM1.3 - Generation of Project-Specific V-Model PM1.5 - Generation of Preliminary Plan PM1.4 - Toolset Management PM4 - Detailed Planning System Requirements Analysis SD1.2 - Description of Application System SD1.5 - User-Level System Structure QA4 - Product Assessment System Architectural Design SD2.1 - Technical System Design SD2.4 - Allocation of User Requirements SD2.5 - Interface Description SD2.3 - Investigation of Feasibility QA4 - Product Assessment Software Requirements Analysis SD3 - SW/HW Requirements Analysis QA4 - Product Assessment PM6 - Phase Review Software Architectural Design SD4.1 - SW Architecture Design SD4.2 - Design of Internal and External SW Interfaces QA2.1 - Definition of Assessment Methods and Criteria QA4 - Product Assessment PM6 - Phase Review

25. Dr. Balla KatalinSoftware quality and management - 5. 25 Process dimension: Conformity between ISO 12207 and V-model Software Detailed DesignSD5.1 - Description of SW Component/Module/DatabaseSD5.1 - Description of SW Component/Module/Database QA1.2 - Generation the Assessment Plan QA2.1 - Definition of Assessment Methods and Criteria QA4 - Product Assessment PM6 - Phase Review Software Coding and TestingSD6.1 - Coding of SW Modules SD6.2 - Realization of Database QA2.3 - Definition of Test Cases QA2.4 - Generation of Assessment Procedure SD6.3 - Self-Assesment of the SW Module/Database QA4 - Product AssessmentSD6.1 - Coding of SW Modules SD6.2 - Realization of Database QA2.3 - Definition of Test Cases QA2.4 - Generation of Assessment Procedure SD6.3 - Self-Assesment of the SW Module/Database QA4 - Product Assessment Software IntegrationSD4.3 - Specification of SW Integration SD7 - SW Integration QA2.3 - Definition of Test Cases QA2.4 - Generation of Assessment Procedure QA4 - Product Assessment PM6 - Phase ReviewSD4.3 - Specification of SW Integration SD7 - SW Integration QA2.3 - Definition of Test Cases QA2.4 - Generation of Assessment Procedure QA4 - Product Assessment PM6 - Phase Review Software Qualification TestingQA4 - Product Assessment CM2 - Product and Configuration Management CM4.4 - Data Backup PM6 - Phase ReviewQA4 - Product Assessment CM2 - Product and Configuration Management CM4.4 - Data Backup PM6 - Phase Review System IntegrationSD8 - System Integration QA2.3 - Definition of Test Cases QA2.4 - Generation of Assessment Procedure QA4 - Product AssessmentSD8 - System Integration QA2.3 - Definition of Test Cases QA2.4 - Generation of Assessment Procedure QA4 - Product Assessment System Qualification TestingQA4 - Product Assessment CM2 - Product and Configuration Management CM4.4 - Data Backup PM6 - Phase ReviewQA4 - Product Assessment CM2 - Product and Configuration Management CM4.4 - Data Backup PM6 - Phase Review Software InstallationSD9 - Transition to UtilizationSD9 - Transition to Utilization Software Acceptance SupportSD9 - Transition to Utilization PM6 - Phase ReviewSD9 - Transition to Utilization PM6 - Phase Review

26. Dr. Balla KatalinSoftware quality and management - 5. 26 Maturity dimension: process maturity 4 „Generic practices” are used to determine process maturity. If a process is on a certain level of maturity, the generic practices associated to that maturity level should be there. 4 If a process is on a certain maturity level, it must satisfy certain goals and must produce certain deliverables.

27. Dr. Balla KatalinSoftware quality and management - 5. 27 Maturity dimension: Characteristics of SPICE maturity levels 4 Level 1. (performed): –The process is performed, it has no specific characteristics.

28. Dr. Balla KatalinSoftware quality and management - 5. 28 Maturity dimension: Characteristics of SPICE maturity levels 4 Level 2. (managed): 4 2.1. Performance management  Resource needs definition  Process efficiency planning  Implementing the defined activities  Managing execution of the activities 4 2.2. Managing work product attribute  Defining the requirements relative to integrity and quality  Definition of the activities needed  Configuration management of work products  Quality management of work products

29. Dr. Balla KatalinSoftware quality and management - 5. 29 Maturity dimension: Characteristics of SPICE maturity levels 4 Level 3. (established) 4 3.1. Process definition  Standard process definition  Standard process customisation  Institutionalisation of standard process  Feedback to standard process 4 3.2. Resources connected to the process  Definition of human resource competencies  Defining infrastructural requirements of the process  Ensuring the human resources having the required competencies  Ensuring proper infrastructure

30. Dr. Balla KatalinSoftware quality and management - 5. 30 Maturity dimension: Characteristics of SPICE maturity levels 4 Level 4. (predictable) 4 4.1.: Process measurement  Defining process goals and associated metrics  Ensuring appropriate resources and infrastructure  Gathering defined measurement data  Monitoring if process goals are reached 4 4.2. Process control  Defining analysis and control techniques  Ensuring appropriate resources and infrastructure  Analysing existing measurement data  Identifying deviances and taking the appropriate corrective action

31. Dr. Balla KatalinSoftware quality and management - 5. 31 Maturity dimension: Characteristics of SPICE maturity levels 4 Level 5. (optimising): 4 5. 1. Process change  Identification and approval of changes needed in the standard process  Ensuring resources needed for institutionalisation  Institutionalising the approved change  Analysing efficiency of the change introduced 4 5.2. Continuous improvement  Identifying improvement opportunities  Defining institutionalisation strategy  Institutionalising changes in the customised procedure  Analysing efficiency of the change introduced

32. Dr. Balla KatalinSoftware quality and management - 5. 32 SPICE audit 4 Working method –Choosing the processes –Questionnaires –Interviews –Report –Process improvement plan –Registration in database 4 Assessment results: maturity profile for the chosen processes.

33. Dr. Balla KatalinSoftware quality and management - 5. 33 SPICE audit process

34. Dr. Balla KatalinSoftware quality and management - 5. 34 Components of SPICE documents set

35. Dr. Balla KatalinSoftware quality and management - 5. 35 SPICE audit 5. Optimising 4. Predictable 3. Established 2. Managed 1. Performed 0. Not performed ProcessesMaturity levels P1 P2P3

36. Dr. Balla KatalinSoftware quality and management - 5. 36 SPICE - profile Process Requirement management Supplier management Requirement analysis Planning Coding Testing Process maturity A12.1 2.2 3.2...5.2

37. Dr. Balla KatalinSoftware quality and management - 5. 37 SPICE assessment 4 SPI plan is an important result of the assessment

38. Dr. Balla KatalinSoftware quality and management - 5. 38 SPICE assessment - example Level 1.: performed processNPWFRemarks 1.1.Process performance 1.1.1.Basic practices needed for the process are in place.

39. Dr. Balla KatalinSoftware quality and management - 5. 39 SPICE assessment - example Level 2.: Managed processNPWFRemarks 2.1.Performance management 2.1.1.Identifying resource needs 2.1.2.Performance planning 2.1.3.Implementing the defined activities 2.1.4.Managing the execution

40. Dr. Balla KatalinSoftware quality and management - 5. 40 SPICE assessment 4 Evaluation of the results: –Base practice existence based algorithm / or –goal based algorithm –Weighting based on most important drivers

41. Dr. Balla KatalinSoftware quality and management - 5. 41 Goal based algorithm 4 Algorithm for calculating the maturity level : 4 The attributes „fully”, „widely”, „partially” and „not existing” have fixed values 4 T – 1, W – 0.666, P – 0.333, N – 0 -this is valid on level 1, on other levels we have other values !!! 4 According to experience / averages, the results are the following: from„quarter”level 0.00001+0.00 0.19502+0.25 0.39503+0.50 0.59504+0.75 0.7950Next level+ 1

42. Dr. Balla KatalinSoftware quality and management - 5. 42 Goal based algorithm: example 4 Process: configuration management GoalsEvaluationConnected value All project members are using the appropriate version of the productsF1 The elements have their Checked in / out stateF1 It is possible at any time to produce a complete release or a test configuration W0,666 Description of configuration elements is constantly updatedW0,666 Possibility for configuration state report existsN0 Changes can be followedW0,666 Configuration elements are complete, consistent and correctP0,333 Configuration elements are fully transparent, and the degree in which they satisfy the requirements can be followed P0,333 from„quarter”level 0.00001+0.00 0.19502+0.25 0.39503+0.50 0.59504+0.75 0.7950Next level+ 1 Average: 0,583 Result: 0,50

43. Dr. Balla KatalinSoftware quality and management - 5. 43 Taking into account basic „drivers” - example Process Businessdrivers Effect Maturity level Time dependence Product quality Service quality Costs Weight 30 20 Project management 31232,22 Quality assurance 13322,21 Configuration management 22211,81 Risk management 22222,01 Subcontractor management 02010,82 Testing 13111,81 Integration 12211,51 Importance: 1-3 effect= time dependence x 30 + product qual x 30 + process qual x 20 + cost x 20 100

44. Dr. Balla KatalinSoftware quality and management - 5. 44 Choosing development direction 0 1 2 3 4 5 123 Effect Maturity level Areas for improvement

45. Dr. Balla KatalinSoftware quality and management - 5. 45 SPICE assessment results 4 The algorithms are always the basis only. 4 Final decision is to be taken by the auditors, taking into account all circumstances.

46. Dr. Balla KatalinSoftware quality and management - 5. 46 What we talked about... ProductProcess Resource Definition Quality attribute Metric