1. University of Sunderland CIFM03Lecture 2 1 Quality Management of IT CIFM03 Lecture 2

2. University of Sunderland CIFM03Lecture 2 2 Software Process Assessment - SEI Capability Maturity Model Quality & Information Systems Strategies CIFM03

3. University of Sunderland CIFM03Lecture 2 3 Objectives for this Lecture To identify the concept of software process To introduce the Capability Maturity Model (CMM) and the Software Engineering Institute (SEI) To overview Software Process Improvement To introduce the Capability Maturity Model Integration (CMMI)

4. University of Sunderland CIFM03Lecture 2 4 The Software Process The software process –A structured set of activities required to develop a software system –Specification –Design –Implementation –Validation –Evolution A software process model –an abstract representation of a process. It represents a useful abstracted description of the process, e.g. useful to a developer.

5. University of Sunderland CIFM03Lecture 2 5 What is the CMM?... ‘The CMM is a 5-level model where each maturity level is “a well-defined evolutionary plateau on the path towards becoming a mature software organisation.’ ~ SEI

6. University of Sunderland CIFM03Lecture 2 6 Again, What is the CMM?... ‘The CMM provides a conceptual structure for improving the management and development of software products in a disciplined and consistent way.’ ~ SEI

7. University of Sunderland CIFM03Lecture 2 7 SEI CMM Levels REPEATABLE INITIAL DEFINED MANAGED OPTIMISING discipline standard, consistent predictable continuous improvement project management engineering management quantitative management change management

8. University of Sunderland CIFM03Lecture 2 8 CMM levels 1 - 3 1)Initial. The process is ad hoc/chaotic. Few processes are defined; success depends on highly competent, committed individuals. 2)Repeatable. Basic project management processes are in place to track cost, schedule, and functionality. The discipline is in place to repeat earlier successes on projects with similar applications. 3) Defined. Project management and software engineering activities are integrated into a standard software process for the organization. All projects use an approved, tailored version of it for developing and maintaining software.

9. University of Sunderland CIFM03Lecture 2 9 CMM levels 4 and 5 4) Managed. Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled. 5) Optimising. Continuous process improvement is supported with quantitative feedback from the process and from controlled testing of new ideas and technologies Each maturity level is decomposed into several key process areas that indicate the areas an organization should focus on to improve its software process.

10. University of Sunderland CIFM03Lecture 2 10 The CMM Structure, ( see notes) Maturity Level Key Process Areas Implementation/ institutionalisation Infrastructure/ activities Common Features Key Practices Process capability Goals contain organised by contain indicate achieve address describe

11. University of Sunderland CIFM03Lecture 2 11 Applying the CMM Self assessment possible because the CMM document is detailed, cheaper to do in-house; but suffers from lack of experience/independence Formal assessment independent and experienced, identifies company improvement priorities Software Capability Evaluation to identify qualified contractors

12. University of Sunderland CIFM03Lecture 2 12 Assessment Steps Team selection small, qualified team chosen, which is as independent as possible Maturity questionnaire using a questionnaire, obtain a sample response from departments/projects; the results are analysed for further investigation/clarification Site visit actual departments/projects are visited; interviews conducted, documents reviewed, priority process areas scrutinised Presentation of findings to management TWO parts - assessment of current level of organisation: identify strengths & weaknesses and key improvement areas

13. University of Sunderland CIFM03Lecture 2 13 Critical Evaluation (I)... Best practice – how do we know its best practice, as defined by who, with respect to what type of development, what application area, with what tools? Management and process discipline - do we know what made the software good and how was it produced? Is the CMM maturity scale really ordinal ? Is there an identifiable graded increase in capability at each level? Are the levels stable evolutionary plateaus ? Yes if each lower level is nested in the subsequent level and all processes are continually applied? (L1 in L2?)

14. University of Sunderland CIFM03Lecture 2 14 Critical Evaluation (II)... Scoring approach- CMM employs a multi-hurdle system: questions and ‘key questions’ Sparse data sets - approx. 100 questions to cover everything in a software development process Number vs. Profile - the number of answers is potentially very large and therefore difficult to interpret, however, the profile is enlightening

15. University of Sunderland CIFM03Lecture 2 15 ISO 9001 and CMM compared CMM Specific to software development Used in USA, less widely elsewhere Provides detailed and specific definition of what is required for given levels Assesses on 5 levels CMM Level 3  ISO 9001 Relevant to software development process No specific time limit Ongoing audit ISO 9001 Intended as generic Recognised and accepted in most countries Specifies concepts, principles and safeguards that should be in place Establishes one acceptable level ISO 9001 Stabilises the customer - supplier relationship Certification valid for three years Auditors regularly return for checks during the lifetime of the certificate

16. University of Sunderland CIFM03Lecture 2 16 It focuses on project management rather than product development. It ignores the use of technologies such as rapid prototyping. It does not incorporate risk analysis as a key process area It does not define its domain of applicability The Software Engineering Institute model’s problems

17. University of Sunderland CIFM03Lecture 2 17 SEI Capability Maturity Model Integration (CMMI) A New generation of maturity model: the CMMI, replacing the existing CMM. CMMI covers a broader domain since it addresses not only software but also system aspects. It is also published in 2 representations: staged (cf. CMM) or continuous (cf. ISO15504, a result of the Software Process Improvement and Capability Determination or SPICE project). SPICE is a major international initiative to support the development of an International Standard for Software Process Assessment. Expertise on CMMI is still rare because the transition to CMMI is ongoing. The emphasis is on each process area in the model and on goals and practices associated with each area.

18. University of Sunderland CIFM03Lecture 2 18 SEI CMMI Levels MANAGED INITIAL DEFINED QUANTITATIVELY MANAGED OPTIMISING discipline standard, consistent predictable continuous improvement project management engineering management quantitative management change management

19. University of Sunderland CIFM03Lecture 2 19 A Final Word “There are serious measurement questions... We must ensure that the models are appropriate (see Ch. 2 of Software Metrics).... We must understand how reliable and valid measurements and models are. We must know what entities are being measured, and we must test the relationships between the maturity scores and the behaviours that ‘maturity’ is supposed to produce or enhance.” Fenton & Pfleeger, Software Metrics, Edn. 2, 1997