The Capability Maturity Model for Software

Software Engineering Institute US DoD funded institute associated with Carnegie Mellon Mission is to promote software technology transfer particularly to defense contractors Work has been very influential in process improvement An important role of the SEI is to use the CMM to assess contractors bidding for US govt defense contracts  Maturity model proposed in mid-1980s, refined in early 1990s.  The model is intended to represent organisational capability not the practices used in particular projects  Within the same organisation, there are often wide variations in processes used  Capability assessment is questionnaire-based with on-site visits

Capability Maturity Model (CMM) Developed by DoD-sponsored Software Engineering Institute (SEI); includes real-world best-practices from hundreds of companies Focus - practices under control of the software group Minimum set of recommended practices that enhance software development and maintenance capability  Defines the expectation (the “what”)  Implements without constraint (the “how”) Used by governments and industries to measure ‘maturity’ of software development organizations …which bridges us into the 5 CMM Maturity Levels…

Level 1 Initial Level 1 Initial Level 2 Repeatable Level 2 Repeatable Level 3 Defined Level 3 Defined Level 4 Managed Level 4 Managed Level 5 Optimizing Level 5 Optimizing CMM Five Maturity Levels Continuous process capability improvement Product quality planning, tracking of measured software process Software process defined and institutionalized to provide product quality control Management oversight and tracking of project; stable planning and product baselines Ad hoc; success depends on heroes

Result Key Process Areas LevelCharacteristic Optimizing Continuous process Process change management (5) capability improvement Technology change mgmt Defect prevention Managed (4) Defined Software process defined (3) and institutionalized to provide product quality control Repeatable (2) Initial (1) Product quality planning; Software quality management tracking of measured Quantitative process management software process Management oversight and tracking of project; stable planning and product baselines Key Process Areas Ad hoc (success depends on heroes) "People" Productivity & Quality Risk Productivity & Quality Software configuration management Software quality assurance Software subcontract management Software project tracking & oversight Software project planning Requirements management Peer reviews Intergroup coordination Software product engineering Integrated software management Training program Organization process definition Organization process focus SEI Capability Maturity Model

1 Time/$/... Target N Probability 2 Time/$/... Target N+a Time/$/... 3 Target N - x Time/$/... 4 Target N - y Time/$/... 5 Schedule and cost targets are typically overrun at Level 1 Plans based on past performance are more realistic at Level 2 but estimates are perceived to be "worse” With well-defined process performance improves at Level 3 Based on quantitative understanding of process and product, performance continues to improve at Level 4 Performance continuously improves at Level 5 Target N - z Actual Why Increase Your Process Maturity?

Level 2 Key Process Areas & Purposes Project-centered focus KPAPURPOSE Requirements Management Establish a common understanding of requirements among the customer and all development groups Software Project PlanningEstablish complete & reasonable project plans Software Subcontractor Management Support selection of qualified subcontractors and effective management of their activities Software Quality Assurance Provide mgmt with visibility into process & products Software ConfigurationEstablish & maintain the integrity management of the software products throughout the life cycle

Level 3 Key Process Areas & Purposes KPAPURPOSE Organization Process FocusTo establish organizational responsibility for software process improvement activities Organization Process DefinitionTo develop and maintain software process assets for the organization Training ProgramTo develop the skills and knowledge of individuals so they can perform their roles effectively and efficiently Integrated Software Management To integrate the software engineering and management activities into a coherent, defined software process for better project management Organization-centered focus

KPAPURPOSE Software Product EngineeringTo integrate all the appropriate software engineering activities to produce correct, consistent software products Intergroup CoordinationTo promote teamwork among all engineering groups on a project, and to better satisfy the customer’s needs Peer ReviewsTo remove defects from the software work product early and efficiently, and reinforce engineering practices and standards Organization-centered focus Level 3 Key Process Areas & Purposes (…continued)

Level 4 Key Process Areas & Purposes KPAPURPOSE Software Quality Management Develop a quantitative understanding of the quality of software products and achieve specific quality goals Quantitative Process Management Control process performance of the software project quantitatively. Focus on identifying special causes of variation within a stable process and correcting the problems that caused the variation to appear. Process-centered focus

Level 5 Key Process Areas & Purposes KPAPURPOSE Process Change Management Continually improve software processes Technology Change Management Identify new and useful technologies and incorporate them into the software process in an orderly manner Defect PreventionIdentify causes of recurring defects, particularly those related to process; prevents such defects from recurring Continuous Software Process Improvement

CMM Pros and Cons Advantages:  The CMM has become a U.S. DoD standard  The CMM widely used internationally  The CMM is well publicized, understood, and supported Lots of training, workshops, and symposia  Process Improvement is Measurable  Many anecdotal reports of benefits

CMM Pros and Cons Disadvantages:  Pressure to “study for the test”  Relatively easy to achieve a given level in name only  Not “modernized” for non-DoD, small software houses  Focuses on project management rather than product development  Ignores the use of technologies like rapid prototyping, risk analysis  Steep learning curve due to immense ambiguity  Overly bureaucratic for a supplier selection model  Encourages/ promotes low maturity (SEI Level 2/ 3)

Organization Payoff SummaryReferences Boeing Info. Systems Project estimates within 20% using historical data, CPK 38% better, defect containment effectiveness at 80%, cycle time improved 36%, staff support needs down 62%, staff size reduced 31%, customer satisfaction score up10%, $5.5M saved in 1996 alone (1992 – 1996 results) Vu, J. (1997)Boeing STSCustomer satisfaction rated excellent, pre-release defect containment effectiveness at 99%, 31% reduction in rework-inspections benefit, employee satisfaction level from mean of 5.7 to 8.3, operational systems performance close to bull's-eye, level 5 process injected into new programs Yamamura, G. & Wigle, G. (1997) BellcoreDefects 10X lower than industry average, customer satisfaction rates improved from 60 to 91% over 4 years, achieved 9 hr. cutover to add 888 to 800 system with no reported defects. Bellcore Press Release, Feb. 5, 1997 HP SESD3X3 SPI program, 1 year benefits include: cycle time reduced by 33%, major open defects reduced from 4.6 to 1.6, fewer missed deadlines, ROI 9:1 Lowe & Cox (1996)Harris ISD DPL 2.5X productivity gain over norm, 90% defect rate reduction, cycle time down to 6-9 monthsRobeson, D., Davidson, S. & Bearden, L. (1997) Motorola3X productivity improvement, 3X cycle time reduction, 7X quality improvement, results from ‘92- ’96 representing 85% of all products & released software, 75% of product development orgs. Are >= level 3 Major, J. (1996)Motorola GEDOn 34 current programs compared to baseline – each CMM level increases quality by 2X, significant decreases in cycle time as higher levels reached (2-7X), productivity increases of 2-3X at highest levels of maturity, 6.77X SPI ROI Diaz, M. & Sligo, J. (1997) SAIC Health Tech. 50% improvement in customer satisfaction, 71% reduction in error rate, 12% annual improvement in developer productivity, production rate up 30% Lane, J. & Zubrow, D. (1997) Resource: Noumena Consulting Group Summary of US Org SPI Payoffs

CMM Costs Example Costs:  Up to 5 to 7 years for solid implementation  US $250,000 to $10,000,000 for initial implementation  US $200,000 per formal external audit  Results in up to 4,000 pages of software processes  Results in over 20,000 pages of formal audit results SEI Response: “Branding” of the CMM: Capability Maturity Model Integration (CMMISM) SW-CMM® Capability Maturity Model for Software P-CMM People Capability Maturity Model SA-CMM Software Acquisition Capability Maturity Model SE-CMM Systems Engineering Capability Maturity Model IPD-CMM Integrated Product Development CMM  Tailored CMM to small organizations  Defined “Team” & “Personal” software processes

CMM: What’s Next? CMM is “sunsetted” (retired)  Training not offered after 2003  Appraisals continuing to 2006 Organizations asked to migrate to CMMI  The “Capability Maturity Model for Integration” Adds KPAs for Measurement (L2), Acquisition (L3) Adds additional capabilities outside the scope of CMM –e.g. outsourcing, acquisition, and risk management Will this fly?  Organizations invested a lot to get CMM levels  Lightweight processes are becoming more popular  CMM will always remain as a historical marker The 1st systemic approach to software process maturity Other standards (ISO-9000) not software specific