The Capability Maturity Model for Software: An Overview Updated by Jim using ppt 97 to CMML2N3.ppt - 11/2/98 Updated again to CMM2n3v11.ppt - 12/98 Updated again to CMM2n3v12.ppt - 1/99 Updated again to CMM2n3v13.ppt - 4/99 Revised to CMMOverview14.ppt from SME’s 4-TheCMM 3.5 to add Level 4 and 5 - 1/01 Updated to CMM Overview 15.ppt (TR-OPD-01 v1.5) - 3/01 This briefing intended for pilot projects or other software engineers desiring to receive an overview of the CMM. This briefing designed to be given by any SPI agent.

The Problem: too many immature software organizations Good performance is possible - but Requirements often misunderstood, uncontrolled Schedules and budgets frequently missed Progress not measured Product content not tracked or controlled Engineering activities nonstandard, inconsistent Teams not coordinated, not trained Defects proliferate Success depends on “heroes” the Tiger Team” “Just send in “Schedules run everything” “Processes limit my creativity” Many projects are immature - may have to reinvent the wheel each time Processes are improvised by practitioners and management Processes not rigorously followed or enforced Reactionary to crises “Processes don’t help my delivery schedule”

Capability Maturity Model Overview Developed at the DoD-sponsored Software Engineering Institute (SEI) Focuses on practices that are under control of the software group Describes common sense, efficient, proven ways of doing business (which you should already be doing) - not a radical new approach Presents a minimum set of recommended practices that have been shown to enhance a software development and maintenance capability It defines the expectation (the “what”) Without overly constraining the implementation (the “how”) Used by Government and industry around the world to measure maturity of software development organizations Being updated in the SEI’s CMM Integration (CMMI) effort CMM had more to do with Software Project Management than software development. CMM is a model that describes how software engineering practices in an organization evolve - work performed is organized and viewed as a process - the evolution of the process is manages separately Like all models, CMM is abstract, but based on experience. Is DESCRIPTIVE, not PRESCRIPTIVE. “All models are wrong, some models are useful.” - Box’s Rule, George Box (quoted in Crosstalk Jan 97) CMM is not a silver bullet. The CMM is a large living document (show it) - Available on web or from SEPO Is DESCRIPTIVE, not PRESCRIPTIVE No specific tools, methods, or technologies are mandated. No application domains, software technologies, people management CMM describes what to do, not how to do it. Has common sense, efficient, proven ways of keeping whole team going in same direction. This is what you (prog. Mgr) are probably already doing because you learned you had to. CMM is not a radical new approach - it is what you should be doing anyway!

Objectives of the CMM To increase customer satisfaction, by producing products according to plan while simultaneously improving the organization’s capability to produce better products To increase software process maturity, the extent to which processes are explicitly defined, managed, measured, controlled, and effective, by: Establishing basic project management controls Standardizing the organization's software process activities Quantitatively analyzing processes and products for monitoring and control Institutionalizing process improvement First goal from CMM section 1.5: “ The goal of customer satisfaction in TQM is also the goal of the CMM. ...The CMM does not explicitly state that the customer should be satisfied (or delighted) with the software product. The CMM does state that the software supplier should work with the customer to understand the customer’s requirements and should build software products that satisfy the customer’s needs as documented in the requirements allocated to the software..” “The CMM is a framework that describes the key elements of an effective software process. (CMM 1.0) The CMM guides software organizations that want to gain control of their processes for developing and maintaining software and to evolve toward a culture of software engineering and management excellence.” Maturity definition, para 1.3

The CMM’s Five Maturity Levels Optimizing Continuous process capability improvement Level 4 Managed Product quality planning, tracking of measured software process Level 3 Defined Software process defined and institutionalized to provide product quality control (animation: grows from bottom up, one Level per click) A Maturity Level is a well defined evolutionary plateau toward achieving mature software process. Level 1 - Can produce good software (Unpredictable, poorly controlled) the processes that exist are generally those developed (consciously or not) by the individual. Over commitment, chaos, crisis, abandon planned procedures and revert to coding and testing. Level 2 - Basic management controls (Can repeat previously-mastered tasks) basic project management capability is in a place supported by fundamental product assurance disciplines. Process focus is on the project. Earlier successes repeated, use info from previous projects, realistic commitments, track planned vs actual, reqts and work products are baselined and controlled, standards are defined and used, documented. Level 3 - Standard, consistent (Processes characterized, fairly well understood) technical engineering practices are established and integrated with the management practices; the foundation for process discipline is established at the organizational level. Standard process(es) used across whole org, a group assigned responsibility, org-wide training program, tailored to project, org wide database Level 4 - Instrumented (Process measured and controlled) measurement capability established at level 2 and grown in level 3 now matures to the point where it provides the capability to monitor and control product and process quality (at the project level). Org sets quantitative quality goals for products and processes. Level 5 - Optimizing continuously (Focus on PI) measurement capability and process understanding at the point of process performance leads to the ability to identify and make changes to the process during the project. Goal: prevent occurrence of defects, root cause, identify new technologies (tools, methods, processes) and rapidly transition into org, continually improve processes. Level 2 Repeatable Management oversight and tracking of project; stable planning and product baselines Level 1 Initial Ad hoc; success depends on heroes

Process Maturity Benefits Level Process Characteristics Predicted Performance 5 Probability Time / $ Target Process improvement Optimizing is institutionalized Probability Time / $ Target 4 Product and process are Managed quantitatively controlled Software engineering and management processes defined and integrated Probability Target 3 Time / $ Defined Maturity helps the organization predict its ability to meet its goals Initial: Target dates (cost, schedule, quality) are often missed by wide variation Repeatable: Variability of actual results around the target decreases - Target is hit more often but target moves out due to realistic estimating and planning Defined: Variability decreases - Target hits increase - Target begins to move in Less rework Managed: Variability continues to decrease - Targets results improve, development time becomes shorter, productivity and quality increase - Process is managed quantitatively Optimizing: Continuous process improvement - Defect prevention further helps to reduce rework shortening the development Probability Time / $ Target 2 Project management System in place; performance is repeatable Repeatable Probability Time / $ Target 1 Process is informal and ad-hoc; performance is unpredictable Initial

CMM Building Blocks: the Maturity Levels 2 5 3 4 ORGANIZATIONAL PROCESSES QUANTITATIVE MANAGEMENT CONTINUOUS IMPROVEMENT PROJECT Institutionalize process improvement Quantitative analysis of processes and products for monitoring and control Standardize the software process activities for all the organization’s projects Each maturity level builds on the preceding level. Establish basic project management controls

Level 1 - Initial Level 1: Just do it. to produce Activity Results

Level 2 - Repeatable Level 2: Think before you act, and think after you act, just to make sure you did it right. Planning to produce Activity Results input to Evaluation to improve

Level 3 - Defined Level 3: Defined Standards are used for all activities. Planning to produce input to Activity Results input to Standards Evaluation input to to improve

Level 4 - Managed Planning Activity Standards Results Evaluation input to to forecast to produce Activity Standards Results input to input to Evaluation to improve

Level 5 - Optimized Planning Activity Standards Results Evaluation input to to forecast to produce Activity Standards Results input to input to Evaluation continuous improvement to improve

Management Organizational Engineering 5 Optimizing 4 Managed 3 Defined Levels/ Process Categories Management Organizational Engineering 5 Optimizing Technology Change Management Process Change Management Defect Prevention 4 Managed Software Quality Management Quantitative Software Management Organization Process Focus Organization Process Definition Training Program 3 Defined Integrated Software Management Inter-group Coordination Software Product Engineering Peer Reviews 2 Repeatable Requirements Management Software Project Planning Software Project Tracking and Oversight Software Subcontract Management Software Quality Assurance Software Configuration Management 1 Initial Ad Hoc Processes

Results, Needs, and Activities the CMM Supports Results Needs (Actions) Activities (steps) Level 2: Clarify requirements Establish Document plans basic project management Track progress processes Control products Level 3: Standardize processes Standardize the organization’s software process Cultivate teamwork activities Reduce defects Level 4: Analyze Set goals processes & products Manage progress for monitoring, quantitatively control Level 5: Institutionalize Optimize process performance improvement Adapt new technologies KPA Baseline the requirements allocated to software RM Estimate project size, effort, cost, resources SPP Establish project plans, estimates, processes, risks SPP Measure actual progress for timely action SPTO Verify adherence of products and activities to reqts. SQA Identify & control products, changes, problems SCM Select qualified contractors, manage their activities SSM Establish organizational responsibility for SPI OPF Define the org’s best practices, set asset database OPD Establish standards for software engrg. activities SPE Tailor the org’s best practices to projects ISM Get agreement from all on reqts. and commitments IC Develop skills and knowledge of team members TP Identify & remove defects early and efficiently PR Set numeric goals for process performance QPM Set quality goals for the software products SQM Measure the performance of the software processes QPM Measure progress toward product quality goals SQM Identify and eliminate the cause of defects DP Continually improve quality, productivity, cycle times PCM Identify new technologies, transition them to use TCM Automation: Activities/KPA columns appear on mouse click This summarizes the - results wanted (prioritize by impact), - corresponding needs (actions) to achieve the results (Prioritize by urgency), - activities (steps) that cause the needed change (Prioritize by feasibility) - related KPAs

Sample Level 1 Software Organization few processes in place The Organization Top Management Dept. A Dept. B Dept. C Middle Management Div. BB Div. AA Project 4 Project 1 Project 2 Projects Project 3 Animation: figures at bottom appear after one mouse click At Level 1, projects may not have their own set of software engineering and management processes. Yet they might produce good products anyway. Projects may use 1. Heroes to solve problems 2. Primitive methods 3. Rudimentary processes 4. Processes that result in fires - need firefighters/tiger teams Processes

Sample Level 2 Software Organization many processes in place; but they are project-specific The Organization Top Management Dept. A Dept. B Dept. C Middle Management Div. BB Div. AA Project 4 Project 1 Project 2 Projects Project 3 At Level 2, projects may have their own set of software engineering and management processes, but they are not coordinated across the organization. Processes may be lousy or GREAT, but not coordinated. Each group at each level is different. “Stovepipes” Must have policies to guide projects in establishing processes for level 2 Processes

Sample Higher-Level Organization processes based on organization’s Process Asset Library (PAL) The Organization Process Asset Library Approved life cycles Standard processes Tailoring guidelines Process database Related documents SEPO Dept. A Dept. B Dept. C Div. AA Div. BB At level 3, 4, and 5, good practices are shared across the whole organization. An organization-wide SEPG (or SEPO) administers the Process Asset Library (PAL) with standard processes. Each dept. tailors the org’s processes to its own environment and then feedback lessons learned to improve the organization’s standard processes. So Dept A projects (1 and 2) share similar processes, and Dept B’s projects (3 and 4) share similar processes. They are not necessarily all the same because of all the factors that affect a process (remember the tailoring guidelines from the previous section?). We should not imply that each department has a single environment or a single set of processes. It is the application and the project environment that drives the process. There may be commonalities across departments; and differences within a department. The issue is the environments and processes have been tested and approved for application within the organization. Specific project characteristics drive the selection of the processes and the tailoring of those processes. Sponsors expect that when they give work to two different SSC San Diego Departments, the expectation of success is the same. Project 3 Project 4 Project 1 Project 2 Projects Processes

SEI Capability Maturity Model Result Level Characteristic Optimizing Continuous process Process change management (5) capability improvement Technology change management Defect prevention Managed (4) Defined Software process defined (3) and institutionalized to provide product quality control Repeatable (2) Initial (1) Product quality planning; Software quality management tracking of measured Quantitative process management software process Management oversight and tracking project; stable planning and product baselines Key Process Areas Ad hoc (success depends on heroes) "People" Software configuration management Software quality assurance Software subcontract management Software project tracking & oversight Software project planning Requirements management Peer reviews Intergroup coordination Software product engineering Integrated software management Training program Organization process definition Organization process focus Risk Productivity & Quality Levels 2 through 5 are decomposed into KPAs: cluster of related activities that achieve goals for enhancing process capability. CMM describes what to do, not how to do it. Has common sense, efficient, proven ways of keeping whole team going in same direction. This is what you (prog. Mgr) are probably already doing because you learned you had to. CMM is not a radical new approach - it is what you should be doing anyway!

Key Process Area (KPA) Structure Each of the 18 CMM Key Process Areas (KPAs) has: Purpose Goals Common Features: - Commitment to perform: sponsorship and policies - Ability to perform: resources, organization, training - Activities to perform - Measurement of performance - Verification of performance The CMM is a large living document (show it) Available on web or from SEPO Every KPA is documented the same way. Each has a purpose, goals, and something called common features which describe the attributes of that particular process. Those of us using the CMM know that each of the KPAs are oftentimes comprised of multiple lower level processes and procedures, e.g. the KPA that describes how to do project planning has a sub-process on how to do estimations. No specific tools, methods, or technologies are mandated. CMM describes what to do, not how to do it. Has common sense, efficient, proven ways of keeping whole team going in same direction. This is what you (prog. Mgr) are probably already doing because you learned you had to; CMM is not a radical new approach.

Example: Structure of the Software Project Planning KPA Purpose • Establish reasonable plans for software engineering and managing the project Goals • Software estimates are documented and used • Activities and commitments are documented • Groups agree to their commitments Commitments • A project software manager is designated • An organizational policy for planning is followed Abilities • Resources and funding are provided • Training in estimating and planning is provided Activities • Estimate project parameters - size, effort, cost, schedules, risks • Plan software activities - goals, life cycle, processes, standards • Get agreements to commitments - from practitioners, management, others Measurements • Track planning status Verifications • Review plans with management • Conduct independent review of planning Let’s look at one KPA for an example of this structure, the SPP KPA. Every project manager must do some form of planning and every software project manager must have some form of SDP, so the purpose of the SPP KPA is to help a manager “establish reasonable plans for software engineering and managing a project”/ And, every KPA is set up this same, simple to understand, way.

CMM Structure 2 5 4 3 RM PP PT SM QA CM Maturity Levels Key Process Areas RM PP PT SM QA CM Goals Common Features Activities Performed Commitment to Perform Ability to Perform Measurement and Analysis Verifying Implementation Key Practices

How is a Maturity Level determined How is a Maturity Level determined? The Software Capability Evaluation (SCE) Description: A structured CMM-Based Appraisal in which a trained team examines an organization’s current software practices. It consists of interviews, questionnaires, and analysis designed to identify the current process capability. Evaluators: A team of 4-6 SCE-trained people, external or internal to the organization Process: Typically one week of preparation off-site, then one week of on-site interviews and analysis, using the SCE process (see guidelines on SSC San Diego PAL) Results: Comprehensive verbal and written findings of strengths, weaknesses, and areas to improve. Can optionally result in a validated maturity level. And how is a maturity level determined? By an SCE

Some CMM Variants SW-CMM Capability Maturity Model for Software T-CMM Trusted CMM SSE-CMM Systems Security Engineering CMM SE-CMM Systems Engineering CMM P-CMM People CMM SA-CMM Software Acquisition CMM IPD-CMM Integrated Product Development CMM FAA-iCMM FAA’s integrated CMM (SW-CMM, SE-CMM, SA-CMM) CMMI CMM Integration (SW-CMM, SE-CMM, SA-CMM, IPD-CMM) The success of SW-CMM reflected by its many clones EIP project rated Level 2 in TCMM, incorporates SW-CMM SE-CMM to be adopted by SSC SW-CMM provides a framework for improving the degree to which certain important software development processes (e.g., planning, CM) are explicitly defined ,managed, measured, controlled, and the extent to which they are effective. Five maturity levels. T-CMM merges SPI and increased software assurance under trusted software development methodology SSE CMM addresses security engineering, measures security as a distinct discipline that is complementary to other disciplines. SE-CMM describes 18 process areas (e.g. analyze candidate solutions, manage risk) of base practices specific to SE. The base practices are “essential elements of good SE.” Six capability levels from “not performed” (L0) through “continuously improving” (L5). P-CMM describes key elements of managing an organization’s workforce. It places practices known to have significantly improved employee performance onto a 5-level evolutionary path starting from the simplest (e.g., an adequate work environment, training) and progressing to ever more sophisticated (e.g. mentoring, coaching). SA-CMM intended for use in any situation where an organization agrees to provide software to the buyer. Written in language that refers to a contractual relationship between the two. Describes practices in five levels with KPAs. IPD-CMM guides orgs in IPD design, development, appraisal, improvement CMMI - SEI - to develop a CM Integrated Product Suite FAA-iCMM by Federal Aviation Administration

CMM References Capability Maturity Model (CMM) for Software, Version 1.1. CMU/SEI-93-TR-24 & 25 , February 1993 http://sepo.spawar.navy.mil/sepo/CMMinfo.html Benefits of CMM-Based Software Process Improvement: Initial Results. CMU/SEI-94-TR-13, August 1994 http://www.sei.cmu.edu/publications/documents/94.reports/94.tr.013.html A Software Process Framework for the SEI CMM, Handbook. CMU/SEI-94-HB-01, September 1994 http://www.sei.cmu.edu/publications/documents/94.reports/94.hb.001.html Article: The Capability Maturity Model for Software. Mark C. Paulk, Bill Curtis, Mary Beth Chrissis, Charles V. Weber. (in Section 7 of SME Guidebook) http://www.sei.cmu.edu/publications/documents/96.reports/96.ar.cmm.v1.1.html SEI CMM Level 5: For the Right Reasons. George Yamamura and Gary Wigle, Boeing Defense & Space Group. CrossTalk, August 1997. http://www.stsc.hill.af.mil/CrossTalk/1997/aug/seicmm5.html SEPO’s Power Point presentation on “The Capability Maturity Model: an Overview” http://sepo.spawar.navy.mil/sepo/CMMinfo.html Key Process Area flow charts, at http://sepo.spawar.navy.mil/sepo/CMMinfo.html First 4 are SEI docs fourth is read-ahead in SME last 2 are SEPO’s