1. Process Management Process Management in software started in late 1960’s (but informally and inconsistently) Software Engineering Institute (SEI) is the center Watts Humphrey (retired IBMer who joined SEI) was a key proponent of: (in sequential order) –CMM (capability maturity model) Process maturity of organization –PSP (personal software process) Process maturity of individual –CMMI (capability maturity model integrated) Follow-on to CMM

2. A Quick Overview of CMM (How You May Want to “Manage” Change & Certification)

3. Software Engineering Institute (SEI)  Federally Funded Research & Development Center (FFRDC) at Carnegie Mellon University  Mission is to foster improvement of software processes  Developed Capability Maturity Model and Assessment Methodology  Educate “Authorized Lead Assessors” and organizations  Many Assessors have been leading process improvement efforts since 1987

4. Process Management Premise An undefined process cannot be controlled (or measured) An uncontrolled process cannot be improved Attempting to improve an unstable process yields further instability Control Execute Process Define Adapted from SEI Measure Improve start

5. A 5 Step Approach to Software Process Improvement (SPI) IDEAL SM

6. Meeting Your Business (SPI) Goals With (some)Assessor’s Products and Services -SEPG Development & Guidance -Strategic &Tactical Planning -Pilot Projects -Process Templates -Post-Assessment Consulting -Process Implementation Plan -CMM Training -KPA/Process Improvement Training -CBA -IPI (CMM based- internal process improvement assessment) -Executive Briefing -CMM Profiles(Pre-Assessments) -Pre-Assessment Consulting Initiating Diagnosing (Assessing) Establishing Acting Learning -Measurement Programs -Action Planning Workshop -Process Definition

7. Behavioral Characteristics Management Process Engineering Process Quantitative Understanding Continuous Improvement Focus Improvement Framework Capability Maturity Model SM Version 1.1 Disciplined Consistency Quantifiable Predictability Total Organization Involvement Chaos Repeatable - Level 2 Defined - Level 3 Initial - Level 1 Managed - Level 4 Optimizing - Level 5 RISK Productivity and Quality

8. Improvement Framework Organization Process Focus Organization Process Definition Training Program Integrated Software Management Software Product Engineering Intergroup Coordination Peer Reviews Repeatable - Level 2 Defined - Level 3 Initial - Level 1: (no process!) Quantitative Process Management Software Quality Management Defect Prevention Technology Change Mgmt Process Change Mgmt. Managed - Level 4 Optimizing - Level 5 RISK Productivity and Quality Capability Maturity Model SM Version 1.1 Requirements Management Software Project Planning Software Project Tracking & Oversight Subcontract Management Software Quality Assurance Software Configuration Management

9. Why Use The CMM? Provides a framework for: benchmarking the process and the organization CMM is not prescriptive; it does not tell an organization how to improve –Provides good “common sense” engineering and management practices –The key practices do not Limit the choice of life cycle Require or preclude specific software technologies or language (e.g., prototyping, design method, coding or testing practices) Require that documentation conform to any particular set of standards The key practices are the result of “national consensus” –Use of standardized language and goals

10. Early Reported Benefits Results of Software Process Improvement in 13 Organizations CategoryRangeMedian Productivity gain per year9% - 67%35% Early error detection gain per year6% - 25%22% (defects discovered pre-test) Yearly reduction in time to market15% - 23%19% Yearly reduction in post-release10% - 94%39% defect reports Value returned on each dollar invested4.0 - 8.85.0 Source: Software Engineering Institute, 1994 ----- 17 years ago

11. Process Maturity “Spanning the Years” Level 1 Level 2 Level 3 Level 4 Level 5 Legend Level 1 Level 3 1987 - 1991 1998 1987 - 19911998 92% 7% 2% 81% 15% 4% 129 Organizations 951 Organizations There was a 737% increase in the number of organizations assessed from 1987 through 1998

12. Trends in Process Maturity* Organizations Assessed 1987-1998 1987 - 1991 129 1992219 1993317 1994414 1995506 1996632 1997782 1998951 Year Org. Number of Assessments * Extracted from SEI’s Process Maturity Profile of the Software Community Assessments through December 1998 10 (1%) 28 (3%) 143 (15%) 247 (26%) 523 (55%)

13. Maturity Profile by Organization Type * Extracted from SEI’s Process Maturity Profile of the Software Community Assessments through December 1998

14. Assessment Overview CMM Profile: –Informal quantitative evaluation –Excellent mechanism as a interim or pre-assessment –Non-intrusive to organization –Guidance on recommended process improvements –Results not forwarded to SEI –Typically a 3-5 day activity CBA IPI (CMM Based Assessment Internal Process Improvements): –Formal Assessment –Automated approach saving your organization up to 40% time/effort –Collaborative –Guidance on recommended process improvements –Results forwarded to SEI –Typically a 10 day activity

15. CMM SM Profile Kick-Off Meeting Interviews Review Documents Findings Presentation Identify Scope -Findings -Next Steps -Organizational -Project -Implementation -Project Leaders -Middle Managers -Practitioners -SEPG Select Projects Collect Documents Fill Out Questionnaire Process Data = Activities completed prior to on-site visit Legend (start)

16. CBA IPI Process Flow Day 1Day 2 Pre Assessment Period Day 1Day 2Day 3 Kick-Off Meeting SEI Questionnaire Response SEI Questionnaire Trend Analysis Document Review Document Review Document Review Scripting Assessment Questions On-Site Notebook Development Introduction CMM Training* Organizational Overview Assessment Activities Walkthrough Project Selection Finalize Assessment Planning and Logistics Training * Additional CMM training as Required

17. CBA IPI Process Flow On-site Period Day 1Day 2Day 3Day 4Day 5 Project Leader Interviews Functional Area Rep Interviews Consolidate Data Follow-on Interviews (As required) Prepare Draft Findings Present Draft Findings( Pls) Present Draft Findings (FARs) Rate Organization Develop Final Finding Presentation Present Final Findings Executive Session Appraisal Wrap-up Next Steps

18. Assessment of any Key Process Area(KPA) 5 Components of Assessment of any Process: –Commitment : organizational/management commitment –Ability : availability of qualified/skilled resources –Activity : specific functions performed –Measurement : what and how status is recorded & gauged –Verification : what and how are the activities verified This is where the assessors earns his/her “money” !!!

19. More Recent (2002) Data CBA IPIs and SPAs conducted since 1987 through Dec. 2001 and returned to SEI by Jan. 2002 –2164 assessments 1676 CBA IPIs (CMM-Based Assessment- Internal Process Improvement) 488 SPAs (Software Process Assessment) –1638 organizations 8925 projects 456 participating companies 427 reassessed organizations Assessments conducted from 1997 through Dec. 2001 –1158 organizations (38% of which is offshore) 5624 projects 365 participating companies Note: CMM was developed mostly from 1987 to 1997.

20. Types of Organizations by SIC (Standard Industry Classification) code 27.9% - Service Industry (business, engineering,health, etc.) 16.4 % - Manufacturing (electronic,printing,instruments,metal, chemical,etc.) 6.1% - Public Administration 5.9% - Finance, Insurance, etc. 4.5% - Transportation, Utility, etc. 38.0% - off shore (non-SIC coded) 1.2% - others Note

21. Based on 1104 organizations who reported size (employees in software development & maintenance ) 1 to 100 45.9 % 201 to 2000+ 30.3% 101 to 200 23.8%

22. Personal Software Process (championed by W. Humphrey) Understanding the person’s individual –Productivity –Skills –Methodology Keeping track of the person’s individual –Efforts (e.g. requirements, design, coding, testing, etc.) –Quality (errors and fixes) Allows improvements in –Estimation –Task performance

23. CMMI (CMM Integrated) CMMI model is a fairly recent extension/modification of CMM (developed around 2002 and still growing & changing) We are interested in CMMI for Software Engineering. (there are others such as CMMI for Systems Engineering, Supplier Sourcing, and Integrated Product and Process Development) There are 2 models : i) Staged and ii) Continuous –Different levels 5 levels for Staged 6 levels for Continuous –25 major processes grouped into 4 categories: Organizational Project management Engineering Support Category –There are Goals and Practices for Each Process – to be satisfied for both models

24. Levels for Continuous versus Staged Models –A. Continuous model has 6 capability levels where each process moves through these 6 different levels of capability: Level ( 0) Incomplete – not performed or partially performed (1) Performed - perform the basic practice (2) Managed - institutionalized process (performed according to policy) (3) Defined - tailored & institutionalized specifically to the organization (4) Quantitatively Managed – process is quantitatively managed (5) Optimizing – process is continuously improved and adapted –B. Staged model has 5 maturity levels (like traditional CMM) each of which contains different processes that must be mastered : Level (1) Initial (2) Managed (3) Defined (4) Quantified (5) Optimizing

25. Levels for Continuous versus Staged models in CMM I Level 5 Level 4 Level 3 Level 2 Level 1 Level 0 Continuous (Capability Levels) Staged (Maturity Levels) Incomplete Performed Managed Defined Quantitatively Managed Optimizing Quantitatively Managed Initial Managed Defined Optimizing - - -

26. 4 CMMI Process Areas Process Management (5 processes) –Organizational Process Focus –Organizational Process Definition –Organizational Training –Organizational Process Performance –Organizational Innovation and Deployment Project Management (8 processes) –Project planning –Project Monitoring and Control –Supplier Agreement Management –Integrated Project Management –Risk Management –Quantitative Project Management –Integrated Supplier Management –Quantitative Project Management

27. 4 CMMI Process Areas (cont.) Engineering: (6 processes) –Requirements Management –Requirements Development –Technical Solution –Product Integration –Verification –Validation Support: (6 processes) –Configuration Management –Process and Product Quality Assurance –Measurement and Analysis –Organizational Environment for Integration –Decision Analysis and Resolution –Causal Analysis and Resolution

28. Capability Level by Process Areas for Continuous Representation Capability Levels 0 1 2 3 4 5 Process Area 1 Process Area 2 Process Area 3 - - - - - Process Area 25 For Continuous Model, each Process Area is Assessed at its own Level. So, we can compare process by process within an organization.

29. Relationships of Goals and Practices Process Area 1 Process Area 25 - - - - - - - - - - - - - - - - - Specific Goal 1 Specific Goal x - - Specific Practice 1 Specific Practice w - - - Specific Goal 1 - - Specific Goal z Generic Goal 1 Generic Goal 5 - - Generic Practice 1 Generic Practice n Generic Practice 1 - - Specific Practice 1 Specific Practice k Generic Practice p - - -

30. Achieving the “Capability Levels” by each Process Area in the Continuous Representation Model Incomplete Performed Managed Defined Quantitatively Managed Optimizing CL0 CL1 CL2 CL3 CL4 CL5 + (Specific Goals) +(Generic Goal 1) + (Generic Goal 2) + (Generic Goal 3) + (Generic Goal 4) + (Generic Goal 5)

31. Staged Model and Maturity Levels (MLs) ML1:Initial ML2:Managed ML3:Defined ML4:Quantitatively Managed ML5:Optimizing No Process 1. Requirements Mgmt, 2. Project Planning, 3. Configuration Mgmt, 4.Supplier Agreement Mgmt, 5. Project Monitoring & Cntrl, 6. Measurement & Analysis, 7. Process & Product QA 1. Req. Development, 2. Technical solutions, 3. Product Integration, 4. Verification, 5. Validation, 6. Organizational Process Focus, 7. Organiz. Process Definition, 8. Organiz. Training, 9. Integrated Proj. Mgmt, 10. Risk Mgmt, 11.Integrated Teaming, 12. Integrated Supplier Mgmt, 13. Decision Analysis and Resloution, 14. Organiz. Environment for Integration 1.Organizational Process Performance, 2.Quantitative Proj. Mgmt 1.Organizational Innovation & Deploy 2. Causal Analysis & Resolution

32. May Use CMMI for 3 Separate Areas of Interest 1.Product and Service Development (CMMI for Development model) 2. Service Establishment, Management, and Delivery (CMMI for Services model) 3. Product and Service Acquisition (CMMI for Acquisition model)