Paul Sesto, Software Engineering Manager Introduction to the Software Engineering Institute (SEI) and the Capability Maturity Model (CMM) Paul Sesto, Software Engineering Manager 11/16/2018

Class Outline The Software Engineering Institute (SEI) and the Capability Maturity Model (CMM) Importance of SEI & CMM in Industry Frame of Reference Project - IADS Major Components of the CMM Northrop Grumman Implementation of CMM Software Engineering Process Organization (SEPO) Organizational Standard Software Process (OSSP) Software Process Asset Library (SPAL) Software Project Planning Northrop Grumman Commercial 11/16/2018

Evolution of the SEI Questionable Quality of Delivered Products “Software Crisis” Developed over Past Few Decades Questionable Quality of Delivered Products Cost & Schedule Overruns Pervasive Department of Defense Catastrophes Persist Cost Overruns of Software Acquisition Rising by the Millions Schedule Delays of Not Months but Years Multi-Billion-Dollar Systems Don’t Perform as Envisioned Projects Cancelled Due to Poor Software Quality Commercial Industry Suffered Similar Problems Complexity of Software Continues to Increase Industry Leaders Deemed Software Crisis to be a Management Problem and Not a Technical Problem 11/16/2018

What is the SEI? Software Engineering Institute Federally Funded Research & Development Center Sponsored by the U.S. Department of Defense Contract Competitively Awarded to Carnegie Mellon University in 1984 CMU Staff is A Consortium of Government, Industry & Academia SEI Mission “Provide Leadership in Advancing the State of the Practice of Software Engineering to Improve the Quality of Systems that Depend on Software” Visit the SEI at www.sei.cmu.edu 11/16/2018

What is the Capability Maturity Model (CMM) Framework to Help Organizations Improve Their Processes Documented Practices for Software Development Model for Continuous Improvement Guidelines, not Rules, on how to Best Run Programs Evolving to Address Other Standards (ISO-9001) Method for Assessing “Maturity” of an Organization Five Maturity Levels Successive Foundations of Process Improvement Used by DoD to Evaluate Contractors Multiple CMMs Exist Software Development Integrated Product Team Development Systems Engineering Personal Software Process CMMI Initiative to Integrate Multiple Models 11/16/2018

Fundamental Concepts of Process Maturity Software Process: set of activities, methods, practices, and transformations that people employ to develop and maintain software and the associated products. internal SDP SW Product Underlying Premise: quality of a software product is largely determined by the quality of the process used to develop and maintain it. 11/16/2018

Five Levels of Software Maturity Level 1: Initial Process is ad hoc, even chaotic; Success depends on individual effort and heroics. Level 4: Managed Detailed measures of software process & product quality collected; SW process & products are quantitatively understood & controlled. Level 2: Repeatable Basic project mgt processes established; Earlier successes can be repeated on projects with similar applications. Level 5: Optimizing Continuous process improvement enabled Quantitative feedback from the process Piloting innovative ideas & technologies. Level 3: Defined Process for mgt and engineering documented, standardized, integrated at organization level; All projects use version of standard software process. 11/16/2018

Practices at Each Level 2 3 4 5 Process At project level only X At organization level X X X Problems Recognized/corrected as they occur X Anticipated & prevented X Sources understood & eliminated X Common sources understood & eliminated X People Success depends on individual X Training provided X X X X IPTs used X X X Strong teamwork in projects X X Strong teamwork across organization X All involved in process improvement X 11/16/2018

Practices At Each Level 2 3 4 5 Technology Assessment Technology support established, X stable activities Planning & mgt data used by projects X New technologies - evaluated qualitatively X Data collected & used in all defined processes X X X Data systematically shared by projects X X X Data definition & collection standardized X X across organization Data used to understand process X X quantitatively & stabilize it Data used to evaluate & select X process improvements 11/16/2018

Class Outline The Software Engineering Institute (SEI) and the Capability Maturity Model (CMM) Importance of SEI & CMM in Industry Frame of Reference Project - IADS Major Components of the CMM Northrop Grumman Implementation of CMM Software Engineering Process Organization (SEPO) Organizational Standard Software Process (OSSP) Software Process Asset Library (SPAL) Software Project Planning Northrop Grumman Commercial 11/16/2018

Benefits of Software Process Maturity 11/16/2018

Why Bother With CMM Certification? Level 5 - 1.2% LM - 3 Sites NASA Ames Boeing (Space Shuttle) Motorola (India) STSC (Hill AFB) Level 4 - 5.4% NG ESSS NG MASD NG ESID Hughes/Raytheon LMTAS Initial - 51.2% Level 2 - 24.8% 76% of Companies are at Levels 1 & 2 Level 3 - 17.4% TRW Logicon Many USAF/Army Sites Level 4 is necessary to remain a leader in today’s market! 11/16/2018

CMM Level 3 is a DOD Requirement SEI Level 1 2 3 4 5 % Of Assessments 11/16/2018

Class Outline The Software Engineering Institute (SEI) and the Capability Maturity Model (CMM) Importance of SEI & CMM in Industry Frame of Reference Project - IADS Major Components of the CMM Northrop Grumman Implementation of CMM Software Engineering Process Organization (SEPO) Organizational Standard Software Process (OSSP) Software Process Asset Library (SPAL) Software Project Planning Northrop Grumman Commercial 11/16/2018

IADS - The Real World Air Defense Requires Centralized Facility Integrates Country Wide Infrastructure Data & Voice Communications Network of Workstations & PCs Distributed Architecture & Displays 1.5 Million Lines of Software Written in Ada95, Java, PWB & SQL Object Oriented Design & Development 85 Software Engineers at Peak on 15 Integrated Product Teams Started with a Specification and Statement of Work of 600 Pages Standard Processes & Procedures Facilitated On Target Completion 11/16/2018

Class Outline The Software Engineering Institute (SEI) and the Capability Maturity Model (CMM) Importance of SEI & CMM in Industry Frame of Reference Project - IADS Major Components of the CMM Northrop Grumman Implementation of CMM Software Engineering Process Organization (SEPO) Organizational Standard Software Process (OSSP) Software Process Asset Library (SPAL) Software Project Planning Northrop Grumman Commercial 11/16/2018

Institutionalization Structure of the CMM Maturity Levels Indicate Process Capability Key Process Areas Goals Contain Achieve 18 Total KPAs Organized by Common Features Implementation or Institutionalization Address 5 Per KPA Key Practices Activities or Infrastructure Contain Describe 11/16/2018

CMM Defined Key Process Areas (KPAs) Continuous improvement Defect prevention Technology change management Process change management 5. Optimizing Measures collected Process/products quantitatively understood & controlled Quantitative process management SW quality management 4. Managed Organization process focus Organization process definition Training program Integrated software management SW product engineering Intergroup coordination Peer reviews 3. Defined Process standardized & integrated at organ. level Requirements management SW project planning SW project tracking & oversight SW subcontract management SW quality assurance SW configuration management 2. Repeatable Basic project mgt processes 1. Initial Success depends on individual effort 11/16/2018

Each KPA Has 5 Common Features 1. Commitment to Perform Written organizational policy required; addresses every KPA ESSS: Software Process Manual 2. Ability to Perform Adequate resources/training required for every KPA ESSS: WSAs at organization and project level 3. Activities Performed Activities, roles, procedures necessary to implement a KPA ESSS: management plans, procedures, technical directives 11/16/2018

Each KPA Has 5 Common Features 4. Measurement and Analysis Basic measurement practices necessary to determine status related to the process ESSS: metrics (technical & management), Level 4 Software Quantitative Management 5. Verifying Implementation Steps to ensure that the activities are performed in compliance with the process that has been established. ESSS: Project and senior management reviews, software quality audits, SEI and ISO audits, reports to SEPO 11/16/2018

KPA Example - Peer Reviews Peer review activities are planned Defects in the software work products are identified and removed Maturity Levels Indicate Process Capability Key Process Areas Goals Contain Achieve Organized by Common Features Implementation or Institutionalization Address Key Practices Activities or Infrastructure Describe 11/16/2018

Peer Review - Key Practices 1. Commitment to Perform The project follows a written organizational policy for performing peer reviews. Work Product Inspections ESSS 2. Ability to Perform Adequate resources and funding are provided for performing peer reviews on each software work product to be reviewed. Peer review leaders receive required training in how to lead peer reviews. Reviewers who participate in peer reviews receive required training in the objectives, principles, and methods of peer reviews. WSAs Training 11/16/2018

Peer Review - Key Practices EAR Tool Plans, Minutes 3. Activities Performed Peer reviews are planned, and the plans are documented. Peer reviews are performed according to a documented procedure. Data on the conduct and results of the peer reviews are recorded 4. Measurement and Analysis Measurements are made and used to determine the status of the peer review activities Metrics 5. Verifying Implementation The SW Quality Group reviews and/or audits the activities and work products for peer reviews and reports the results. QA 11/16/2018

KPA Summary KPAs Describe What Key Process Area Goals Common Features Commitment to Perform Ability to Perform Activities Performed Measurement and Analysis Verifying Implementation KPAs Describe What is to be Done to Satisfy the Desired Goals During an Assessment Each KPA is Evaluated for Objective Evidence of the Common Features for Organizational Institutionalization 11/16/2018

Class Outline The Software Engineering Institute (SEI) and the Capability Maturity Model (CMM) Importance of SEI & CMM in Industry Frame of Reference Project - IADS Major Components of the CMM Northrop Grumman Implementation of CMM Software Engineering Process Organization (SEPO) Organizational Standard Software Process (OSSP) Software Process Asset Library (SPAL) Software Project Planning Northrop Grumman Commercial 11/16/2018

Software Engineering Process Organization SEPO is the Enterprise Governing Organization Provides Project Support Software Project Planning Process Definition Process Improvement Artifact Collection 11/16/2018

Process Definition & Project Planning Management Plans Technical Directives Resource Estimates Project’s Comprehensive Program Plan (CPP) Internal Software Development Plan (iSDP) Software Process Group Process Training SEPO Representative Organization’s Standard Software Process Integrated Dev. Plan Policies Procedures Templates Command Media CPP Template Integrated Dev. Plan Project Planning Tailor Standard Process Project Web Page 11/16/2018

SEPO Provides Process Improvement Integrated Dev. Plan Project’s Internal Software Development Plan (iSDP) Software Process Group / PICT Policies Procedures Templates Management Plans Technical Directives Resource Estimates Planning and Process Improvement Development / PICT Team Periodic Process Evaluation Analyze Metrics Develop and Improve Process Process Improvements Project Command Media Web Page Updates Organization’s Standard Process Pilots 11/16/2018

Organizational Standard Software Process OSSP Facilitates Organizational Standard Processes Accessible via Corporate Intranet Web-Site Supports SEI Level-4 Objectives Promotes Standardization Increases Software Productivity & Quality 11/16/2018

Software Process Asset Library (SPAL) Policies Procedures Compliance Checklists iSDP Generation Tailoring Process Training 11/16/2018

Software Project Planning is Simplified Internal SDP 1. Project IPT Org 2. System Definition (for planning) 3. Project Life Cycle 4. Management Plans 5. Technical Directives 6. Size, Cost, Schedule Estimates 7. Approvals Project’s Comprehensive Program Plan Organization Standard Software Process Software Policies and Procedures Metrics/ Training Databases Tool Standards Command Media Project Web Page Organization Standard Process Assets Project Planning Process . . . When Projects Tailor from the Standard Process 11/16/2018

Templates Exist for Each Plan iSDP IPT Organization Plan Ref. Development Directives Ref. Risk Management Plan Project Life Cycle Definition Requirements Management Plan Project Indicators Risk Management Plan S/SEE Implementation Plan Software Maintenance Plan Software Reuse Plan Subcontract Management Plan Software CM Plan Software QA Plan Training Plan Development and Review Directives Estimation Methods iSDP Compliance and Approval ESSS SPAL Standard Procedures Planning Step 1 - Develop Project IPT Operations Plan Planning Step 2 - Define System to Support Planning Planning Step 3 - Define Project Life Cycle Planning Step 4 - Define Management Plans Planning Step 5 - Define Project Directives Planning Step 6 - Estimate Scope and Resources Planning Step 7 - Review and Deliver Planning Products 11/16/2018

. . . Reduces Software Development Costs Standard Tool Usage . . . iSDP IPT Organization Plan Ref. Development Directives Ref. Risk Management Plan Project Life Cycle Definition Requirements Management Plan Project Indicators Risk Management Plan S/SEE Implementation Plan Software Maintenance Plan Software Reuse Plan Subcontract Management Plan Software CM Plan Software QA Plan Training Plan Development and Review Directives Estimation Methods iSDP Compliance and Approval ESSS SPAL Standard Procedures Planning Step 1 - Develop Project IPT Operations Plan Planning Step 2 - Define System to Support Planning Planning Step 3 - Define Project Life Cycle Planning Step 4 - Define Management Plans Planning Step 5 - Define Project Directives Planning Step 6 - Estimate Scope and Resources Planning Step 7 - Review and Deliver Planning Products Supports Standard Processes Reduces Tool Licensing Costs Reduces Tool Training & Support Costs Increase Software Reuse . . . Reduces Software Development Costs 11/16/2018

Training is CMM Requirement iSDP IPT Organization Plan Ref. Development Directives Ref. Risk Management Plan Project Life Cycle Definition Requirements Management Plan Project Indicators Risk Management Plan S/SEE Implementation Plan Software Maintenance Plan Software Reuse Plan Subcontract Management Plan Software CM Plan Software QA Plan Training Plan Development and Review Directives Estimation Methods iSDP Compliance and Approval SPAL Standard Procedures Planning Step 1 - Develop Project IPT Operations Plan Planning Step 2 - Define System to Support Planning Planning Step 3 - Define Project Life Cycle Planning Step 4 - Define Management Plans Planning Step 5 - Define Project Directives Planning Step 6 - Estimate Scope and Resources Planning Step 7 - Review and Deliver Planning Products . New Technologies Software Process Defined by Roles 11/16/2018

Legal Issues Associated with Software Development OSSP Core Curriculum Introductory Intermediate Specialty Topics Integrated Product Teams (IPTs) (ESSS Development Course) Software Development Overview for Program Managers Quantitative Software Mgmt for Executives Executives & Program Managers Software Engineering Lifecycle Overview Introduction to the CMM and the ESSS Organizat’l Standard Software Process Work Product Inspections Managing Software Development Requirements Management Software Managers & Software Technical Leads Software Project Estimation Software Project Planning Training Plan Development Legal Issues Associated with Software Development Subcontract Management Quantitative Software Mgmt Software Creation Engineers Software Testing Software Quality Assurance Software Configuration Mgmt 11/16/2018

Software Standard Tools Web Page Electronic Format of the Software Tools Web Page Is Located on the Intranet At: http://spal.md.essd.northgrum.com  click on SPAL  click on Tools on the SPAL diagram 11/16/2018

… Provide Project & Process Insight Standard Metrics . . . Project C Data Project B Data Refine Project Process Improve Organizational Support New Project Planning Efforts Project A Data Metrics Database … Provide Project & Process Insight 11/16/2018

Class Outline The Software Engineering Institute (SEI) and the Capability Maturity Model (CMM) Importance of SEI & CMM in Industry Frame of Reference Project - IADS Major Components of the CMM Northrop Grumman Implementation of CMM Software Engineering Process Organization (SEPO) Organizational Standard Software Process (OSSP) Software Process Asset Library (SPAL) Software Project Planning Northrop Grumman Commercial 11/16/2018

Summary SEI CMM Process Maturity Northrop Grumman The SEI was established and funded by the DoD to address the “Software Crisis”. It provides leadership to advance the practice of software engineering to improve the quality of systems. SEI The CMM represents the model by which software organizations are assessed to establish organizational process maturity (5 Levels). CMM Process Maturity The quality of a software product is largely determined by the quality of the process used to develop and maintain it. Northrop Grumman Northrop Grumman’s CMM Implementation is governed by SEPO and implemented using OSSP & SPAL. 11/16/2018