1. Paul Sesto, Software Engineering Manager
Introduction to the Software Engineering Institute (SEI) and the Capability Maturity Model (CMM)
Paul Sesto, Software Engineering Manager
11/16/2018

2. Class Outline
The Software Engineering Institute (SEI) and the Capability Maturity Model (CMM)
Importance of SEI & CMM in Industry
Frame of Reference Project - IADS
Major Components of the CMM
Northrop Grumman Implementation of CMM
Software Engineering Process Organization (SEPO)
Organizational Standard Software Process (OSSP)
Software Process Asset Library (SPAL)
Software Project Planning
Northrop Grumman Commercial
11/16/2018

3. Evolution of the SEI Questionable Quality of Delivered Products
“Software Crisis” Developed over Past Few Decades
Questionable Quality of Delivered Products
Cost & Schedule Overruns Pervasive
Department of Defense Catastrophes Persist
Cost Overruns of Software Acquisition Rising by the Millions
Schedule Delays of Not Months but Years
Multi-Billion-Dollar Systems Don’t Perform as Envisioned
Projects Cancelled Due to Poor Software Quality
Commercial Industry Suffered Similar Problems
Complexity of Software Continues to Increase
Industry Leaders Deemed Software Crisis to be a Management Problem and Not a Technical Problem
11/16/2018

4. What is the SEI? Software Engineering Institute
Federally Funded Research & Development Center
Sponsored by the U.S. Department of Defense
Contract Competitively Awarded to Carnegie Mellon University in 1984
CMU Staff is A Consortium of Government, Industry & Academia
SEI Mission
“Provide Leadership in Advancing the State of the Practice of Software Engineering to Improve the Quality of Systems that Depend on Software”
Visit the SEI at
11/16/2018

5. What is the Capability Maturity Model (CMM)
Framework to Help Organizations Improve Their Processes
Documented Practices for Software Development
Model for Continuous Improvement
Guidelines, not Rules, on how to Best Run Programs
Evolving to Address Other Standards (ISO-9001)
Method for Assessing “Maturity” of an Organization
Five Maturity Levels
Successive Foundations of Process Improvement
Used by DoD to Evaluate Contractors
Multiple CMMs Exist
Software Development
Integrated Product Team Development
Systems Engineering
Personal Software Process
CMMI
Initiative to Integrate Multiple Models
11/16/2018

6. Fundamental Concepts of Process Maturity
Software Process: set of activities, methods, practices, and transformations that people employ to develop and maintain software and the associated products.
internal SDP
SW Product
Underlying Premise: quality of a software product is largely determined by the quality of the process used to develop and maintain it.
11/16/2018

7. Five Levels of Software Maturity
Level 1: Initial
Process is ad hoc, even chaotic;
Success depends on individual
effort and heroics.
Level 4: Managed
Detailed measures of software process
& product quality collected;
SW process & products are
quantitatively understood & controlled.
Level 2: Repeatable
Basic project mgt processes established;
Earlier successes can be repeated
on projects with similar applications.
Level 5: Optimizing
Continuous process improvement
enabled
Quantitative feedback from the process
Piloting innovative ideas & technologies.
Level 3: Defined
Process for mgt and engineering
documented, standardized, integrated
at organization level;
All projects use version of standard
software process.
11/16/2018

8. Practices at Each Level
Process
At project level only X
At organization level X X X
Problems
Recognized/corrected as they occur X
Anticipated & prevented X
Sources understood & eliminated X
Common sources understood & eliminated X
People
Success depends on individual X
Training provided X X X X
IPTs used X X X
Strong teamwork in projects X X
Strong teamwork across organization X
All involved in process improvement X
11/16/2018

9. Practices At Each Level
Technology Assessment
Technology support established, X
stable activities
Planning & mgt data used by projects X
New technologies - evaluated qualitatively X
Data collected & used in all defined processes X X X
Data systematically shared by projects X X X
Data definition & collection standardized X X
across organization
Data used to understand process X X
quantitatively & stabilize it
Data used to evaluate & select X
process improvements
11/16/2018

10. Class Outline
The Software Engineering Institute (SEI) and the Capability Maturity Model (CMM)
Importance of SEI & CMM in Industry
Frame of Reference Project - IADS
Major Components of the CMM
Northrop Grumman Implementation of CMM
Software Engineering Process Organization (SEPO)
Organizational Standard Software Process (OSSP)
Software Process Asset Library (SPAL)
Software Project Planning
Northrop Grumman Commercial
11/16/2018

11. Benefits of Software Process Maturity
11/16/2018

12. Why Bother With CMM Certification?
Level %
LM - 3 Sites
NASA Ames
Boeing (Space
Shuttle)
Motorola (India)
STSC (Hill AFB)
Level %
NG ESSS
NG MASD
NG ESID
Hughes/Raytheon
LMTAS
Initial %
Level %
76% of Companies
are at Levels 1 & 2
Level %
TRW
Logicon
Many USAF/Army Sites
Level 4 is necessary to
remain a leader in
today’s market!
11/16/2018

13. CMM Level 3 is a DOD Requirement
SEI Level
% Of Assessments
11/16/2018

14. Class Outline
The Software Engineering Institute (SEI) and the Capability Maturity Model (CMM)
Importance of SEI & CMM in Industry
Frame of Reference Project - IADS
Major Components of the CMM
Northrop Grumman Implementation of CMM
Software Engineering Process Organization (SEPO)
Organizational Standard Software Process (OSSP)
Software Process Asset Library (SPAL)
Software Project Planning
Northrop Grumman Commercial
11/16/2018

15. IADS - The Real World Air Defense Requires Centralized Facility
Integrates Country Wide Infrastructure
Data & Voice Communications
Network of Workstations & PCs
Distributed Architecture & Displays
1.5 Million Lines of Software
Written in Ada95, Java, PWB & SQL
Object Oriented Design & Development
85 Software Engineers at Peak on 15 Integrated Product Teams
Started with a Specification and Statement of Work of 600 Pages
Standard Processes & Procedures Facilitated On Target Completion
11/16/2018

16. Class Outline
The Software Engineering Institute (SEI) and the Capability Maturity Model (CMM)
Importance of SEI & CMM in Industry
Frame of Reference Project - IADS
Major Components of the CMM
Northrop Grumman Implementation of CMM
Software Engineering Process Organization (SEPO)
Organizational Standard Software Process (OSSP)
Software Process Asset Library (SPAL)
Software Project Planning
Northrop Grumman Commercial
11/16/2018

17. Institutionalization
Structure of the CMM
Maturity Levels
Indicate
Process
Capability
Key Process Areas
Goals
Contain
Achieve
18 Total KPAs
Organized by
Common Features
Implementation or
Institutionalization
Address
5 Per KPA
Key Practices
Activities or
Infrastructure
Contain
Describe
11/16/2018

18. CMM Defined Key Process Areas (KPAs)
Continuous improvement
Defect prevention
Technology change management
Process change management
5. Optimizing
Measures collected
Process/products quantitatively
understood & controlled
Quantitative process management
SW quality management
4. Managed
Organization process focus
Organization process definition
Training program
Integrated software management
SW product engineering
Intergroup coordination
Peer reviews
3. Defined
Process standardized &
integrated at organ. level
Requirements management
SW project planning
SW project tracking & oversight
SW subcontract management
SW quality assurance
SW configuration management
2. Repeatable
Basic project mgt processes
1. Initial
Success depends on individual effort
11/16/2018

19. Each KPA Has 5 Common Features
1. Commitment to Perform
Written organizational policy required; addresses every KPA
ESSS: Software Process Manual
2. Ability to Perform
Adequate resources/training required for every KPA
ESSS: WSAs at organization and project level
3. Activities Performed
Activities, roles, procedures necessary to implement a KPA
ESSS: management plans, procedures, technical directives
11/16/2018

20. Each KPA Has 5 Common Features
4. Measurement and Analysis
Basic measurement practices necessary to determine status related to the process
ESSS: metrics (technical & management), Level 4 Software Quantitative Management
5. Verifying Implementation
Steps to ensure that the activities are performed in compliance with the process that has been established.
ESSS: Project and senior management reviews, software quality audits, SEI and ISO audits, reports to SEPO
11/16/2018

21. KPA Example - Peer Reviews
Peer review activities are planned
Defects in the software work products are identified and removed
Maturity Levels
Indicate
Process
Capability
Key Process Areas
Goals
Contain
Achieve
Organized by
Common Features
Implementation or
Institutionalization
Address
Key Practices
Activities or
Infrastructure
Describe
11/16/2018

22. Peer Review - Key Practices
1. Commitment to Perform
The project follows a written organizational policy for performing peer reviews.
Work Product
Inspections
ESSS
2. Ability to Perform
Adequate resources and funding are provided for performing peer reviews on each software work product to be reviewed.
Peer review leaders receive required training in how to lead peer reviews.
Reviewers who participate in peer reviews receive required training in the objectives, principles, and methods of peer reviews.
WSAs
Training
11/16/2018

23. Peer Review - Key Practices
EAR Tool
Plans, Minutes
3. Activities Performed
Peer reviews are planned, and the plans are documented.
Peer reviews are performed according to a documented procedure.
Data on the conduct and results of the peer reviews are recorded
4. Measurement and Analysis
Measurements are made and used to determine the status of the peer review activities
Metrics
5. Verifying Implementation
The SW Quality Group reviews and/or audits the activities and work products for peer reviews and reports the results. QA
11/16/2018

24. KPA Summary KPAs Describe What Key Process Area
Goals
Common Features
Commitment to Perform
Ability to Perform
Activities Performed
Measurement and Analysis
Verifying Implementation
KPAs Describe What
is to be Done to Satisfy
the Desired Goals
During an Assessment
Each KPA is Evaluated
for Objective Evidence
of the Common Features
for Organizational
Institutionalization
11/16/2018

25. Class Outline
The Software Engineering Institute (SEI) and the Capability Maturity Model (CMM)
Importance of SEI & CMM in Industry
Frame of Reference Project - IADS
Major Components of the CMM
Northrop Grumman Implementation of CMM
Software Engineering Process Organization (SEPO)
Organizational Standard Software Process (OSSP)
Software Process Asset Library (SPAL)
Software Project Planning
Northrop Grumman Commercial
11/16/2018

26. Software Engineering Process Organization
SEPO is the Enterprise Governing Organization
Provides Project Support
Software Project Planning
Process Definition
Process Improvement
Artifact Collection
11/16/2018

27. Process Definition & Project Planning
Management Plans
Technical Directives
Resource Estimates
Project’s
Comprehensive
Program
Plan (CPP)
Internal
Software
Development
Plan
(iSDP)
Software Process Group
Process Training
SEPO Representative
Organization’s
Standard
Software
Process
Integrated
Dev. Plan
Policies
Procedures
Templates
Command
Media
CPP
Template
Integrated
Dev. Plan
Project Planning
Tailor Standard Process
Project
Web Page
11/16/2018

28. SEPO Provides Process Improvement
Integrated
Dev. Plan
Project’s
Internal
Software
Development
Plan
(iSDP)
Software Process Group / PICT
Policies
Procedures
Templates
Management Plans
Technical Directives
Resource Estimates
Planning
and Process
Improvement
Development / PICT
Team
Periodic Process Evaluation
Analyze Metrics
Develop and Improve Process
Process
Improvements
Project
Command
Media
Web Page
Updates
Organization’s
Standard
Process Pilots
11/16/2018

29. Organizational Standard Software Process
OSSP Facilitates Organizational Standard Processes
Accessible via Corporate Intranet Web-Site
Supports SEI Level-4 Objectives
Promotes Standardization
Increases Software Productivity & Quality
11/16/2018

30. Software Process Asset Library (SPAL)
Policies
Procedures
Compliance Checklists
iSDP Generation
Tailoring Process
Training
11/16/2018

31. Software Project Planning is Simplified
Internal SDP
1. Project
IPT Org
2. System
Definition
(for planning)
3. Project
Life Cycle
4. Management
Plans
5. Technical
Directives
6. Size, Cost,
Schedule
Estimates
7. Approvals
Project’s
Comprehensive
Program
Plan
Organization Standard Software Process
Software
Policies and
Procedures
Metrics/
Training
Databases
Tool
Standards
Command
Media
Project
Web Page
Organization
Standard Process
Assets
Project Planning
Process
. . . When Projects Tailor from the Standard Process
11/16/2018

32. Templates Exist for Each Plan
iSDP
IPT Organization Plan
Ref. Development Directives
Ref. Risk Management Plan
Project Life Cycle Definition
Requirements Management Plan
Project Indicators
Risk Management Plan
S/SEE Implementation Plan
Software Maintenance Plan
Software Reuse Plan
Subcontract Management Plan
Software CM Plan
Software QA Plan
Training Plan
Development and Review
Directives
Estimation Methods
iSDP Compliance and
Approval
ESSS SPAL
Standard Procedures
Planning Step 1 - Develop
Project IPT Operations Plan
Planning Step 2 - Define
System to Support Planning
Planning Step 3 - Define
Project Life Cycle
Planning Step 4 - Define
Management Plans
Planning Step 5 - Define
Project Directives
Planning Step 6 - Estimate
Scope and Resources
Planning Step 7 - Review and
Deliver Planning Products
11/16/2018

33. . . . Reduces Software Development Costs
Standard Tool Usage . . .
iSDP
IPT Organization Plan
Ref. Development Directives
Ref. Risk Management Plan
Project Life Cycle Definition
Requirements Management Plan
Project Indicators
Risk Management Plan
S/SEE Implementation Plan
Software Maintenance Plan
Software Reuse Plan
Subcontract Management Plan
Software CM Plan
Software QA Plan
Training Plan
Development and Review
Directives
Estimation Methods
iSDP Compliance and
Approval
ESSS SPAL
Standard Procedures
Planning Step 1 - Develop
Project IPT Operations Plan
Planning Step 2 - Define
System to Support Planning
Planning Step 3 - Define
Project Life Cycle
Planning Step 4 - Define
Management Plans
Planning Step 5 - Define
Project Directives
Planning Step 6 - Estimate
Scope and Resources
Planning Step 7 - Review and
Deliver Planning Products
Supports Standard Processes
Reduces Tool Licensing Costs
Reduces Tool Training & Support Costs
Increase Software Reuse
. . . Reduces Software Development Costs
11/16/2018

34. Training is CMM Requirement
iSDP
IPT Organization Plan
Ref. Development Directives
Ref. Risk Management Plan
Project Life Cycle Definition
Requirements Management Plan
Project Indicators
Risk Management Plan
S/SEE Implementation Plan
Software Maintenance Plan
Software Reuse Plan
Subcontract Management Plan
Software CM Plan
Software QA Plan
Training Plan
Development and Review
Directives
Estimation Methods
iSDP Compliance and
Approval
SPAL
Standard Procedures
Planning Step 1 - Develop
Project IPT Operations Plan
Planning Step 2 - Define
System to Support Planning
Planning Step 3 - Define
Project Life Cycle
Planning Step 4 - Define
Management Plans
Planning Step 5 - Define
Project Directives
Planning Step 6 - Estimate
Scope and Resources
Planning Step 7 - Review and
Deliver Planning Products .
New Technologies
Software Process
Defined by Roles
11/16/2018

35. Legal Issues Associated with Software Development
OSSP Core Curriculum
Introductory
Intermediate
Specialty Topics
Integrated
Product
Teams
(IPTs)
(ESSS
Development
Course)
Software Development Overview
for Program Managers
Quantitative Software Mgmt
for Executives
Executives & Program
Managers
Software
Engineering
Lifecycle
Overview
Introduction
to the
CMM
and the
ESSS
Organizat’l
Standard
Software
Process
Work
Product
Inspections
Managing
Software
Development
Requirements Management
Software Managers &
Software Technical Leads
Software Project Estimation
Software Project Planning
Training Plan Development
Legal Issues Associated with Software Development
Subcontract Management
Quantitative Software Mgmt
Software Creation
Engineers
Software Testing
Software Quality Assurance
Software Configuration Mgmt
11/16/2018

36. Software Standard Tools Web Page
Electronic Format of the Software Tools Web Page Is Located on the Intranet At:
 click on SPAL
 click on Tools on the SPAL diagram
11/16/2018

37. … Provide Project & Process Insight
Standard Metrics . . .
Project C Data
Project B Data
Refine Project
Process
Improve
Organizational
Support New
Project Planning
Efforts
Project A Data
Metrics
Database
… Provide Project & Process Insight
11/16/2018

38. Class Outline
The Software Engineering Institute (SEI) and the Capability Maturity Model (CMM)
Importance of SEI & CMM in Industry
Frame of Reference Project - IADS
Major Components of the CMM
Northrop Grumman Implementation of CMM
Software Engineering Process Organization (SEPO)
Organizational Standard Software Process (OSSP)
Software Process Asset Library (SPAL)
Software Project Planning
Northrop Grumman Commercial
11/16/2018

39. Summary SEI CMM Process Maturity Northrop Grumman
The SEI was established and funded by the DoD
to address the “Software Crisis”. It provides
leadership to advance the practice of software
engineering to improve the quality of systems.
SEI
The CMM represents the model
by which software organizations are
assessed to establish organizational
process maturity (5 Levels).
CMM
Process
Maturity
The quality of a software product
is largely determined by the
quality of the process used to
develop and maintain it.
Northrop
Grumman
Northrop Grumman’s CMM Implementation
is governed by SEPO and implemented
using OSSP & SPAL.
11/16/2018