COPYRIGHT 2010 GENERAL DYNAMICS ADVANCED INFORMATION SYSTEMS Published and used by TwinSPIN with permission CMMI Level 5 Journey (A.K.A. Things I Wish Others Shared With Me) Linda Schultz February 2010

2 Agenda l Why is High Maturity CMMI Important to General Dynamics? l Our CMMI Journey l Why the Backslide? l Lessons Learned ä Quality and Process-Performance Objectives (QPPOs) ä Process Performance Models (PPMs) ä Process Performance Baselines (PPBs) ä Causal Analysis and Resolution Activities (CARs) ä Organizational Innovations and Deployments (OIDs)

3 Importance of CMMI to General Dynamics l Organization has a strong affinity for process ä CMM for software since 1980s ä IEEE, MIL- and DoD- standards l CMMI Level 3 is a stated management expectation l We have contract requirements for certain CMMI levels l Strong belief that higher levels of CMMI maturity will make our developed products better, cheaper, and ready faster

4 CMMI Staged Representation Eliminate Common Causes of Variation Eliminate Special Causes of Variation Ad hoc Each Program May Have Own Process, Reactive Common Org Processes, More Proactive

5 CMMI Journey 1999 CMM Level 3 SW 2004 CMMI Level 3 version 1.1 SE, SW, IPPD, SS 2005 CMMI Level 5 version 1.1 SE, SW, IPPD, SS 2008 CMMI Level 3 version 1.2 CMMI-DEV+IPPD SYS, SW, HW 2009 CMMI Level 5 version 1.2 CMMI- DEV+IPPD SYS, SW, HW

6 CMMI Journey We are here!

7 Why the Backslide? Less Internal Focus on Process Improvement Initiatives 15% Organizational Changes 5% New CMMI Model – CMMI version 1.2 0% Changed Expectations from SEI 80%

8 CMMI Journey l We were appraised at CMMI Level 5 in 2005 l We just wanted to “renew” our credentials before they expired in 2008 l Started down the appraisal path in 2007 l What we did in the past didn’t work anymore ä Big changes for the Level 4 and 5 Process Areas: OPP, QPM, OID and CAR ä Artifacts that were acceptable in 2005 were no longer acceptable in 2007 l Needed to listen and learn... fast!

9 Changed Expectations QPPO Lessons Learned l A Quality and Process-Performance Objective (QPPO) is not just any well-stated goal or objective ä QPPOs relate back to business objectives ä QPPOs must be quantified and time-based n Reduce the escape of software defects by 15% prior to Integration Testing by December 31, 2010 ä QPPOs are impacted by 1 or more well-documented sub-processes n Sub-processes must be composed of clearly defined steps n Sub-processes must be under statistical process control

10 Changed Expectations QPPO Lessons Learned (continued) ä Organizational QPPOs flow down to the programs n Programs need to do some data analysis to assure that org QPPOs are achievable n If programs modify org QPPOs, they must provide rationale based on data analysis n Programs document their QPPOs in their Program QMP n For each program QPPO, there should be a strategy as to how it will be measured during the execution of the project and documented in the QMP n During program execution, if a program can’t meet a QPPO, they implement process changes to try to achieve it ä If it’s not possible to achieve the QPPO, programs negotiate a change to their QMP and obtain management approval using the CAR process

11 Changed Expectations PPM Lessons Learned l A Process Performance Model (PPM) is not a model [at least as we knew it] ä It’s not an application model – like CoCoMo l It is... ä An equation that relates controllable and uncontrollable factors of a sub-process ä Generated using modeling techniques (e.g., regression analysis) with tools such as Minitab ä Used throughout a program’s lifecycle to evaluate current status and predict progress towards meeting QPPOs

12 Changed Expectations PPB Lessons Learned l Be careful about selecting the data for Process Performance Baselines (PPBs) l PPBs need to be based on: ä “Clean” data vs. “Dirty” data ä Data representative of current process ä Sufficient # of data points on which to draw conclusions ä Dis-aggregated data n Don’t combine all Systems Data in a single PPB; Break separately into Requirements, Integration & Test, etc.

13 Changed Expectations PPB Lessons Learned (continued) l Generated using a tool like Minitab l Initially, a program may use organizational PPBs until they have enough data points to calculate their own PPBs ä Organizational PPBs represent a rollup of many programs’ data; programs should realize the risk of using Org PPBs l A program may use PPBs from a similar program that may be “closer” to their own (instead of using the org PPBs) until they have enough data points to calculate their own PPBs

14 Changed Expectations CAR Lessons Learned l A Causal Analysis and Resolution (CAR) activity isn’t just root cause analysis done on any problem l It is... ä A program process area ä Based on Common Cause (proactive) vs. Special Cause Variation (reactive) ä Must be related to a quantitative measure on the program -- even better if related to a QPPO ä Requires a measure of effectiveness n Effectiveness must be shown in terms of a process performance change in the program’s data n Not an ROI calculation

15 Changed Expectations OID Lessons Learned l An Organizational Innovation and Deployment (OID) isn’t just any process improvement ä even if it’s well-planned, been piloted and been monitored l It is... ä An organizational process area - the org version of CAR ä Change affects a stable process ä Process change requires a phased implementation (Pilot, Deployment, etc.) ä Based on Common Cause ä Requires a measure of effectiveness in terms of a process performance change in the program’s data - not an ROI calculation ä Apply process changes across the organization – not on 1 program or function

16 Benefits From Our CMMI Journey l Lots of new training, tools, and templates being used l Managing more with objective data rather than subjective gut feel l Data being collected is of higher quality; allows better decisions to be made l More rigorous statistical analysis done for the organization and the programs as a matter of course l More dramatic process improvements on the programs – can see the difference in their charts l Organizational process improvements have broader applicability l Tighter coupling between business objectives, process improvement goals and program QPPOs

17 Questions? Comments? Contact Information: Linda Schultz General Dynamics Advanced Information Systems Senior Process and Quality Manager, and CMMI Program Manager

18