Statistical Process Control of Project Performance Walt Lipke Software Division Tinker AFB, OK SCEA 2002 June Scottsdale, AZ

2Objective To discuss the application of SPC Control Charts to the EVM indicators,SPI and CPI EVM CPI SPI Control Charts SPC

3Overview Introduction SPC applied to Software Development? Review EVM & SPC SPC with EVM – Does What? Problems / Cause Solution Criteria Proposed Solutions Testing / Results Summary

4Introduction Software Division –SEI CMM Level 2 (1993) – First in Air Force –SEI CMM Level 4 (1996) – First in Federal Service –ISO 9001 / TickIT (1998) –IEEE / SEI Software Process Achievement Award (1999) EVM Facilitated the Achievements

5 Why SPC? SEI CMM Level 4 – Then & Now “Statistically Manage the Sub-process” CMM Evaluators “Show me the SPC Control Charts” Quality Control vs Performance Management

6 SPC Review Several Methods Control Charts Control Charts Several Types Individuals and Moving Range Process Behavior Anomalous Behavior Anomalous Behavior

7 Control Chart Observed Values Anomalous (“signal”) Observations – in sequence

8 EVM Review Time BCWS ACWP BCWP $ Total Allocated Budget Budget at Completion Management Reserve Project Completion Date Negotiated Completion Date

9 SPC with EVM – Does What? Performance Prediction –Probability of Success –EAC & ECD – range Project Planning –Historical Data –Risk MR Strategy Process Improvement –Plan Execution –Decreasing Variation

10 Planning/Performance/Improvement Time $$ Cost Distribution Schedule Distribution Performance Window (PW) Negotiated Performance (> 50% PW) Planned Performance (= 50% PW) Total Allocated Budget at Completion Planned Project Completion Negotiated Project Completion

11Problems SPI Control Chart SPI -1 Control Chart

12Problems SPI (signal removed)SPI -1 (no signal)

13Problems Legend: Solid Line ()-actual Dashed line ( )-expected Legend: Solid Line ()-actual Dashed line ( )-expected

14 More Problems Observations

15 Problem Example SPI SPI -1

16 Problem Summary > PI cum & > PI -1 cum Signals (nearly always) > 1.0 PI signals PI -1 signals PI sigma PI -1 sigma Histograms Normal Distribution Without Resolution SPC Application

17 Problem – Cause? PI or PI -1 Skewed Distribution Normal Distribution Average Signals Sigma

18 Solution Criteria (1) -1 = (2) PI Signals = PI -1 Signals (3) PI Sigma = PI -1 Sigma (4) Histograms Normal Distribution

19 Problem Solution * Invert Data < Inverted Data behave as if 1.0 * Distinguish Inverted Data * Use Inverted Data and Unchanged Data for SPC analysis SPI a SPI b -1 SPI b ~SPI b -1

20 Data Transform Rules If PI  1.0, then ~PI = PI If PI < 1.0, then ~PI = 2 - PI -1 If  ~PI   1.0, then  PI  u = If  ~PI  < 1.0, then  PI  u = (2-  ~PI  ) -1 Perform SPC analysis with Transformed Data

21 Problem Solution -Example SPI ~SPI

22 Proposed Solution Evaluation Demonstrates meeting criteria 1, 2, and 3 Mathematically meets criteria 1, 2, and 3 Proof enough?

23 Data Transform – Histogram Test CPI -1 Histogram~CPI -1 Histogram

24 Proposed Solution - #2 SPI ln SPI 0.2 (1.609) (-1.609) Resolves PI vs PI -1 Resolves PI < 1.0 Transformation Simplicity Satisfy Criteria? Logarithm Property: x ln x x -1 -ln x ln 1 = 0

25 Natural Log – Criteria Test ln SPIln SPI -1

26 Natural Log – Histogram Test Count Legend: Solid Line ( ) - actual Dashed line ( ) - expected ln CPI -1 Histogram

27 Testing Summary TestRawTransformationLogarithm 1.  PI  -1 =  PI -1  NoYes 2.PI Signals = PI -1 SignalsNoYes 3. PI Sigma = PI -1 Sigma NoYes 4.Histograms ~ Normal Distribution Very Unlikely Likely

28 Sensitivity Analysis  SPI s  (0.284,0.025)  SPI  (0.625,0.112)  SPI s  u (0.327,0.007)  SPI  u (0.651,0.082)  ln  SPI s  u (0.266,0.01)  ln  SPI  u (0.384,0.018)   PI  - PI cum  Note:1.Subscript s indicates the signal is removed from the calculations. 2.Subscript u indicates the average value is untransformed from the average value determined from the SPC analysis

29Summary SPC application to Software Development SPC applied to CPI & SPI –Project Execution –Project Planning –Process Improvement Problems –Data Representation –SPC Results

30Summary Solutions –Data Transform –Natural logarithm Criteria –Results independent from data representation –Results derived from Normal Distribution Testing/Results –Data Transform – Good –Natural Logarithm - Better

31 Final Remarks Equivalent to CPI and SPI –CV% = 1 – CPI -1 –SV% = SPI –1 Distribution is skewed Data transformation is needed Managing to CV% and SV%

32 Final Remarks SPC – Better Management Decisions Weekly EV – More Management Decisions Weekly EV w/o SPC – Process Tampering Try SPC – It’s Not Difficult Weekly EV vs Monthly SPC