1 Predictive Models to Achieve Business Results Place your image on top of this gray box. If no graphic i applicable, delete gray box and notch-out behind gray box, from the Title Master 19th International Forum on COCOMO and Software Cost Modeling Cvetan Redzic, Michael Crowley, Nancy Eickelmann, Jongmoon Baik Motorola, Inc. October 26, 2004

2 Outline Overview Business Goals Models Used –COQUALMO –CoQ-DES –MotoROI Primary Model Inputs –CMM –Life Cycle Scope –PCE / PSE Results –Cost –Quality

3 Business Goal – Improved Customer Satisfaction Must Be Delighters Attractive Satisfier Features Quality SW Quality Type of needs 1.Basic Expectations (Must Be) 2.Satisfier - Features 3.Delighters (Attractive) Kano Analysis

4 Cause & Effect Diagram Improved Customer Satisfaction

5 Integrating Predictive Models Models Used –COQUALMO –CoQ-DES –MotoROI

6 COQUALMO

7 Combined COQUALMO Injection Factors

8 CoQ-DES

9 CoQ-DES Simulation

10 MotoROI

11 MotoROI - DOORS ROI Analysis

12 Model Integration - Primary Model Inputs CMM Life Cycle PCE / PSE

13 CMM – Process Maturity Knox Theoretical Model of TCOQ (About 50% at CMM Level 3) COQUALMO –PMAT (process maturity has the greatest +/-impact) on injection rates CoQ-DES –Not Used directly but is inherent in organizational calibration MotoROI –Process maturity as represented by the cost of quality/cost of poor quality financial structure is a primary factor.

14 Life Cycle Requirement Design Code ImplementationUnit Test Component/Integration Test System Test Inspections Testing COQUALMO –Req., Des., Imp., and Code CoQ-DES –Full Life Cycle MotoROI –Full Life Cycle or Individual Phases

15 PCE and PSE COQUALMO –PCE and PSE as evidenced by injection and removal rates CoQ-DES –PCE and PSE as evidenced by injection and removal rates MotoROI –PCE for DP or PSE for technology effectiveness Phase Containment Effectiveness & Phase Screening Effectiveness

16 Measuring and Monitoring Results Quality Cost

17 Quality - Sources of Variation For Release with about 100 Delta KLOC, no significant difference estimates & actuals in DI & DR For large size Release over 100 Delta KLOC, there is significant difference b/w estimates & actuals in DI & DR for Code REQDESCODE Calculated Chi-Square Value Chi-Square (2;0.05)5.99 SignificanceNo Yes Actual vs. COQUALMO Estimate

18 Quality - Sigma Level Sigma Level: Defects per Million Opportunities DPMO = 1M * D/(N*O) D = 2464 HS Faults (from PCE) N = 139,595 Delta LOC DPMO = 1M * 2464/139,595 DPMO = s Stable processes Need Leap improvement: SEI CMM Level 5 TCM From PCE, SRE & CRUD data What is Sigma Level from release perspective ? Relatively stable across the releases

19 Quality - SRE Goal Setting

20 Quality: As-Is Process

21 Quality - Rayleigh Model Analysis

22 Quality - Impact of Tactical Changes Monte-Carlo simulation, to include uncertainty & risks In the expert based opinion

23 Quality - New Process Baseline

24 Cost - Vital X Monthly Review Charts SLIM

25 Quality - Vital X Monthly Review Charts Fault Injection & Removal vs. Baselines

26 Quality - Vital X Monthly Review Charts SRE

27 CRUD Goal Tracking

28 Summary Integrating predictive models provides multiple views of project quality, cost and schedule issues. More accurate predictions of defect injection are possible More accurate predictions of defect removal are possible More accurate predictions of overall staffing and project cost are possible