1. Copyright QSM Associates, Inc. All Rights Reserved 1 Michael C. Mah Managing Partner QSM Associates, Inc. 75 South Church Street Pittsfield, MA 01201 413-499-0988 Fax 413-447-7322 e-mail: michaelm@qsma.com Web Site: www.QSMA.com Boston SPIN October 19, 1999 Software Projects in Crisis Dealing with Dynamics of Imposed Deadlines

2. Copyright QSM Associates, Inc. All Rights Reserved 2 Industry Overrun Statistics v $250 Billion + Spent on IT Application Development v 31% of Projects Will Be Cancelled, Representing $81 Billion in Losses v 52.7% of Projects Will Overrun by >189% v Only 16.2% On-time, Under Budget v But with only 42% Original Functionality! * Source: Standish Group, Dennis MA, 1995

3. Copyright QSM Associates, Inc. All Rights Reserved 3 Other Noteworthy Trends v BILLION-DOLLAR OUTSOURCING DEALS DOUBLE IDC estimates that the number of large outsourcing contracts rose 100% between 1997 and 1998. (Source: *Computerworld*; www.idg.net/go.cgi?id=125306)www.idg.net/go.cgi?id=125306) v 75% of organizations surveyed have backlogs running from seven months to over two years. (Source: Chris Pickering's 1998 *Survey of Advanced Technology*;www.cutter.com/itgroup/reports/sat98.html)www.cutter.com/itgroup/reports/sat98.html

4. Copyright QSM Associates, Inc. All Rights Reserved 4 Worldwide Software Productivity Trend v The Good News: Over the last decade, productivity has increased fairly consistently. 10% faster speed, 25% less cost, about every 2.5 years. v The Bad News: It’s Not Enough. Demand continues to outstrip capacity. v Companies are reporting growing backlogs ranging from 7 months to over 2 years. v The Badder News: Continued labor shortages. v The More Badder News: Don’t expect any schedule relief in an “Internet Speed” economy.

5. Copyright QSM Associates, Inc. All Rights Reserved 5 Reality Bites “We don’t have the luxury of determining our schedules. They’re told to us. Then, given the time frame, we try to tell the client what we can build. In the end, we usually wind up working lots of overtime, because they want everything.” - Manager of Development Wall St. Financial Firm

6. Copyright QSM Associates, Inc. All Rights Reserved 6 Anyone Know What This Is?

7. Copyright QSM Associates, Inc. All Rights Reserved 7 Real Time Engineering Info Systems Impossible Zone Are Deadlines/Plans in the “Impossible Zone”?

8. Copyright QSM Associates, Inc. All Rights Reserved 8 Good to Know... " As impressive as growth of the software industry has been, it is outpaced by growth of software- related litigation. It is not unusual for a large software development organization today to have upwards of 50 active cases on its hands." Tom DeMarco, Cutter Consortium Bulletin, July 16, 1998

9. Copyright QSM Associates, Inc. All Rights Reserved 9 Two “Old School” Ways of Estimation v Level of Effort, “Bottoms Up” Activity Based Method v Estimate and Add Up Person Months v Force into Deadline v “Productivity Ratio” Based Method v Divide Size into “Productivity” v Calculate Effort v Force Into Schedule

10. Copyright QSM Associates, Inc. All Rights Reserved 10 Example # 1 Level of Effort “ Here’s how we get an estimate. First, we look at each requirement. We estimate the person hours for the work based on complexity. We have an Excel spreadsheet with a table for the person hours needed to make a simple, modest, or complex change, on a simple, modest, or complex program. So, we get something like 20 person hours for high level design, 32 person hours for detailed design, another 40 for coding and unit testing for a given requirement.” “That comes out to 112 person hours. We multiply that by 30% to get the integration. That makes 145.”

11. Copyright QSM Associates, Inc. All Rights Reserved 11 Example # 1 Level of Effort “ We add all the person hours up for the entire project. The grand total might add up to 34,000 person hours, or 200 person months. If we have a 10 month deadline, then it means we need to assign 20 people to the project.” “If the schedule gets shortened on us to 6 months vs 10, then dividing 200 person months into a 6 month schedule means we need 33 people instead of 20.”

12. Copyright QSM Associates, Inc. All Rights Reserved 12 Example # 2 “Productivity Ratio” “ We ascertained that this project will be 800 total new + modified Function Points. Our productivity is 10 FP/staff month. So this project will take 80 staff months. We have 8 people on the team. So it will take 10 months.” “If the schedule gets shortened on us to 6 months vs 10, then dividing 80 staff months into a 6 month schedule means we need 13 people instead of 8.”

13. Copyright QSM Associates, Inc. All Rights Reserved 13 Violation of Brook’s Law Manpower and Time are not interchangeable. That is, you can not cut schedule in half by doubling the number of staff. - Fred Brooks, The Mythical Manmonth ( Corollary: You can not get a baby in 1 month with 9 women.)

14. Copyright QSM Associates, Inc. All Rights Reserved 14 Laws of Cause and Effect v The fundamental, or core metrics of time, cost, and quality, across small, medium, and large projects (size) are interdependent. v We, as managers and developers, have treated them as independent variables. v For example, we want to believe that we can keep a deadline fixed in spite of size growth. The think the effort is the effort, and reliability should not be impacted much (we hope). v Not only are they interdependent, but data has revealed to us that changes in one dimension can result in geometric impacts in another dimension

15. Copyright QSM Associates, Inc. All Rights Reserved 15 How Development Lifecycles Behave A Pop Quiz: “If you tried to shorten the schedule on an application development project by adding staff to say, double (20 people versus 10), how much will you able to compress it? Will defects go up or down, and by how much?”

16. Copyright QSM Associates, Inc. All Rights Reserved 16 How Software Lifecycles Behave Answer: Schedules will only compress (nominally) by about 20 percent. Defects typically rise by about 6 fold.* Rule of Thumb: “20/200/6x” *Source: QSM Industry Database Statistics

17. Copyright QSM Associates, Inc. All Rights Reserved 17 When Time & Effort are “Fixed” Something’s Got to Give... Size Time Effort Defects

18. Copyright QSM Associates, Inc. All Rights Reserved 18 “Feature Creep” - Impact on Quality with Fixed Schedule Schedule Fixed at 14 Months Size Increases in 5KSLOC Increments from 72K to 92K Quality Levels Cut in Half - MTTD 3.5 Days to 1.75 Days

19. Copyright QSM Associates, Inc. All Rights Reserved 19 Size Probability on Time Scenario: Size Growth, or “Feature Creep”

20. Copyright QSM Associates, Inc. All Rights Reserved 20 Compress Schedule: Plan 1 uses 9 People Staffing v 9 People v 69 Staff Months v MTTD = 3.2 Days Elapsed Time - Months

21. Copyright QSM Associates, Inc. All Rights Reserved 21 Compress Schedule: Plan 2 uses 24 People Staffing Elapsed Time - Months v 24 People v 144 Staff Months v MTTD = 1.2 Days

22. Copyright QSM Associates, Inc. All Rights Reserved 22 Dangerous Productivity Metrics: Only 2 Dimensions Lines of Code Staff Month Function Points Staff Month

23. Copyright QSM Associates, Inc. All Rights Reserved 23 Summary: Both Ratios Are Equally Poor Measures Size = 75,000 New and Modified SLOC 707 FP Peak Staff Sched (Mos) FP/PM 6 9 14 24 33 66 13.6 12.3 11.3 10.2 9.5 8.3 15.4 10.2 7.3 4.9 3.8 2.1 LOC/PM 1630 1086 773 520 398 225 LOC/DAY 254 281 306 339 364 417 Effort (PM) 46 69 97 144 188 333

24. Copyright QSM Associates, Inc. All Rights Reserved 24 SEI Special Report CMU/SEI-95-SR-005 1) A Corporate Memory (Historical Database) 2) Structured Processes for Estimating Size and Reuse 3) Mechanisms for Extrapolating from Demonstrated Accomplishments on Past Projects

25. Copyright QSM Associates, Inc. All Rights Reserved 25 4) Audit Trails (Values for Cost Model Parameters Used to Produce Each Estimate Are Recorded and Explained) 5) Integrity in Dealing with Dictated Costs and Schedules 6) Data Collection and Feedback Processes That Foster Capturing and Correctly Interpreting Data from Work Performed SEI Special Report CMU/SEI-95-SR-005

26. Copyright QSM Associates, Inc. All Rights Reserved 26 The Role of Measurement, and Commitments Estimation & Planning Control & Forecasting Support Future Commitments Manage Commitment CommitmentAnalyze Performance on Commitment History Repository Assess Viable Strategies Monitor Status & Replan Post Project Analysis Make Commitment

27. Copyright QSM Associates, Inc. All Rights Reserved 27 Mapping to An “Efficiency”, or “Productivity Index ” Productivity Index Functionality (Size) Effort Time ~ Goodness (Quality/Defects)

28. Copyright QSM Associates, Inc. All Rights Reserved 28 Example PI Calculation Size = 375 Function Points 35,000 LOC Effort = 24 Person-Months Time = 6 Months * PI = SIZE TIMEEFFORT = 17

29. Copyright QSM Associates, Inc. All Rights Reserved 29 Productivity Index Industry Baselines - 1997 CategoryPI STD Dev v Business 17.3 +/- 4.1 v System Software 13.7 +/- 4.9 v Telecom 12.2 +/- 4.0 v Scientific 12.1 +/- 3.5 v Process Control 12.1 +/- 3.4 v Command & Ctrl 11.3 +/- 4.3 v Avionic 8.2 +/- 4.8 v Real-time 7.8 +/- 3.8 v Microcode 6.3 +/- 2.8 Finance Retail Insurance Others

30. Copyright QSM Associates, Inc. All Rights Reserved 30 v Given a Certain Development Efficiency (PI), v And Given the Deadline v With a Team of “X” People... “Real World Estimation” How Much Functionality Can We Build? How Much Functionality Should We Promise?

31. Copyright QSM Associates, Inc. All Rights Reserved 31

32. Copyright QSM Associates, Inc. All Rights Reserved 32

33. Copyright QSM Associates, Inc. All Rights Reserved 33 In Summary... v Productivity is Multi-Dimensional. 2-Dimensional Measures Are Inadequate v Visual Explanations Are Vital v The Need to Quantify Size of New, Modified, and Total Delivered Functionality Will Arise v No One Size Measure is a “Silver Bullet”

34. Copyright QSM Associates, Inc. All Rights Reserved 34 In Summary... v Metrics of Schedule, Cost, and Quality are Interdependent. v Productivity Metrics in the Form of Ratios Are Dangerous. They Omit Time. Time = Deadline. Deadline is the Independent Variable, Upon Which the Dependent Variables Behave.

35. Copyright QSM Associates, Inc. All Rights Reserved 35 Establishing a Software Management Program: Lessons Learned v Management Awareness and Acuity are Essential v A Metrics “Champion” is Essential v Collect Core Metrics; “Know Your Capability” v Dull Metrics Means You’ve Got the Wrong Metrics v Turn Numbers Into Pictures, Show the Right Dimensions, Reveal What is Hidden

36. Copyright QSM Associates, Inc. All Rights Reserved 36 Recommended Reading  Carleton, Anita, Park, Robert, and Goethert, Wolfhart, “The SEI Core Measures: Background Information and Recommendations for Use and Implementation” © 1994 The Journal of the Quality Assurance Institute.  Mah, Michael C., and Putnam, Lawrence H., “Software by the Numbers: An Aerial View of the Software Metrics Landscape” © 1997 American Programmer.  Putnam, Lawrence H., and Myers, Ware, “Executive Briefing: Controlling Software Development” © 1996 IEEE Computer Society Press.  Tufte, Edward, “Visual Explanations, Images and Quantities, Evidence and Narrative” © 1997 Graphics Press.