1. Software Technology Support Center BE AMERICA’S BEST The Problems in Software Estimating Dr. Randall Jensen Software Technology Support Center Hill AFB, UT

2. O G D E N A I R L O G I S T I C S C E N T E R Outline Evolution of software estimating models Software problems Management is the issue Data collection concerns New models? BE AMERICA’S BEST It’s all about models – “Existence is elsewhere”

3. O G D E N A I R L O G I S T I C S C E N T E R Project Uncertainty Principle BE AMERICA’S BEST

4. O G D E N A I R L O G I S T I C S C E N T E R Software model genealogy Nordon IBM 1970 Putnam/ SLIM US Army/GE/ QSM 1976 Doty RADC 1977 JS1,2,3 CEI 1980 SEER-SEM GAI 1989 Seer II / Sage SEI 1995 COCOMO TRW 1981 COCOMO II USC 1995 Validation Seer HAC 1979 PRICE-S Price 1977 TRUE-S Price 2003 REVIC USAF 1991 Management

5. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Sage/SEER-SEM - Normal form where K = life cycle cost (person years), T d = development time (years) sloc

6. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Common estimate dilemma

7. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Software problems are not new Unreliable Late delivery Modification costs prohibitive Impossible to maintain Inadequate performance Product exceeds budget costs 1968 CS Conference, Munich, Germany KEY

8. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Historic note: “ More software projects have gone awry for lack of calendar time than for all other causes combined…” F. P. Brooks, Jr., Mythical Man Month

9. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Common technology claim If you use (new technology), Productivity will improve by an order of magnitude And Defects will reduce to zero Note: None of the history-based estimating models (tools) support this claim, not even close. Neither does history.

10. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST There is always HOPE 1960 1970 1980 1990 100 90 80 70 60 50 Ada OOD Structured Analysis Structured Design Process Maturity PWB Structured Programming 3rd Generation Languages P R O D U C T I V I T Y lppm

11. O G D E N A I R L O G I S T I C S C E N T E R 3 Dimensions of development BE AMERICA’S BEST 1975 1985 1995 1938

12. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Impact of management on cost “Poor management can increase software costs more rapidly than any other factor… Despite this variation, COCOMO does not include a factor for management quality, but instead provides estimates which assume that the project will be well managed [ italics mine, RWJ ]” Boehm, B. W., Software Engineering Economics, (Englewood Cliffs, NJ: Prentice-Hall, Inc.), 1981, pg. 486 Note: On a scale of 0 to 10 well managed must rank approximately 3.4?

13. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Where would you focus effort? 3 11 17 64 TOOLS PEOPLE SYSTEMS MANAGEMENT Source: G. Weinberg, Quality Software Management, Vol. 3

14. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Biased research? ToolsPeopleSystems Managers 0 10 20 30 40 50 60 70 SEI Papers Percent Source: G. Weinberg, Quality Software Management, Vol. 3

15. O G D E N A I R L O G I S T I C S C E N T E R Cost driver relative impact It takes people to implement processes and use technology. April 21 st, 2009

16. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST BIG productivity drivers Analyst capability Management style Motivation Use of team methods Working environment Problem solving skills Programmer capability Application domain experience Use of modern practices Automated tool support

17. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Capability shift (environment) Basic Technology Constant 19802005 550075006500620085008000 8635 (1990)

18. O G D E N A I R L O G I S T I C S C E N T E R Historic data observations Models are made from historic data More data is better Adequate historic data takes time to collect (Lots of time) Repository vs Database Repository = a place where things may be stored for safekeeping Data stored without concern for a formal definition of the data placed in the repository SRDR is defined as a financial report that happens to contain technical information Database = a collection of data arranged for ease and speed of retrieval Requires consistency of entries Each data field must satisfy common definition DCARC is a repository as was SMC database Data definitions? BE AMERICA’S BEST18

19. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Problems?

20. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST Popular estimating solution Let’s create a new software model Models are out of date Models are inaccurate Too difficult to use Oh, really! Are you sure?

21. O G D E N A I R L O G I S T I C S C E N T E R BE AMERICA’S BEST New estimating models? Lack of confidence in existing tools (current issues) Optimistic estimates Unacceptable estimates Aging tools (?) Culture is constant New models require validation New models -- No silver bullets Quality data (if, where, proprietary) New models require validation New (or better) estimators? Experience Skill Integrity

22. O G D E N A I R L O G I S T I C S C E N T E R Estimating Prediction is very hard, especially when it’s about the future