1. an employee-owned company Extensions of Auto-Generated Code and NOSTROMO Methodologies (U) Pam McDonaldSandra GilesDan Strickland THAAD Project OfficeTHAAD Project OfficeDynetics, INC DISTRIBUTION A: Approved for public release; distribution unlimited.

2. an employee-owned company Rebus (U) rebus - a mode of expressing words and phrases by pictures of objects whose names resemble those words, or the syllables of which they are composed; enigmatical representation of words by figures; hence, a peculiar form of riddle made up of such representations. - a puzzle where you decode a message consisting of pictures representing syllables and words

3. an employee-owned company Rebus 1 (U) Cocoa Moe Two COCOMO II

4. an employee-owned company Rebus 2 (U) Donald Reef Donald J. Reifer Jay

5. an employee-owned company Rebus 3 (U) Barry Boehm Strawberry Straw Boeheim Hi

6. an employee-owned company Overview (U) Background AGC Extensions AGC Worksheet Demo NOSTROMO Extensions NOSTROMO Demo Future Enhancements

7. an employee-owned company Background (U) In 2002, TPO and Dynetics presented a methodology on the estimated effort of Auto-Generated Code (AGC) In 2003, TPO and Dynetics presented a methodology on using Monte Carlo simulation and uncertainty in COCOMO II to estimate risk in estimates Both methodologies have been extended and built upon to cover more situations Methodologies combined into a single tool suite to offer expanded solutions to estimation needs

8. an employee-owned company AGC Extensions

9. an employee-owned company AGC Terminology (U) generating – related to the development, input effort used to auto-generate code resultant – related to the code that is output by the auto-generation synthetic – the normalized estimate of effort that takes into account the effort in non-testing activities (generating language) and testing activities (resultant language)

10. an employee-owned company Testing vs. Non-Testing Efforts – COCOMO II Phases (U) Requirements Prelim. Design Detailed Design Code and Unit Testing Integration Testing COCOMO II Effort Distribution – Large Project Non-Testing Testing Waterfall Model Requirements, Preliminary Design and Detailed Design are non- test related; Integration Testing is test related Code and Unit Testing contains activities in both areas

11. an employee-owned company Testing vs. Non-Testing Efforts - Code and Unit Testing Phase (U) Code and Unit Testing Requirements Design Programming Test Plans V & V Project Office CM/QA Manuals 4% 8% 56.5% 5.5% 8.5% 6% 6.5% 5% Code and Unit Test broken up into default activities Requirements, Design, and Programming are non-test activities Test Plans and V & V are test activities Project Office, CM/QA, and Manuals are non-specified - split evenly between both efforts 77.25% Non- Test 22.75% Test Non-Testing Testing

12. an employee-owned company Testing vs. Non-Testing Effort - Synthetic SLOC (U) COCOMO II Effort Distribution 67.2 % Non-Test 32.8% Test Normalized effort values for non-test and test efforts 67.2% of development size counted in synthetic size 32.8% of translated size counted in synthetic size Example: 10KSLOC translated to 50KSLOC –10,000 (0.672) + 50,000 (0.328) = 23,120 SLOC Synthetic SLOC Rule of Thumb: 66.7% non-test, 33.3% test Non-Testing Testing

13. an employee-owned company Adapted AGC (U) NON-AGC CODE GENERATING RESULTANT % Redesign % Recode % Retest AGC CODE % Redesign % Recode % Retest ADAPTED NON-AGC ADAPTED AGC New Non-AGC + Synthetic SLOC + Adapted Code = ESLOC

14. an employee-owned company Sample Calculation (U) Adapted Code of 10000 SLOC 5000 SLOC Non-AGC Ada95 5000 SLOC in Ada95 auto-generated by 250 SLOC in a 4GL Modification Percentages Re-design: 5%Re-code: 10%Re-test: 25% Non-AGC Adapted: = 5000 * ((0.4)*(0.05) + (0.3)*(0.1) + (0.3)*(0.25)) = 5000 * (0.125) = 625 ASLOC AGC Adapted: = 250 * ((0.4)*(0.05) + (0.3)*(0.1)) + 5000 * ((0.3)*(0.25)) = 250 * (0.05) + 5000 * (0.075) = 388 ASLOC Adapted Synthetic SLOC: = 625 + 388 = 1013 Adapted Synthetic SLOC ESLOC Calculation: = (1013 Adapted Synthetic SLOC) + (10000 New Non-AGC SLOC) + (2300 New Synthetic SLOC) = 13313 ESLOC

15. an employee-owned company AGC Worksheet (U) Microsoft Excel Workbook with two main worksheets: Synthetic SLOC Worksheet – calculates Synthetic SLOC estimate New Size Worksheet – calculates ESLOC from Synthetic SLOC, New Non-AGC SLOC, and Adapted SLOC (above) Cells with yellow background are calculated Cells with tan background require inputs Assumes estimates for Generating and Resultant Adapted SLOC are discrete and known

16. an employee-owned company NOSTROMO Extensions

17. an employee-owned company NOSTROMO Background (U) There are 31 factors normally associated with a COCOMO II model –New and Adapted Size (SLOC) –17 Cost Drivers –5 Scale Factors –3 Adaptation Percentages of Modification –3 Adaptation Cost Drivers –Requirements Evolution and Volatility With any parametric model there are degrees of uncertainty and subjectivity; some factors are more concrete than others (Process Maturity is more concrete than Complexity) Based on the ranges of uncertainty, a Monte Carlo Simulation can model several hundreds/thousands of COCOMO II estimates with varying degrees A Monte Carlo Simulation of COCOMO II can produce a range of estimates and the risk factors associated with each estimate GOAL – Develop a Monte Carlo Simulation of COCOMO II based on the ranges of uncertainty associated with each factor

18. an employee-owned company Original NOSTROMO Concept (U) NOSTROMO first takes normal COCOMO II inputs NOSTROMO accounts for uncertainty in the settings NOSTROMO uses Monte Carlo to simulate hundreds of COCOMO II estimates using the inputs and uncertainties NEWT – NOSTROMO Entry Writing Tool – a Delphi polling tool that captures Uncertainty NEWT DATA PAGE REPORTS NOSTROMO Notional Obscurity STatistical Risk Observation MOdel

19. an employee-owned company Distributions of Uncertainty (U) Setting n-1 Setting n Setting n+1 Level 1 - Certain Level 2 - Low Uncertainty - Normal Level 3 - Medium Uncertainty - Triangle Level 4 - High Uncertainty - Uniform NOSTROMO assumes ceiling and floor limits of the highest and lowest default values for each COCOMO II Scale Factor and Cost Driver NOSTROMO changes the distribution and sets the high and low points based on the level of uncertainty with each factor

20. an employee-owned company NOSTROMO Application (U) Microsoft Excel 2002 worksheet with attached macros and Visual Basic code Currently on version 0.3.14 (as of 10/4/04) Does not use outside applications for Monte Carlo simulation (standard random number generation from Visual basic) Limited testing performed on a proof-of-concept application Suggest limiting Monte Carlo run size to 500-2000 runs Generates two pages of output – Data and Charts Data page contains all outputs from Monte Carlo simulation, charting data, and histograms Charts page contains Confidence Intervals for output of COCOMO II and Putnam models, charts, and histograms NOSTROMO – Data Sheet and CM Control Sheet are Developer pages and should not be removed

21. an employee-owned company New NOSTROMO Features (U) COCOMO 18 Cost Driver Difficulties –In 2003, a breakout group focused on rating the difficulty of COCOMO II Cost Drivers –Results were grouped and assigned Uncertainty values based on grouping in NOSTROMO COCOMO 18 Conference Comments –Added Default COCOMO II output to NOSTROMO (data and charts) –Added EAF as output to NOSTROMO (data and charts) Conditional Formatting –NOSTROMO Entry Page has red-yellow-green-blank notation associated with Uncertainty values Delete Project Function –Removes all Subcomponent Data and Chart pages Limited Data Validation –No zero size estimates –All Cost Drivers and Scale Factors must be populated –No blank Subcomponent names Summation of Subcomponents –Tracks at the 25 th, 50 th, 75 th, percentiles and default COCOMO II AGC Calculator –Works for a single subcomponent at a time –Calculates ESLOC based on new AGC Calculation rules

22. an employee-owned company NOSTROMO Application Version History (U)

23. an employee-owned company Current NOSTROMO Concept (U) NEWT DATA PAGE REPORTS NOSTROMO DEFAULT UNCERTAINTY COMBINATION AGC CALCULATOR NOSTROMO 0.3.14 offers: multiple methods for inputting Uncertainty combination of subcomponents more reporting capability AGC methodology

24. an employee-owned company Screenshots – NOSTROMO Input Sheet – Notional Data (U) Software Item Name Scale Factors Code Size Cost Drivers Iterations Putnam Productivity Function Buttons Conditional Formatting

25. an employee-owned company Screenshots – NOSTROMO COCOMO II Charts Sheet – Notional Data (U)

26. an employee-owned company Screenshots – NOSTROMO AGC Calculator – Notional Data (U)

27. an employee-owned company Using NOSTROMO (U) Initial Estimates –Use NEWT or Default Setting to identify Uncertainty –Run NOSTROMO to get a range of results –Select results based on business criteria “What-If” Scenarios –Isolate Cost Drivers and Scale Factors in question by setting Uncertainty > 1 –Set non-factors to Uncertainty = 1 –Run NOSTROMO to identify risks Model Validation –Input current model settings as first entry in NEWT –Poll for additional entries based on survey techniques –Run NOSTROMO with size and phase inputs to get a range of results AGC Tradeoffs –Run NOSTROMO without AGC estimated –Run NOSTROMO with an AGC estimate and subsequent size Uncertainty –Compare results

28. an employee-owned company Future Enhancements (U) Monte Carlo Enhancement for AGC More Combination reporting and charts More data checking Options to change model settings

29. an employee-owned company Conclusion (U) New methodologies deliver insight into the real risks associated with software development New AGC methodology allows software estimators to address the real-world scenario of AGC carried across incremental software builds New NOSTROMO methodology incorporates comments from COCOMO 18, new default Uncertainty values, and combinations of subcomponents without losing the original functionality of previous releases The AGC and NOSTROMO tools are combined into a single tool suite that addresses issues in software estimation that are relevant, prevalent, and typically problematic