1. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE1 Tutorial: System and Software Sizing Barry Boehm, Ali Afzal Malik, USC-CSSE COCOMO/SSCM Forum Sizing Workshop October 31, 2007

2. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE2 Outline Nature and value of sizing Sizing paradoxes and challenges Sizing metrics Sizing methods and tools Sizing research and development needs Conclusions and references

3. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE3 Nature and Value of Sizing Definitions and general characteristics Criteria for good sizing parameters Value provided by sizing When does sizing add less value?

4. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE4 Sizing Definitions and Characteristics Dictionary definition: bigness, bulk, magnitude Generally considered additive –Size (A U B) = Size (A) + Size (B) Often weighted by complexity –Some artifacts “heavier” to put in place –Requirements, function points

5. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE5 Criteria for Good Sizing Metrics (Stutzke, 2005) CriteriaDefinition RelevanceRepresents the characteristic of interest. AccuracyFaithfully represents the amount or degree of that characteristic. Adequate PrecisionThe level of detail needed by user is achievable. DependableValues are measured consistently. Mathematical operations “work” as expected. TimelyUser receives values in time to act on them. AffordableThe value of the information exceeds the cost of obtaining it. PredictabilityAdequately accurate forecasts of future values are possible. ControllabilityActions exist that can influence the measured value.

6. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE6 Value Provided by Sizing Often aids in cost and effort estimation –(Cost, Effort) = (# Size Units) x () Denominator for productivity, quality comparisons –Cost, effort / size unit –Defects / size unit Cost, Effort Size Unit

7. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE7 When Does Sizing Add Less Value? Often easier to go directly to estimating effort –Imprecise size parameters GUI builders; COTS integration –Familiar, similar-size applications Analogy to previous effort: “yesterday’s weather” Value in having performers do direct effort estimation –Stronger in-budget completion commitment –Better understanding of job to be done Sizing can add some value here When size is a dependent variable –Time-boxing with prioritized features

8. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE8 Outline Nature and value of sizing Sizing paradoxes and challenges Sizing metrics Sizing methods and tools Sizing research and development needs Conclusions and references

9. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE9 Paradox 1: More Software May Not Be Better ProductSize (SLOC)Effort (PM)SLOC/PM TRW UNAS 0.712,20028436 TRW UNAS 0.95,20038137 Benjamin Franklin (paraphrased): “My apologies for the length of this letter. Had I more time, it would have been shorter.”

10. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE10 Productivity Paradoxes 2, 3 Paradox 2: Cheaper software may not be better –More critical defects (Ariane V, Mars Climate Orbiter) –Later to market (cube-root vs. square-root law) Paradox 3: More reuse may not be better –Reusing obsolete software –Gaming the metrics: localization as reuse

11. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE11 Traditional Minimum-Cost Schedule Months = 3 * ∛ Person-Months 27 PM  9 months, average of 3 persons Low communications overhead, but late delivery Preferable RAD approach: 5.2 months, average of 5.2 persons –Square-root law

12. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE12 Productivity Pitfalls WYMIWIG: What you measure is what you get –Weinberg data I One approach fits all situations –Weinberg data II Assuming that asking for precise data will produce precise data –Not if it takes much extra effort

13. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE13 WYMIWYG: What You Measure Is What You Get* *Weinberg-Schulman, 1974**1=Best

14. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE14 Effect of Objectives on Productivity (Weinberg-Schulman, 1974) - Same application: Solve linear equations

15. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE15 Challenges: Product Emergence and Elaboration Brooks factors: software system, product Seaver emergence and elaboration data Cone of uncertainty data

16. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE16 Brooks’ Factor of 9 for Programming System Product A Program A Programming System A Programming Product A Programming Systems Product x3 Product: Handles off-nominal cases, testing; well-documented System: Handles interoperability, data management, business management Telecom: 1/6 basic call processing; 1/3 off-nominals; 1/2 billing Adapted from Brooks, 1995

17. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE17 Emergence and Elaboration Data (Seaver, 2007) Example: 3 cost estimates for the same project – Employee information database

18. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE18 Emergence and Elaboration 2 (Seaver, 2007) How big is it now? How much of what we have now do we need to keep? How much data do we need to transition? Any Security, Compliance, Privacy Issues?

19. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE19 Emergence and Elaboration 3 (Seaver, 2007) Are all business processes and rules covered?

20. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE20 Emergence and Elaboration 4 (Seaver, 2007) Do we need all of this?

21. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE21 The Cost and Size Cone of Uncertainty If you don’t know what you’re building, it’s hard to estimate its size or cost Boehm et al. 2000

22. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE22 Outline Nature and value of sizing Sizing paradoxes and challenges Sizing metrics Sizing methods and tools Sizing research and development needs Conclusions and references

23. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE23 Sizing Metrics vs. Time and Degree of Detail (Stutzke, 2005) Process Phase Possible Measures Primary Aids ConceptElaborationConstruction Increasing Time and Detail Subsystems Key Features User Roles, Use Cases Screens, Reports, Files, Application Points Function Points ComponentsObjects Source Lines of Code, Logical Statements Product Vision, Analogies Operational Concept, Context Diagram Specification, Feature List Architecture, Top Level Design Detailed Design Code

24. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE24 Example Metric Rating: Dependability Subsystems, Use Cases, Requirements –Low: hard to pin down uniform level of detail Screens, Function Points, Components, Objects –Medium: takes some training and experience Lines of Code, Code Information Content (Halstead) –High: can be automatically counted

25. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE25 COSYSMO Sizing 4 size drivers* –Number of system requirements –Number of major interfaces –Number of operational scenarios –Number of critical algorithms *Each weighted by complexity, volatility, and degree of reuse

26. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE26 Cockburn Use Case Level of Detail Scale (Cockburn, 2001)

27. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE27 Function Points 5 Function Types (IFPUG) –External Input (EI), External Output (EO), External Query (EQ), Internal Logical File (ILF), External Interface File (EIF) –Complexity levels: Low, Average, High –Each combination of complexity level and function type assigned a weight –Unadjusted Function Points (UFPs): weighted sum of count * weight Implementation-independent metric Available at an early stage Variant: Feature Points (Jones, 1996) –Average complexity only, for all types –Sixth type: algorithms –Simple

28. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE28 Object-Oriented Metrics Tradeoff (Stutzke, 2005) –Objects: application domain vs. solution domain Six OO metrics (Chidamber and Kemerer, 1994) –Weighted Methods Per Class (WMC) –Depth of Inheritance Tree (DIT) –Number of Children (NOC) –Coupling Between Object Classes (CBO) –Response for a Class (RFC) –Lack of Cohesion in Methods (LCOM)

29. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE29 Lines of Code Source Lines of Code (SLOC) = logical source statements Logical source statements = data declarations + executable statements CodeCount tools available on USC CSSE web site Adapted from Madachy, 2005

30. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE30 SLOC Counting Rules Adapted from Madachy, 2005 Standard definition for counting lines - Based on SEI definition checklist from CMU/SEI-92-TR-20 - Modified for COCOMO II

31. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE31 Relationship Among Sizing Metrics Two broad categories of sizing metrics –Implementation-specific e.g. Source Lines of Code (SLOC) –Implementation-independent e.g. Function Points (FP) Need to relate the two categories –e.g. SLOC/FP backfire ratios

32. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE32 Multisource Estimation Implementation-independent/dependent Implementation-independent estimators –Use cases, function points, requirements Advantage: implementation-independent – Good for productivity comparisons when using VHLLs, COTS, reusable components Weakness: implementation-independent –Gives same estimate when using VHLLs, COTS, reusable components, 3GL development Multisource estimation reduces risk

33. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE33 SLOC/FP Backfiring Table (Jones, 1996): other backfire ratios up to 60% higher

34. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE34 Reused and Modified Software Effort for adapted software (reused or modified) is not the same as for new software. Approach: convert adapted software into equivalent size of new software.

35. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE35 Nonlinear Reuse Effects The reuse cost function does not go through the origin due to a cost of about 5% for assessing, selecting, and assimilating the reusable component. Small modifications generate disproportionately large costs primarily due the cost of understanding the software to be modified, and the relative cost of interface checking.

36. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE36 COCOMO Reuse Model A nonlinear estimation model to convert adapted (reused or modified) software into equivalent size of new software:

37. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE37 COCOMO Reuse Model cont’d ASLOC - Adapted Source Lines of Code ESLOC - Equivalent Source Lines of Code AAF - Adaptation Adjustment Factor DM - Percent Design Modified. The percentage of the adapted software's design which is modified in order to adapt it to the new objectives and environment. CM - Percent Code Modified. The percentage of the adapted software's code which is modified in order to adapt it to the new objectives and environment. IM - Percent of Integration Required for Modified Software. The percentage of effort required to integrate the adapted software into an overall product and to test the resulting product as compared to the normal amount of integration and test effort for software of comparable size. AA - Assessment and Assimilation effort needed to determine whether a fully- reused software module is appropriate to the application, and to integrate its description into the overall product description. See table. SU - Software Understanding. Effort increment as a percentage. Only used when code is modified (zero when DM=0 and CM=0). See table. UNFM - Unfamiliarity. The programmer's relative unfamiliarity with the software which is applied multiplicatively to the software understanding effort increment (0-1).

38. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE38 Assessment and Assimilation Increment (AA)

39. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE39 Software Understanding Increment (SU) Take the subjective average of the three categories. Do not use SU if the component is being used unmodified (DM=0 and CM =0).

40. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE40 Programmer Unfamiliarity (UNFM) Only applies to modified software

41. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE41 Software Maintenance Reuse model also addresses software maintenance sizing –all of reused software becomes the maintenance base, not “equivalent SLOC”

42. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE42 Outline Nature and value of sizing Sizing paradoxes and challenges Sizing metrics Sizing methods and tools Sizing research and development needs Conclusions and references

43. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE43 Basic Methods, Strengths, and Weaknesses (Adapted from Boehm, 1981) MethodStrengthsWeaknesses Pair-wise comparisonAccurate assessment of relative size Absolute size of benchmark must be known Expert judgmentAssessment of representativeness, interactions, exceptional circumstances No better than participants Biases, incomplete recall AnalogyBased on representative experience Representativeness of experience ParkinsonCorrelates with some experience Reinforces poor practice Price to winOften gets the contractGenerally produces large overruns Top-downSystem level focus Efficient Less detailed basis Less stable Bottom-upMore detailed basis More stable Fosters individual commitment May overlook system level costs Requires more effort

44. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE44 Counting Artifacts Artifacts: requirements, inputs, outputs, classes, use cases, modules, scenarios –Often weighted by relative difficulty –Easy to count at initial stage –Estimates may differ based on level of detail

45. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE45 Comparison with Previous Projects Comparable metadata: domain, user base, platform, etc. Pair-wise comparisons Differential functionality –analogy; yesterday’s weather

46. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE46 Group Consensus Wideband Delphi –Anonymous estimates –Facilitator provides summary –Estimators discuss results and rationale –Iterative process –Estimates converge after revision in next rounds –Improves understanding of the product’s nature and scope –Works when estimators are collocated Planning Poker (Cohen, 2005) –Game: deck of cards –Moderator provides the estimation-item –Participants privately choose appropriate card from deck –Divergence is discussed –Iterative – convergence in subsequent rounds –Ensures everyone participates –Useful for estimation in agile projects

47. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE47 Probabilistic Methods PERT (Putnam and Fitzsimmons, 1979) –3 estimates: optimistic, most likely, pessimistic –Expected Size = optimistic + 4 * (most likely) + pessimistic –Std. Dev. = (pessimistic – optimistic) / 6 –Easy to use –Ratio reveals uncertainty –Example 6 Componentaiai mimi bibi EiEi δiδi SALES6K10K20K11K2.33K DISPLAY47137.51.5 INEDIT8121912.51.83 TABLES481281.33 TOTALS22376439δ E = 3.6

48. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE48 Why Do People Underestimate Size? (Boehm, 1981) Basically optimistic and desire to please Have incomplete recall of previous experience Generally not familiar with the entire software job

49. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE49 Sizing R & D Needs Counting rules: unambiguous, hard to game –level of detail for implementation-independent metrics (e.g. # of requirements) grows with elaboration Relations among metrics –SLOC per FP, requirement, use case, etc. Accounting for reuse, volatility, complexity –avoiding double counting Automating the counting process Critical success factors for judgment-based sizing methods More empirical data and analysis

50. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE50 Conclusions Size plays an important role in estimation and project management activities Software estimation is a “garbage in garbage out” process –Bad size in; bad cost out A number of paradoxical situations and challenges make estimation of size difficult Different sizing metrics with varying degree of detail are available at different stages of the project lifecycle A plethora of sizing methods exists with each method having a unique mix of strengths and weaknesses Sizing R & D needed to bridge the gap between Systems and Software Engineering

51. University of Southern California Center for Systems and Software Engineering 10/31/2007©USC-CSSE51 References Books –Boehm, Barry W., Software Engineering Economics, Prentice Hall, 1981. –Boehm, Barry W., et al., Software Cost Estimation With COCOMO II, Prentice Hall, 2000. –Brooks, Jr., Frederick P., The Mythical Man-Month, Addison-Wesley, 1995. –Cockburn, A., Writing Effective Use Cases, Addison-Wesley, 2001. –Cohen, M., Agile Estimating and Planning, Prentice Hall, 2005. –Jones, C., Applied Software Measurement: Assuring Productivity and Quality, McGraw-Hill, 1996. –Futrell, Robert T., et al., Quality Software Project Management, Prentice Hall, 2002. –Stutzke, Richard D., Estimating Software-Intensive Systems, Addison-Wesley, 2005. Journals –Chidamber, S. and C. Kemerer, 1994, “A Metrics Suite for Object Oriented Design,” IEEE Transactions on Software Engineering. –Putnam, L.H., and Fitzsimmons, A., 1979. Estimating software costs. Datamation. Tutorials/Lectures/Presentations –Boehm, Barry W., CSCI 577a Lecture, “Cost Estimation with COCOMO II”, 2004. –Boehm, Barry W., BAE Systems SPIRE Tutorial, “The COCOMO II Suite of Software Cost Estimation Models”, 2004. –Boehm, Barry W., Microsoft Presentation, “Software Productivity Perspectives, Paradoxes, Pitfalls, and Prospects”, 1999. –Madachy, Ray, CSCI 510 Lecture, “COCOMO II Overview”, 2005. –Seaver, David, “Tactical Benchmarking – The Rosetta Stone for Linking IT and Business Results”, 2007. –Valerdi, Ricardo, INCOSE Presentation, “The Constructive Systems Engineering Cost Model”, 2005. Websites –http://www.crisp.se/planningpoker/http://www.crisp.se/planningpoker/ –http://www.ifpug.org/http://www.ifpug.org/ –http://www.spr.com/http://www.spr.com/