Cost Estimation Software Quality Assurance and Testing

2 Cost Estimation Is Needed 55% of projects over budget –24 companies that developed large distributed systems 53% of projects cost 189% more than initial estimates –Standish Group of 8,380 projects

3 Cost Estimation An approximate judgment of the costs for a project –Many variables Often measured in terms of effort (i.e., person months/years) Different development environments will determine which variables are included in the cost value –Management costs –Development costs Training costs Quality assurance –Resources

4 Cost Estimation Affects Planning and budgeting –Requirements prioritization –Schedule –Resource allocation Project management –Personnel –Tasks

5 Cost Estimation During the Software Life Cycle Cost estimation should be done throughout the software life cycle to allow for refinement Need effective monitoring and control of the software costs to verify and improve accuracy of estimates –At appropriate level of detail –Gathering data should not be difficult Success of a cost estimate method is not necessarily the accuracy of the initial estimates, but rather the rate at which estimates converge to the actual cost

6 Who is the Estimator? Someone responsible for the implementation –Can compare previous projects in organization to current project –Usually experienced Someone from outside the organization –Can provide unbiased estimate –Tend to use empirical methods of estimation Difficulties: –Lack of confidence that a model will outperform an expert –Lack of historical data to calibrate the model

7 General Steps and Remarks Establish Plan –What data should we gather –Why are we gathering this data –What do we hope to accomplish Do cost estimation for initial requirements –Decomposition Use several methods –There is no perfect technique –If get wide variances in methods, then should re- evaluate the information used to make estimates

8 General Steps and Remarks (cont.) Do re-estimates during life cycle Make any required changes to development Do a final assessment of cost estimation at the end of the project

9 Software Cost Estimation Process Definition –A set of techniques and procedures that is used to derive the software cost estimate Set of inputs to the process and then the process will use these inputs to generate the output

Effort Estimation Estimation Should be Done Repeatedly Uncertainty early in the project can affect the accuracy of cost and size estimations

Effort Estimation Key causes –Frequent request for change by users –User's lack of understanding of the requirements –Insufficient analysis when developing estimate –Lack of coordination of system development, technical services, operations, data administration, and other functions during development –Lack of an adequate method or guidelines for estimating

Effort Estimation (Cont.) Key influences –Complexity of the proposed application system –Required integration with existing system –Complexity of the program in the system –Size of the system expressed as number of functions or programs –Capabilities of the project team members –Project team's experience with the application, the programming language, and hardware –Capabilities of the project team members –Database management system –Number of project team member –Extent of programming and documentation standards

13 Types of Cost Models Experiential –derived from past experience Static –derived using “regression” techniques –doesn’t change with time Dynamic –derived using “regression” techniques –often includes the effects of time

Experiential Estimation Methods Expert judgment – Expert Guessing : pessimistic (x), optimistic (y), most likely (z); estimate as (x + 4y + z)/6 – Delphi technique: based on the average of “secret” expert judgments – Wolverton model: old (mid 70’s) Algorithmic methods: E = (a + bS c ) m(X) – Walston and Felix model: E = 5.25S

15 Expert Guessing A = The most pessimistic estimate (الأكثر تشاؤما). B = The most likely estimate (الأكثر احتمالا). C = The most optimistic estimate (الأكثر تفاؤلا). Ê = (A + 4B + C) 6 (Weighted average; where Ê = estimate).

16 Delphi Technique 1.Group of experts, make "secret" guesses. 2."secret" guesses are used to compute group average. 3.Group average is presented to the group. 4.Group, once again makes "secret" guesses. 5.Individual guesses are again averaged. 6.If new average is different from previous, then goto (4). 7.Otherwise Ê = new average.

17 Expert Judgment Advantages –Useful in the absence of quantified, empirical data. –Can factor in differences between past project experiences and requirements of the proposed project –Can factor in impacts caused by new technologies, applications and languages. Disadvantages –Estimate is only as good expert’s opinion –Hard to document the factors used by the experts

18 Function Points ParameterSimple+Average+Complex=FiFi Distinct input items3( )+4( )+6( )=? Output screens/reports4( )+5( )+7( )=? Types of user queries3( )+4( )+6( )=? Number of files7( )+10( )+15( )=? External interface5( )+7( )+10( )=? Total=?

19 Function Point Equation F.P.’s = T * ( * Q) T = unadjusted table total Q = score from questionnaire (14 items with values 0 to 5) Cost of producing one function point? May be organization specific.

20 Function Point Questionnaire 1.Backup. 2.Data communication. 3.Distributed processes. 4.Optimal performance. 5.Heavily used operating system. 6.On-line data security. 7.Multiple screens. 8.On-line master file update. 9. Complex inputs, queries, outputs. 10. Complex internal processing. 11. Reusable code. 12. Conversion or installation. 13. Multiple user sites. 14. Ease of use.

Demo urse.des/cis525/js/f00/artan/functionpoints.htm 21

Algorithmic Method: Watson and Felix Model A productivity index is included in the equation There are 29 factors that can affect productivity –1 if increase the productivity –0 if decrease the productivity 22

23 Watson and Felix Model E = 5.25 * S 0.91 composite productivity factor p =  w i * x i L = LOC per person-month = f(p) E = S / L

Watson and Felix Model Productivity Factors 24

25 COCOMO Model COCOMO stands for COnstructive COst MOdel It is an open system First published by Dr Barry Bohem in 1981 Worked quite well for projects in the 80’s and early 90’s Could estimate results within ~20% of the actual values 68% of the time

26 COCOMO II Main objectives of COCOMO II: –To develop a software cost and schedule estimation model tuned to the life cycle practices –To develop software cost database and tool support capabilities for continuous model improvement

Has three different models: Application composition – Good for projects built using GUI development tools – size estimates in object points Early design – This model can get rough estimates before the entire architecture has been decided – size estimates in function points Post-architecture – Most detailed model, used after overall architecture has been decided on – size estimates in lines of code COCOMO II

28 COCOMO Model COCOMO has three different models (each one increasing with detail and accuracy): –Basic, applied early in a project –Intermediate, applied after requirements are specified. –Advanced, applied after design is complete COCOMO has three different modes: –Organic – “relatively small software teams develop software in a highly familiar, in-house environment” –Embedded – operate within tight constraints, product is strongly tied to “complex of hardware, software, regulations, and operational procedures” –Semi-detached – intermediate stage somewhere between organic and embedded.

COCOMO COCOMO uses two equations to calculate effort in man months (MM) and the number on months estimated for project (TDEV) MM is based on the number of thousand lines of delivered instructions/source (KDSI) MM = a(KDSI) b * EAF TDEV = c(MM) d EAF is the Effort Adjustment Factor derived from the Cost Drivers, EAF for the basic model is 1 The values for a, b, c, and d differ depending on which mode you are using Modeabcd Organic Semi-detached Embedded

COCOMO Equations Organic Mode: E = 2.4 * (KLOC)^1.05 D = 2.5 * (E)^0.38 Semi-Detached Mode: E = 3.0 * (KLOC)^1.12 D = 2.5 * (E)^0.35 Embedded Mode: E = 3.6 * (KLOC)^1.20 D = 2.5 * (E)^

31 Example A simple example: Project is a flight control system (mission critical) with 310,000 DSI in embedded mode Reliability must be very high (RELY=1.40). So we can calculate: Effort = 1.40*3.6*(319) 1.20 = 5093 MM Schedule = 2.5*(5093) 0.32 = 38.4 months Average Staffing = 5093 MM/38.4 months = 133 FSP

Demo des/cis525/js/f00/gamel/help.html des/cis525/js/f00/gamel/cocomo.html Download COCOMOII ocomo_downloads.htm 32

33 Is COCOMO the Best? COCOMO is the most popular method however for any software cost estimation you should really use more then one method Best to use another method that differs significantly from COCOMO so your project is examined from more then one angle COCOMO is the most popular software cost estimation method Easy to do, small estimates can be done by hand

34 Static Model Problems Existing models rely at least in part on expert judgment Most static estimates require estimation of the product in lines of code (LOC) Not clear which cost factors are significant in all development environments

35 Dynamic Models It is helpful to know when effort will be required on a project as well as how much total effort is required Most models are time or phase sensitive in their effort computations

36 Putnam Model The Putnam model is an empirical software effort estimation model.

37 Putnam Model Details Putnam used his observations about productivity levels to derive the software equation:

38 Putnam Equations where: Size is the product size (whatever size estimate is used by your organization is appropriate). B is a scaling factor and is a function of the project size. Productivity is the Process Productivity, the ability of a particular software organization to produce software of a given size at a particular defect rate. Time is the total schedule of the project in years. Effort is the total effort applied to the project in person-years.

39 Parr Model

40 Parr Equation Putnam variation Staff may already be familiar with project tools, methods, and requirements Staff(t) = (sech 2 (a*t + c) / 2) / 4

41 Jensen Model Putnam variation Less sensitive to schedule compression than Putnam S = C te * T * K 1/2

42 Cooperative Programming Model Includes size of project team in estimate as well as code size E = E 1 (S) + E 2 (M) S = code size M = average # team members E 1 (S) = a + b * S effort of single team member E 2 (M) = c * M d effort required for coordination with other members

43 Dynamic Model Problems Still rely on expert judgment Not clear that all project costs have been accommodated here either