Introduction to Software Project Estimation I (Overview) Barry Schrag Software Engineering Consultant MCSD, MCAD, MCDBA

What Are Your Expectations From this class?

Introductions Your background Reason for taking this workshop An unusual fact to remember you by!

What we will learn This course focuses on an overview of Software Estimation knowledge to include the tools, techniques, and key concepts used in estimating size and effort on software projects. Class content, including hands-on exercises, is designed to provide team members and Project Managers with resources and skills to more accurately apply estimating techniques at various points in a software engineering process and how to interpret the result, reducing risk during the software engineering effort. Prerequisite: Experience in the field of software engineering or project management, or relevant coursework/knowledge.

Info Breaks Food Facilities

Workshop Format Introduction Exercise without tools Break Lines of Code Lesson Function Point Lesson Exercise applying Function Point technique Break Lessons learned

Software Project Definitions Successful: The project is completed on time and on budget, with all features and functions as specified Challenged: The project is completed and operational, but over budget, late, and with fewer features and functions that initially specified Failed: The project is cancelled before completion, never implemented, or scrapped following installation

Project Success is Rare 2004: 15% failed, 51% challenged, 34% succeeded 2000: 23% failed, 49% challenged, 28% succeeded 1995: 40% failed, 33% challenged, 27% succeeded 1994: 31% failed, 53% challenged, 16% succeeded Source: Randy Miller, Microsoft

Project Status (Standish 2004)

Overview Why care about estimates? Repeatability Accuracy to a date Manage resources and cost FP standards group says: +-4%  12% LOC estimation error is history dependent % down to +-5% with PSP/PROBE

What is an "estimate" ? Our Premise: Effort in hours from size More accurate for your team/process Do engineers estimate? Do architects? Do scientists? Size  Effort hours  Schedule

Cone of Uncertainty It is very large during requirements It gets smaller as you proceed through your process!

Scenario I Read Exercise 1 Initial Estimate Solidify any questions and assumptions Can you make an initial estimate in hours? If not, make a guess! If you can, estimate how many defects there will be

Results What were your estimates? What tools would you use to perform this task?

Some facts help in estimation Organization Is there IT Support? What is their current and projected load? Engineering team in place? Engineering Methodology System Specs OS Platform Architecture Backend Database/File System, etc… Calculation rule specifics? Report output method /Architecture in place? Retention rate of data? Quality Specification Performance? Concurrency? Sales Lead time Pipleline already full? Project Management Supplier lead time Partner co-operation Partner dependencies

Important for Estimation What quality is required How many defects are acceptable after release? How many defects do we need to find? What is our team size What is our team productivity (do we know?) How many hours is the team is available to work on this project per day? 6? 8? 10? Are vacation days scheduled?

An Estimator uses Product Specs (size only) Team Size (size  effort) PSP uses a team size of 1 Hours available per day for work (effort  schedule) Days off (effort  schedule)

The Cone of Uncertainty What does it look like? What percent of software projects are on time and on budget? 29% Where does estimation error come from? Guessing Historical Analysis

Estimation Error Unknown specs account for 33% of error (McConnell 2006) Lost project hours account for error on schedule estimation, not size estimation Remember Size leads to Effort leads to Schedule

Scope Creep 2% increase per calendar month in design and coding phases (McConnell 2006) From end of requirements to start of coding phases chart: (Capers Jones, 2002)

Estimation Formula Effort in hours Size of Specs Size of Team How do we find the size of specs? Only two ways accepted as better than guessing! Function Points Lines of Code using PROBE in the PSP

Line of Code Definitions KLOC (thousands) SLOC (source) LOC CLOC (commented) NCLOC (non commented)

System Size: Lines of Code Review Table 1 The Paradox of Source Code Metrics Read Reading 1 Lines of Code as a metric Read Reading 2 A Few Words about the PSP and PROBE

Size estimating LOC/PROBE

LOC PSP/PROBE defects A benefit of this method: % of defects can be found before first compile!

LOC Has Problems! No theoretical foundation No relationship between lines of code and program operation C = A+B; C = *A + *B; Complexity and errors can increase with equal LOC

System Size: Lines of Code There is no standard way to measure Wide range of estimates for a language Visual Basic 15 to 41 LOC! LOC counts can be easily misinterpreted and misused. Don’t mix LOC counts from different languages and types of code (i.e. test support, product etc…) You can generate almost any productivity number you want by changing the way you count LOC. Tools? Code Counter Pro Can estimate ratio of Comment Lines per SLOC

Break Be back in …. Next up Function Points

System Size: Function Points 1979, A.J. Albrecht of IBM published a Function Point metric as a ‘measure of software productivity’ or unit of work.

An Ideal Size Measure Should Be Measurable Be Accountable Be Precise Be Independent of the measurer

System Size: Function Points Albrecht considered five operations The inputs to the application The outputs from it Inquiries by users (define user) The data files that would be updated by the application The interfaces to other applications

The generic application Data values in Output simple data Output Calculated data Data store Application

Modern Function Points After research, empirical weighting factors became a standard The number of inputs was weighted by 4 The Outputs by 5 Inquiries by 4 The data file updates by 10 The interfaces by 7 These weights change based on number of data fields used by each operation

System Size: Function Points IFPUG (International Function Point Users Group) FP ISO 20626: COSMICON (COmmon Software Measurement International CONsortium) FFP ISO/IEC 19761: FP measures size of an operation not the direct complexity of algorithms Abstracted from language or implementation

FP Defect Rates are Known DEFECT ORIGINSPer FP Requirements1.00 Design1.25 Coding1.75 Documentation0.60 Bad Fixes0.40 TOTAL5.00

Generic Application Data values in External Input Output Calculated data Data store (Internal Logical Files) Application Output simple data

External Outputs (EO) An elementary process in which derived data passes across the boundary from inside to outside the application A report where data is calculated is an example For elaborated definition see Glossary

External Inputs (EI) Is an elementary process in which data crosses the boundary from outside to inside the application For elaborated definition see Glossary

External inQuiry (EQ) An elementary process with both input and output components that result in data retrieval from one or more internal logical files and external interface files A report where data is pre-calculated is an example For elaborated definition see Glossary

Internal Logical Files (ILF) A group of logically related data within the application boundary Storage location for the users profile, for product, system control info… For elaborated definition see Glossary

Rating Logical Files

External Interface Files (EIF) Data used for reference purposes which resides entirely outside the application and is maintained by another application This is an Internal Logical File for another application For elaborated definition see Glossary

EQ and EO IFPUG is very clear on the difference between EQ and EO EQ should NOT update an ILF, no derived data is created and NO formula or calculations should be performed i.e. an EQ is only a query May require a certified FPA to be sure!

Function Point Terms Diagram

Note About Terms There may be more than one of ANY of the 5 FP operations Often more than one ILF in even the smallest project Imagine one ILF for Users, another for Invoice, Products, System Control data

FPA Exercise (Exercise I) We want to build an internet based system which signs up users who want service. The system will record the user information. It will look up the service price for the monthly payment and display this to the user before they approve. It will use the payment rates which are held in a legacy system. Once this payment is calculated, it is stored with the users data and is not re-calculated again. The system must provide two reports; 1) a monthly report which details the invoice for the user, and 2) an on- demand report for management which aggregates the projected income across all signed up users for the monthly period

Exercise I – How Many? Match these concepts to the exercise ILF (Internal Logical Files) EIF (External Interface Files) EI (External Input) EO (External Output) -- calculated EQ (External inQuiry) -- just a query

Exercise I - Calculate FP Count The number of External Inputs __*4= __ The External Outputs __*5= __ External inQuiries __*4= __ The Internal Logical Files __*10= __ The External InterFaces __*7= __ TOTAL FP Estimate = __ TOTAL Defects Estimate = FP * 5 = __

Exercise I - Calculate FP Count The number of External Inputs 2*4= __ The External Outputs 2*5= __ External inQuiries 2*4= __ The Internal Logical Files 1*10= __ The External InterFaces 1*7= __ TOTAL FP Estimate = __ TOTAL Defects Estimate = FP * 5 = __

Exercise I - Calculate FP count The number of External Inputs 2*4= 8 The External Outputs 2*5=10 External inQuiries 2*4= 8 The Internal Logical Files 1*10= 10 The External InterFaces 1*7= 7 TOTAL FP Estimate = 43 TOTAL Defects Estimate = FP * 5 =215

Exercise I TOTAL FP Estimate = 43 Note that this is an independent measure of the SIZE of the counted functionality!

Exercise I hours to release TOTAL FP Estimate = 43 EFFORT = FP * process efficiency Now apply the variables hours to release = 43 unadjusted function points * 12 hours/fp Note that 12 is a LOW estimate of process efficiency

Note the Weighting Factors Weighting changes based on complexity but for that use a certified FP analyst! We will assume average complexity Example: Data Elements, see counting practices manual!

Using Agile process + FPA Agile: Analyze, Test, Code, Deliver Agile FPA: Analyze, Measure, Test, Code, Deliver Use the count as an opportunity to elaborate on the requirements Track estimates over time

Historical Effort Estimation ISBSG (International Software Benchmarking Standards Group) Assumes Average Productivity Staff Months =.425 * (FP Count ^.488) * (Maximum Team Size ^.697) Assumes 132 project focused hours per staff month Assumes 3rd GL, calibrated by 600 projects

Historical Measurement Thousands of projects Consistent sizing with FPA Record of time for each activity Trends emerge Some activities are not performed on every project Cost for the activity doesn’t vary based on project type

Activities by Project Type Activity by Project Type ActivityEnd UserMISOutsourceCommercialSystemsMilitary Requirements XXXXX PrototypingXXXXXX Architecture XXXXX Project Plans XXXXX Initial Design XXXXX Detailed Design XXXXX Design Reviews XXXX CodingXXXXXX Reuse AcquisitionX XXXX Package Purchase XX XX Code Inspections XXX Independent verification and validation X Configuration Management XXXXX Integration XXXXX User DocumentationXXXXXX Unit TestingXXXXXX Function Testing XXXXX Integration Testing XXXXX System Testing XXXXX Field Testing XXX Acceptance Testing XX XX Independent Testing X Quality Assurance XXXX Installation and Training XX XX Project Management XXXXX Commonly Used Activities*

National Average Productivity Work Hours per FP ActivityMinimumModeMaximum Requirements Prototyping Architecture Project Plans Initial Design Detailed Design Design Reviews Coding Reuse Acquisition Package Purchase Code Inspections Independent verification and validation Configuration Management Integration User Documentation Unit Testing Function Testing Integration Testing System Testing Field Testing Acceptance Testing Independent Testing Quality Assurance Installation and Training Project Management Total hours per Function Point

EFFORT is Estimated by Wideband Delphi-consensus-based NO! Perform organization calibration to get Hours per Function Point Historical Data gets better over time

Estimation Influences Are Additive Error due to Size  Effort hours  Schedule Error in Size Estimate Error in Effort Estimate Productivity changes due to New team size Work tasks change Hours available to work are altered

Estimation Techniques Function Points estimate size independently, and can find effort hours after one use PSP/PROBE Proxy-based estimates guess about a size to find the effort hours, but get better over time See chart slide 24

Software Estimation Tools PSP/PROBE is an estimation tool Function Points are an estimation tool LOC counting can be automated, but is only useful for comment lines and PSP Function Points are not easy to automate!

Estimation Procedures First Estimate Size Count Function Points as a size measurement Or estimate LOC using PSP/PROBE method Determine Productivity Hours/FP or Hours/LOC Calibrate using local history Total Effort Hours Size FP * Hours/FP or Use PSP/TSP/PROBE to determine total hours

Estimation Method Costs Function Point certification PSP/TSP and PROBE training

Project Cost Estimation Effort x (Salary + Burden) = Cost COCOMO 82  COCOMOII (2000)  F2COCOMO Requirement phase and cost is not estimated by any COCOMO method! Assumes 152 hours project time/month F2COCOMO see; pdf

Size Issues Using FP Little applicability to effort without historical data Some Standards are in place, ISO, IEEE History is available using ISBSG database

Size Issues Using LOC Problems applying it to different code bases i.e. SQL Data Driven Applications XML, XSLT, ASP, VBScript, Jscript, ASP.NET No standards Can’t convert between size models From LOC to FP or FP to LOC NO! Range of LOC/FP is too large to use!

Effort Estimation Issues Effort = Size * Productivity Productivity measured as hours/Function Point Use local productivity Data and ISBSG averages Team history and cohesion do affect results Main point - Record hours worked

Schedule Estimation Issues Support Documentation QA (How many defects do you want removed?) Change Requests which are implemented Turnover Team History Record hours worked to do your next estimate Process change

Process Change Issues About failure rates of metrics estimation initiatives Steven Kan, 2000 Management buy-in is most critical!

Exercise II Read Exercise II in the handout and perform a rough Function Point size estimate using the information given Derive hours to complete product using 12 hours per function point efficiency Estimate defects in product using the US standard of 5 defects per Function Point

Scenario II -- what is critical? The user will be presented with a calendar They may choose a date or a holiday They will identify it with a label They will then choose an notification This data is recorded in the database The system will provide an notification

Scenario II how many items? The user will be presented with a calendar Calendar Date ILF, EO They may choose a date or a holiday EI They will identify it with a label EI They will then choose an notification EI This data is recorded in the database User Event ILF The system will provide an notification EO

Scenario II EI 3*4 = 12 EO 2*5 = 10 EQ 0 * 4 = 0 ILF 2 * 10 = 20 We are still assuming average complexity! TOTAL Unadjusted Function Points = 42 DEFECTS = 42 * 5 = 210

Exercise II hours to release TOTAL FP Estimate = 42 EFFORT = FP * process efficiency Now apply the variables hours to release = 42 unadjusted function points * 20 hours/fp Remember that 20 is an average estimate of process efficiency

Effort Analysis FP count of SIZE is about equal Exercise I FP count = 43 about 516 hours Exercise II FP count = 42 about 840 hours Why is there a large hours difference? Calibrate your process efficiency

What did we learn? Overview of Software Estimation knowledge Tools, techniques, and key concepts Size and Effort Resources and skills to more accurately apply estimating techniques

Conclusion Choose an estimating technique Make it part of your process at each step and for each change requested It can reveal process efficiency Track error over time and use to predict cone of uncertainty in the next cycle

Conclusion -- Costs PSP training cost = 2 weeks (80 hours) After 2 hours of use, your team process is CMMi Level 5 TSP training is available for managers Function Point certification costs = 2 days (16 hours) Will help if you keep history between projects No projections about CMMi level

Follow OnCourse Function Point Analysis Certification PSP Training Be sure to write it in your course evaluation if you are inclined to work toward either certification