Metrics

A Good Manager Measures measurement What do we use as a basis? size? size? function? function? project metrics process metrics process product product metrics

Why Do We Measure? assess the status of an ongoing project track potential risks uncover problem areas before they go “critical,” adjust work flow or tasks, evaluate the project team’s ability to control quality of software work products.

Measures, Metrics and Indicators A measure provides a quantitative indication of the extent, amount, dimension, capacity, or size of some attribute of a product or process The IEEE glossary defines a metric as “a quantitative measure of the degree to which a system, component, or process possesses a given attribute.” An indicator is a metric or combination of metrics that provide insight into the software process, a software project, or the product itself

Process Measurement We measure the efficacy of a software process indirectly. That is, we derive a set of metrics based on the outcomes that can be derived from the process. That is, we derive a set of metrics based on the outcomes that can be derived from the process. Outcomes include Outcomes include measures of errors uncovered before release of the software defects delivered to and reported by end-users work products delivered (productivity) human effort expended calendar time expended schedule conformance other measures. other measures. We also derive process metrics by measuring the characteristics of specific software engineering tasks.

Process Metrics Guidelines Use common sense and organizational sensitivity when interpreting metrics data. Provide regular feedback to the individuals and teams who collect measures and metrics. Don’t use metrics to appraise individuals. Work with practitioners and teams to set clear goals and metrics that will be used to achieve them. Never use metrics to threaten individuals or teams. Metrics data that indicate a problem area should not be considered “negative.” These data are merely an indicator for process improvement. Don’t obsess on a single metric to the exclusion of other important metrics.

Software Process Improvement SPI Process model Improvement goals Process metrics Process improvement recommendations

Process Metrics Quality-related focus on quality of work products and deliverables focus on quality of work products and deliverablesProductivity-related Production of work-products related to effort expended Production of work-products related to effort expended Statistical SQA data error categorization & analysis error categorization & analysis Defect removal efficiency propagation of errors from process activity to activity propagation of errors from process activity to activity Reuse data The number of components produced and their degree of reusability The number of components produced and their degree of reusability

Project Metrics used to minimize the development schedule by making the adjustments necessary to avoid delays and mitigate potential problems and risks used to assess product quality on an ongoing basis and, when necessary, modify the technical approach to improve quality. every project should measure: inputs—measures of the resources (e.g., people, tools) required to do the work. inputs—measures of the resources (e.g., people, tools) required to do the work. outputs—measures of the deliverables or work products created during the software engineering process. outputs—measures of the deliverables or work products created during the software engineering process. results—measures that indicate the effectiveness of the deliverables. results—measures that indicate the effectiveness of the deliverables.

Typical Project Metrics Effort/time per software engineering task Errors uncovered per review hour Scheduled vs. actual milestone dates Changes (number) and their characteristics Distribution of effort on software engineering tasks

Typical Size-Oriented Metrics errors per KLOC (thousand lines of code) defects per KLOC $ per LOC pages of documentation per KLOC errors per person-month Errors per review hour LOC per person-month $ per page of documentation

Typical Function-Oriented Metrics errors per FP (thousand lines of code) defects per FP $ per FP pages of documentation per FP FP per person-month

Comparing LOC and FP Representative values developed by QSM

These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, Software Project Planning The overall goal of project planning is to establish a pragmatic strategy for controlling, tracking, and monitoring a complex technical project. Why? So the end result gets done on time, with quality!

These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, Steps in Estimating Estimate cost and effort Decompose the problem Decompose the problem Develop two or more estimates using size, function points, process tasks or use-cases Develop two or more estimates using size, function points, process tasks or use-cases Reconcile the estimates Reconcile the estimates

16 Estimation Estimation of resources, cost, and schedule for a software engineering effort requires experience experience access to good historical information (metrics access to good historical information (metrics the courage to commit to quantitative predictions when qualitative information is all that exists the courage to commit to quantitative predictions when qualitative information is all that exists Estimation carries inherent risk and this risk leads to uncertainty

17 Project Estimation Project scope must be understood Elaboration (decomposition) is necessary Historical metrics are very helpful At least two different techniques should be used Uncertainty is inherent in the process

18 Estimation Techniques Past (similar) project experience Conventional estimation techniques task breakdown and effort estimates task breakdown and effort estimates size (e.g., FP) estimates size (e.g., FP) estimates Empirical models Automated tools

19 Estimation Accuracy Predicated on … the degree to which the planner has properly estimated the size of the product to be built the degree to which the planner has properly estimated the size of the product to be built the ability to translate the size estimate into human effort, calendar time, and dollars (a function of the availability of reliable software metrics from past projects) the ability to translate the size estimate into human effort, calendar time, and dollars (a function of the availability of reliable software metrics from past projects) the degree to which the project plan reflects the abilities of the software team the degree to which the project plan reflects the abilities of the software team the stability of product requirements and the environment that supports the software engineering effort. the stability of product requirements and the environment that supports the software engineering effort.

20 Functional Decomposition functionaldecomposition StatementofScope Perform a Grammatical “parse”

21 Conventional Methods: LOC/FP Approach compute LOC (lines of code)/FP (function points) using estimates of information domain values use historical data to build estimates for the project

22 Example: LOC Approach Average productivity for systems of this type = 620 LOC/pm. Burdened labor rate =$8000 per month, the cost per line of code is approximately $13. Based on the LOC estimate and the historical productivity data, the total estimated project cost is $431,600 and the estimated effort is 54 person-months.

Function Points Number of external inputs – from user or another application Number of external outputs Number of external inquiries – request from user that generates an on-line output Number of internal logical files (maintained by system) Number of external interface files (provides data but not maintained by system) FP = count-total X [ X Sum (F i )] 23

Count-Total Count-Total = sum(number X weight) Where weights are: Simple Average Complex Simple Average Complex External Inp346 External Out457 External Inq346 Internal Files71015 External Files

Calculation of sum(F i ) Total of 1-5 rating of following 14 questions: Does the system require reliable back-up/recovery? Does the system require reliable back-up/recovery? Are specialized data communications required? Are specialized data communications required? Are there distributed processing functions? Are there distributed processing functions? Is performance critical? Is performance critical? Will run in heavily utilized operating environment? Will run in heavily utilized operating environment? On-line data entry required? On-line data entry required? For on-line data entry, will it require multiple screens? For on-line data entry, will it require multiple screens? Are ILF’s updated on-line? Are ILF’s updated on-line? Are input, output, files, or inquiries complex? Are input, output, files, or inquiries complex? Is the internal processing complex? Is the internal processing complex? Is the code designed to be reusable? Is the code designed to be reusable? Are conversion and installation included? Are conversion and installation included? Is the system designed for installation in different organizations? Is the system designed for installation in different organizations? Is the application designed to facilitate change and ease of use? Is the application designed to facilitate change and ease of use? 25

26 Example: FP Approach The estimated number of FP is derived: FP estimated = count-total X [ X Sum (F i )] FP estimated = 375 organizational average productivity = 6.5 FP/pm. burdened labor rate = $8000 per month, the cost per FP is approximately $1230. Based on the FP estimate and the historical productivity data, the total estimated project cost is $461,000 and the estimated effort is 58 person-months.

Software Sizing

Software Project Estimation Software Sizing: Fuzzy logic sizing: establish software’s magnitude Fuzzy logic sizing: establish software’s magnitude Function point sizing Function point sizing Standard component sizing Standard component sizing # of generic components Historical project size Change sizing Change sizing # of changes required Type of change required e.g. add, change, delete

Software Project Estimation Expected value of Size can be measured as: S=(S opt +4S m +S pess )/6 S=(S opt +4S m +S pess )/6 Where Where S opt = Optimistic S opt = Optimistic S m =Most Likely S m =Most Likely S pess =Pessimistic S pess =Pessimistic

LOC Based Estimation You are required to establish estimates for Virtual Class room System. For first increment you have to provide automation of course registration, Attendance, lectures. LOC estimates for each Functionality are provided: User screens 3000 User screens 3000 Course registration 1550 Course registration 1550 Attendance 1800 Attendance 1800 Lectures 3000 Lectures 3000 Student query 800 Student query 800

LOC Based Estimation Suppose productivity of your Organization is 500 LOC/pm Suppose productivity of your Organization is 500 LOC/pm Labor rate is 8000Rs/pm Labor rate is 8000Rs/pm Calculate Effort and Total cost of this increment. Calculate Effort and Total cost of this increment.

Total LOC User screens 3000 User screens 3000 Course registration 1550 Course registration 1550 Attendance 1800 Attendance 1800 Lectures 3000 Lectures 3000 Student query 800 Student query 800 = Total LOC=10150 LOC

Effort LOC/ Productivity =10150/500 =20.3 pm or 21 pm

Total Cost Total cost=Rate * pm = 8000 * 21 = Rs. Cost per LOC =168000/10150 =16.55 Rs.

FP based Estimation For above mentioned system Student can be registered Student can be registered Courses can be added/updated. Courses can be added/updated. Student can view courses Student can view courses Student can inquire his/her registration status Student can inquire his/her registration status

Count Simple Average Complex Count Simple Average Complex External Inp3 * = 9 External Out4 * =16 External Inq1 * = 3 Internal Files4 * =28 External Files0 * = 0 Count Total= =56

Calculation of sum(F i ) Total of 0-5 rating of following 14 questions: Does the system require reliable back-up/recovery? 3 Does the system require reliable back-up/recovery? 3 Are specialized data communications required? 2 Are specialized data communications required? 2 Are there distributed processing functions? 0 Are there distributed processing functions? 0 Is performance critical? 2 Is performance critical? 2 Will run in heavily utilized operating environment? 1 Will run in heavily utilized operating environment? 1 On-line data entry required? 4 On-line data entry required? 4 For on-line data entry, will it require multiple screens? 5 For on-line data entry, will it require multiple screens? 5 Are ILF’s updated on-line? 5 Are ILF’s updated on-line? 5 Are input, output, files, or inquiries complex? 2 Are input, output, files, or inquiries complex? 2 Is the internal processing complex? 3 Is the internal processing complex? 3 Is the code designed to be reusable? 0 Is the code designed to be reusable? 0 Are conversion and installation included? 1 Are conversion and installation included? 1 Is the system designed for installation in different organizations? 0 Is the system designed for installation in different organizations? 0 Is the application designed to facilitate change and ease of use? 1 Is the application designed to facilitate change and ease of use? 1 37

The estimated number of FP is derived: FPestimated = count-total X [ X Sum (Fi)] FPestimated = 56 * ( *29) =52.64 or 53 =52.64 or 53 If organizational average productivity = 5 FP/pm. labor rate = 8000Rs. per month, Cost = labor rate * Person Month Based on the FP estimate and the historical productivity data, the total estimated project cost is 84800Rs. and the estimated effort is 10.6 person-months.

Reference Pressman 15, 22,23 Section: ,15.3, 22.2,23.6 Section: ,15.3, 22.2,23.6