Objectives Introduce the necessity for software metrics Differentiate between process, project and product metrics Compare and contrast Lines-Of-Code (LOC) and Function Point (FP) metrics Consider how quality is measured in Software Engineering Describe statistical process control for managing variation between projects

Terminology Measure: Quantitative indication of the extent, amount, dimension, or size of some attribute of a product or process. A single data point Metrics: The degree to which a system, component, or process possesses a given attribute. Relates several measures (e.g. average number of errors found per person hour) Indicators: A combination of metrics that provides insight into the software process, project or product Direct Metrics: Immediately measurable attributes (e.g. line of code, execution speed, defects reported) Indirect Metrics: Aspects that are not immediately quantifiable (e.g. functionality, quantity, reliability) Faults:  Errors: Faults found by the practitioners during software development  Defects: Faults found by the customers after release

Process Metrics Focus on quality achieved as a consequence of a repeatable or managed process. Strategic and Long Term. Statistical Software Process Improvement (SSPI). Error Categorization and Analysis: All errors and defects are categorized by origin The cost to correct each error and defect is recorded The number of errors and defects in each category is computed Data is analyzed to find categories that result in the highest cost to the organization Plans are developed to modify the process

Process Metrics Focus on quality achieved as a consequence of a repeatable or managed process. Strategic and Long Term. Statistical Software Process Improvement (SSPI). Error Categorization and Analysis: All errors and defects are categorized by origin The cost to correct each error and defect is recorded The number of errors and defects in each category is computed Data is analyzed to find categories that result in the highest cost to the organization Plans are developed to modify the process

Project Metrics Used by a project manager and software team to adapt project work flow and technical activities. Tactical and Short Term. Purpose:  Minimize the development schedule by making the necessary adjustments to avoid delays and mitigate problems  Assess product quality on an ongoing basis Metrics:  Effort or time per SE task  Errors uncovered per review hour  Scheduled vs. actual milestone dates  Number of changes and their characteristics  Distribution of effort on SE tasks

Product Metrics Focus on the quality of deliverables Product metrics are combined across several projects to produce process metrics Metrics for the product:  Measures of the Analysis Model  Complexity of the Design Model 1.Internal algorithmic complexity 2.Architectural complexity 3.Data flow complexity  Code metrics

Metrics Guidelines Use common sense and organizational sensitivity when interpreting metrics data Provide regular feedback to the individuals and teams who have worked to collect measures and metrics. Don’t use metrics to appraise individuals Work with practitioners and teams to set clear goals and metrics that will be used to achieve them Never use metrics to threaten individuals or teams Metrics data that indicate a problem area should not be considered “negative.” These data are merely an indicator for process improvement Don’t obsess on a single metric to the exclusion of other important metrics

Typical Metrics Size-Oriented:  errors per KLOC (thousand lines of code), defects per KLOC, R per LOC, page of documentation per KLOC, errors / person- month, LOC per person-month, R / page of documentation Function-Oriented:  errors per FP, defects per FP, R per FP, pages of documentation per FP, FP per person-month ProjectLOCFP Effort (P/M) R(000 ) Pp. doc Error s Defect s People alpha beta gamma

Computing Function Points Establish count for input domain and system interfaces Analyze information domain of the application and develop counts Weight each count by assessing complexity Assign level of complexity (simple, average, complex) or weight to each count Grade significance of external factors, F_i, such as reuse, concurrency, OS,... Assess the influence of global factors that affect the application Compute function points FP = SUM(count x weight) x C where complexity multiplier C = ( x N) degree of influence N = SUM(F_i)

Analyzing the Information Domain

Taking Complexity into Account Complexity Adjustment Values (F_i) are rated on a scale of 0 (not important) to 5 (very important): 1.Does the system require reliable backup and recovery? 2.Are data communications required? 3.Are there distributed processing functions? 4.Is performance critical? 5.System to be run in an existing, heavily utilized environment? 6.Does the system require on-line data entry? 7.On-line entry requires input over multiple screens or operations? 8.Are the master files updated on-line? 9.Are the inputs, outputs, files, or inquiries complex? 10.Is the internal processing complex? 11.Is the code designed to be reusable? 12.Are conversion and instillation included in the design? 13.Multiple installations in different organizations? 14.Is the application designed to facilitate change and ease-of-use?

Exercise: Function Points Compute the function point value for a project with the following information domain characteristics: Number of user inputs: 32 Number of user outputs: 60 Number of user enquiries: 24 Number of files: 8 Number of external interfaces: 2 Assume that weights are average and external complexity adjustment values are not important. Answer:

Example: SafeHome Functionality User SafeHome System Sensors User Monitor and Response System System Config Data Password Zone Inquiry Sensor Inquiry Panic Button (De)activate Messages Zone Setting Sensor Status (De)activate Alarm Alert Password, Sensors, etc. Test Sensor

Example: SafeHome FP Calc complexity multiplier function points number of user inputs number of user outputs number of user inquiries number of files number of ext.interfaces measurement parameter count weighting factor simple avg. complex = = = = = count-total X X X X X

15 Defect Removal Efficiency Defect removal efficiency provides benefits at both the project and process level It is a measure of the filtering ability of QA activities as they are applied throughout all process framework activities –It indicates the percentage of software errors found before software release It is defined as DRE = E / (E + D) –E is the number of errors found before delivery of the software to the end user –D is the number of defects found after delivery As D increases, DRE decreases (i.e., becomes a smaller and smaller fraction) The ideal value of DRE is 1, which means no defects are found after delivery DRE encourages a software team to institute techniques for finding as many errors as possible before delivery

Control Chart Compare sequences of metrics values against mean and standard deviation. e.g. metric is unstable if eight consecutive values lie on one side of the mean. Mean - std. dev. + std. dev.

17 Motivation The software cost estimation provides: the vital link between the general concepts and techniques of economic analysis and the particular world of software engineering. Software cost estimation techniques also provides an essential part of the foundation for good software management.

18 Cost of a project The cost in a project is due to: –due the requirements for software, hardware and human resources –the cost of software development is due to the human resources needed –most cost estimates are measured in person-months (PM)

19 Project Management 1.Logic is a systematic method of coming to the wrong conclusion with confidence. 2.Technology is dominated by those who manage what they do not understand. 3.Nothing ever gets built on schedule or within budget.

20 Cost of a project (.) the cost of the project depends on the nature and characteristics of the project, at any point, the accuracy of the estimate will depend on the amount of reliable information we have about the final product.

21 Software Cost Estimation

22 Introduction to COCOMO models The COstructive COst Model (COCOMO) is the most widely used software estimation model in the world. It The COCOMO model predicts the effort and duration of a project based on inputs relating to the size of the resulting systems and a number of "cost drives" that affect productivity.

23 Productivity Productivity equation –(LOC) / (PM) where PM = number of person-month (156hrs), LOC = "delivered source instructions"

24 Effort Effort Equation –PM = C * (KLOC) n (person-months) where PM = number of person-month, C = a constant, KLOC = thousands of "delivered source instructions" and n = a constant.

25 Productivity Productivity equation –(LOC) / (PM) where PM = number of person-month, LOC = "delivered source instructions"

26 Schedule Schedule equation –TDEV = C * (PM) n (months) where TDEV = number of months estimated for software development.

27 Average Staffing Average Staffing Equation –(PM) / (TDEV) (FSP) where FSP means Full-time-equivalent Software Personnel.

28 COCOMO Models COCOMO is defined in terms of three different models: –the Basic model, –the Intermediate model, and –the Detailed model. The more complex models account for more factors that influence software projects, and make more accurate estimates.

29 The Development mode the most important factors contributing to a project's duration and cost is the Development Mode Organic Mode: The project is developed in a familiar, stable environment, and the product is similar to previously developed products. The product is relatively small, and requires little innovation. Semidetached Mode: The project's characteristics are intermediate between Organic and Embedded.

30 The Development mode the most important factors contributing to a project's duration and cost is the Development Mode: Embedded Mode: The project is characterized by tight, inflexible constraints and interface requirements. An embedded mode project will require a great deal of innovation.

31 Modes

32 Cost Estimation Process Cost=SizeOfTheProject x Productivity

33 Cost Estimation Process Errors Effort Development Time Size Table Lines of Code Number of Use Case Function Point Estimation Process Number of Personnel

34 Project Size - Metrics 1.Number of functional requirements 2.Cumulative number of functional and non-functional requirements 3.Number of Customer Test Cases 4.Number of ‘typical sized’ use cases 5.Number of inquiries 6.Number of files accessed (external, internal, master) 7.Total number of components (subsystems, modules, procedures, routines, classes, methods) 8.Total number of interfaces 9.Number of System Integration Test Cases 10.Number of input and output parameters (summed over each interface) 11.Number of Designer Unit Test Cases 12.Number of decisions (if, case statements) summed over each routine or method 13.Lines of Code, summed over each routine or method

35 Example

36 Example (.)

37 Example (..) Technical Complexity Factors: –1.Data Communication3 –2.Distributed Data Processing0 –3.Performance Criteria4 –4.Heavily Utilized Hardware0 –5.High Transaction Rates3 –6.Online Data Entry3 –7.Online Updating3 –8.End-user Efficiency3 –9.Complex Computations0 –10.Reusability3 –11.Ease of Installation3 –12.Ease of Operation5 –13.Portability3 –14.Maintainability3 »DI =30(Degree of Influence)

38 Example (…) Function Points –FP=UFP*( *DI)= 55*( *30)=52.25 –That means the is FP=52.25

39 Relation between LOC and FP Relationship: –LOC = Language Factor * FP –where LOC (Lines of Code) FP (Function Points)

40 Relation between LOC and FP(.) Assuming LOC’s per FP for: Java = 53, C++ = 64 aKLOC = FP * LOC_per_FP / 1000 It means for the SpellChekcer Example: (Java) LOC=52.25*53= LOC or 2.76 KLOC

41 Effort Computation The Basic COCOMO model computes effort as a function of program size. The Basic COCOMO equation is: –E = aKLOC^b Effort for three modes of Basic COCOMO. Mode a b Organic Semi- detached Embedded

42 Example

43 Effort Computation The intermediate COCOMO model computes effort as a function of program size and a set of cost drivers. The Intermediate COCOMO equation is: –E = aKLOC^b*EAF Effort for three modes of intermediate COCOMO. Mode a b Organic Semi- detached Embedded

44 Effort computation(.) Effort Adjustment Factor Cost Driver Very Low NominalHighVery High Extra High Required Reliability Database Size Product Complexity Execution Time Constraint Main Storage Constraint Virtual Machine Volatility Comp Turn Around Time Analyst Capability Application Experience Programmers Capability Virtual machine Experience Language Experience Modern Prog Practices SW Tools Required Dev Schedule ,10

45 Effort Computation (..) Total EAF = Product of the selected factors Adjusted value of Effort: Adjusted Person Months: APM = (Total EAF) * PM

46 Example

47 Software Development Time Development Time Equation Parameter Table: Development Time, TDEV = C * (APM **D) Number of Personnel, NP = APM / TDEV ParameterOrganic Semi- detached Embedded C2.5 D

48 Distribution of Effort A development process typically consists of the following stages: - Requirements Analysis - Design (High Level + Detailed) - Implementation & Coding - Testing (Unit + Integration)

49 Distribution of Effort (.) The following table gives the recommended percentage distribution of Effort (APM) and TDEV for these stages: Percentage Distribution of Effort and Time Table: Req Analysi s Design, HLD + DD Implementatio n Testing Effor t 23%29%22%21%100 % TDE V 39%25%15%21%100 %

50 Error Estimation Calculate the estimated number of errors in your design, i.e.total errors found in requirements, specifications, code, user manuals, and bad fixes: –Adjust the Function Point calculated in step1 AFP = FP ** 1.25 –Use the following table for calculating error estimates Error TypeError / AFP Requirements1 Design1.25 Implementation1.75 Documentation0.6 Due to Bug Fixes0.4

51 All Together Design Unadjusted Function Point (UFP table) Modify FP=UFP*TCF ClassesClasses*(2Function Points) TCF KLOC=Max[aKLOC, bKLOC] LOC=13.20*Num of Method LOC=18.25*Num of Method AFP=FP*1.25 Compute Errors = AFP*Y Compute Effort: Person Month, PM=A*(KLOC**B) bKLOC=∑ (LOCs for all Classes)/1000 Adjusted PM: APM=(total EAF)*PM Development Time: TDEV=C*(APM**D) Factor:1-15 Number of personnel: NP=APM/TDEV DI=∑ratings of selected factors 14 TCF= *∑(DI) j 1 Min[TCF]=0.65; Max[TCF]=1.35 aKLOC=FP*LOC_per_FP/1000 Java=53; C++=64 EAF=Product of selected factor NPEfforttime ReqAPMTDEV Result

52 COCOMO Models

53 Defect Removal Efficiency Defect removal efficiency provides benefits at both the project and process level It is a measure of the filtering ability of QA activities as they are applied throughout all process framework activities –It indicates the percentage of software errors found before software release It is defined as DRE = E / (E + D) –E is the number of errors found before delivery of the software to the end user –D is the number of defects found after delivery As D increases, DRE decreases (i.e., becomes a smaller and smaller fraction) The ideal value of DRE is 1, which means no defects are found after delivery DRE encourages a software team to institute techniques for finding as many errors as possible before delivery