Advanced Software Engineering Lecture 4: Process & Project Metrics

Today’s Topics l Measurements, measures, metrics & indicators l Size-oriented metrics (LOC) l Function-oriented metrics (FP) l Measuring quality

Measurement Measurement is: “A prerequisite for all science” (Lord Kelvin) A hallmark of all engineering disciplines The foundation for a quantitative, scientific approach to software

Why Do We Measure? l To characterize “How can we characterize the robustness of our system?” l To evaluate “How can we demonstrate we meet our robustness goals?”

Why Do We Measure? [2] l To predict “How will the software behave if the number of current transactions is doubled?” l To improve “What is the gap between current and desired performance?” [From Park, Goethart & Florac 1996]

Software Metrics Information from measurement can be used for: Continuous improvement of a process Estimation Quality control Productivity assessment

Reflective Practice “Those who ignore the past are doomed to repeat it...” Flaws in a product or process will be a source of continual productivity loss Good engineers rarely make the same mistake twice Measurement is a key enabler for CMM process evolution

Measurements, Measures, and Indicators l measurement = action taken “Count the avg. defects / 1K LOC.” l measure = a measurement result “1.2 defects / 1K LOC” l indicator = comparison (trend) “v1.1 has.5 fewer defects / 1K LOC”

Process Indicators l Assess the paradigm l Assess ongoing SE tasks l Assess work products l Evaluate milestones l Adapt / change life-cycle model (or individual process components) “From project to project, what works and what doesn’t?”

Project Indicators l Track task status l Track potential risk factors l Evaluate quality control efforts l Impact: Uncover problem areas early Adjust workflow / tasks / resources “How can I assess the health of my current project?”

Outcome Measures l Errors noted before delivery l Errors noted after delivery l Work products delivered l Human effort expended l Calendar time elapsed l Schedule conformance l Effort on “umbrella” activities

Private vs. Public Measurement l Per individual & per module l Sensitive data: internal to project “Where can we improve?” vs. “Who’s doing poorly?” l PSP - Personal Software Process [Humphrey ‘95] “Software improvement must begin at the individual level”

Statistical Software Process Improvement (SSPI) l Categorize errors by origin (specification, logic, etc.) l Estimate cost to correct l Sort according to frequency l Estimate cost of each error type l Find highest-cost problems l Prioritize debugging efforts

[From SEPA 5/e] Defects and Their Origin

Error Analysis l “Fishbone” diagram used to analyze the causes of defects l The analysis can be used to derive indicators for future improvements [Grady ‘92]

[From SEPA 5/e] Fishbone Diagram

Direct Measures l Cost l Effort applied l LOC (lines of code) l Execution speed l Memory size l Defects reported

Indirect Measures l Functionality l Quality l Complexity l Efficiency l Reliability l Maintainability

Size-Oriented Metrics l Errors per KLOC l Defects per KLOC l $ per KLOC l Documentation pages per KLOC l Errors per person-month l LOC per person-month l $ per documentation page

Size-Oriented Metrics [From SEPA 5/e]

Is LOC a Good Measure? Lines of code are easily counted, but… LOC not necessarily related to quality Programming languages differ widely in LOC per functional requirement Difficult to estimate LOC There is more to SE than writing code!

What LOC Can’t Measure... l People factors (team size, skill) l Problem factors (complexity, change) l Process factors (techniques, tools) l Product factors (reliability, performance) l Resource factors (people, tools)

Function-Oriented Metrics l The function points (FP) metric analyzes the information domain and SW complexity: Number of inputs & outputs Number of user queries / commands Number of files Number of external interfaces [Albrecht, 1979] “Measure functionality, not LOC”

Weighting Factors [From SEPA 5/e]

l Set of 14 questions, answered on a scale from 0 to 10; e.g.: “Does the system require reliable backup and recovery?” “Are there distributed processing functions?” (See full list, page 91, SEPA 5/e) Complexity Adjustment Values

Computing Function Points FP = WF x [ x CAV] Weighting Factor Count Total Complexity Adjustment Value Total Constants (Empirically Determined) An abstract, relative measure (not concrete or absolute!) PRO: Useful way to compare the estimated effort on two different systems, or for projects over time CON: Must be tuned for each organization, domain; e.g.: feature points [Jones, 1991]: algorithmic complexity 3D function points [Whitmire, 1995]: data, function, control

Measures with Function Points l Errors per FP l Defects per FP l $ per FP l Pages of documentation per FP l FP per person-month

Lines of Code Per Function Point [from SEPA 5/e]

Metrics for Software Quality l Software is only as good as the quality of... the requirements description the design of the solution the code / program produced the tests used to find errors l QA = life-cycle task, not just a “finishing” activity l QA must address process, too!

Measuring Quality l Correctness defects per KLOC? errors per inputs processed? l Maintainability mean time to change (MTTC)? (Note: changes to software vary widely in scope and complexity!) l Integrity assess threats, security of software

Measuring Quality (2) l Usability physical / intellectual skill required? slope of the learning curve? net increase in productivity? subjective assessment from users?

[From SEPA 5/e] Integrating Metrics in the Process

Summary l Measurements, measures, metrics & indicators l Size-oriented metrics (LOC) l Function-oriented metrics (FP) l Measuring quality l Individual methods must be adjusted and tuned for a given software domain!