LECTURE 15: Software Complexity Metrics

Slides:



Advertisements
Similar presentations
Chapter 14 Software Testing Techniques - Testing fundamentals - White-box testing - Black-box testing - Object-oriented testing methods (Source: Pressman,
Advertisements

Path Analysis Why path analysis for test case design?
1 Ivan Marsic Rutgers University LECTURE 15: Software Complexity Metrics.
CYCLOMATIC COMPLEXITY 1. Invented by Thomas McCabe (1974) to measure the complexity of a program ’ s conditional logic Cyclomatic complexity of graph.
SOFTWARE TESTING. INTRODUCTION  Software Testing is the process of executing a program or system with the intent of finding errors.  It involves any.
-- Finishing Design Unit -- Intro to Glass Box Testing ECEN5543 / CSCI 5548 SW Eng of Standalone Programs University of Colorado, Boulder.
CS447/ECE453/SE465 Prof. Alencar University of Waterloo 1 CS447/ECE453/SE465 Software Testing Tutorial Winter 2008 Based on the tutorials by Prof. Kontogiannis,
Chapter 8: Path Testing Csci 565.
IMSE Week 18 White Box or Structural Testing Reading:Sommerville (4th edition) ch 22 orPressman (4th edition) ch 16.
BASIS PATH TESTING ● By Tom McCabe ● McCabe, T., "A Software Complexity Measure," IEEE Trans. Software Engineering, vol. SE-2, December 1976, pp
Chapter 18 Testing Conventional Applications
Unit Testing CS 414 – Software Engineering I Don Bagert Rose-Hulman Institute of Technology January 16, 2003.
Path testing Path testing is a “design structural testing” in that it is based on detailed design & the source code of the program to be tested. The methodology.
1 Ivan Marsic Rutgers University LECTURE 16: Class Cohesion Metrics.
A Complexity Measure THOMAS J. McCABE Presented by Sarochapol Rattanasopinswat.
Cyclomatic Complexity Dan Fleck Fall 2009 Dan Fleck Fall 2009.
Software Systems Verification and Validation Laboratory Assignment 3
©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 1 Defect testing l Testing programs to establish the presence of system defects.
CS4311 Spring 2011 Unit Testing Dr. Guoqiang Hu Department of Computer Science UTEP.
Agenda Introduction Overview of White-box testing Basis path testing
Lec 06 Path Testing Part II - 1 CSCE 747 Fall 2013 CSCE 747 Software Testing and Quality Assurance Lecture 06 – Path Testing Part II 9/16/
White-box Testing.
1 Program Testing (Lecture 14) Prof. R. Mall Dept. of CSE, IIT, Kharagpur.
BASIS PATH TESTING.
Software Testing White Box Testing. Agenda What is White Box Testing Correctness Tests and Path Coverage Correctness Tests and Line Coverage McCabe Cyclomatic.
Chapter 8 Path Testing. Path testing Path testing is a “design structural testing” in that it is based on detailed design & the source code of the program.
White Box Testing Arun Lakhotia University of Southwestern Louisiana P.O. Box Lafayette, LA 70504, USA
1 Ivan Marsic Rutgers University LECTURE 15: Software Complexity Metrics.
Theory and Practice of Software Testing
SOFTWARE TESTING. Introduction Software Testing is the process of executing a program or system with the intent of finding errors. It involves any activity.
Agent program is the one part(class)of Othello program. How many test cases do you have to test? Reversi [Othello]
CMPSC 16 Problem Solving with Computers I Spring 2014 Instructor: Tevfik Bultan Lecture 4: Introduction to C: Control Flow.
White Box Testing by : Andika Bayu H.
Cyclomatic complexity (or conditional complexity) is a software metric (measurement). Its gives the number of indepented paths through strongly connected.
Cyclomatic Complexity Philippe CHARMAN Last update:
White-Box Testing Techniques I Prepared by Stephen M. Thebaut, Ph.D. University of Florida Software Testing and Verification Lecture 7.
CS223: Software Engineering Lecture 21: Unit Testing Metric.
Dynamic White-Box Testing What is code coverage? What are the different types of code coverage? How to derive test cases from control flows?
SOFTWARE TESTING LECTURE 9. OBSERVATIONS ABOUT TESTING “ Testing is the process of executing a program with the intention of finding errors. ” – Myers.
Software Test Metrics When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure,
TQS - Teste e Qualidade de Software (Software Testing and Quality) Test Case Design: White Box Testing João Pascoal Faria.
Software TestIng White box testing.
TQS - Teste e Qualidade de Software (Software Testing and Quality) Test Case Design: White Box Testing João Pascoal Faria.
BASIS PATH TESTING.
TQS - Teste e Qualidade de Software (Software Testing and Quality) Test Case Design: White Box Testing João Pascoal Faria.
Software Testing.
White-Box Testing Pfleeger, S. Software Engineering Theory and Practice 2nd Edition. Prentice Hall, Ghezzi, C. et al., Fundamentals of Software Engineering.
Software Testing.
Cyclomatic complexity
Software Engineering (CSI 321)
Structural testing, Path Testing
Types of Testing Visit to more Learning Resources.
White Box Testing.
Programming Fundamentals
JavaScript: Control Statements.
Chapter 9 Path Testing–Part 1
Cyclomatic Complexity
Software Testing (Lecture 11-a)
Cyclomatic Complexity
Chapter 14 Software Testing Techniques
Chapter 8 JavaScript: Control Statements, Part 2
Chapter 9 Structuring System Requirements: Logic Modeling
Cyclomatic Complexity
How does the complexity of the flow of control affect test cases?
White-Box Testing Techniques I
Control Structure Testing
Software Testing “If you can’t test it, you can’t design it”
Chapter 9 Structuring System Requirements: Logic Modeling
1. Cyclomatic complexity
Unit III – Chapter 3 Path Testing.
Presentation transcript:

LECTURE 15: Software Complexity Metrics Ivan Marsic Rutgers University

Topics Measuring Software Complexity Cyclomatic Complexity

Measuring Software Complexity Software complexity is difficult to operationalize complexity so that it can be measured Computational complexity measure big O (or big Oh), O(n) Measures software complexity from the machine’s viewpoint in terms of how the size of the input data affects an algorithm’s usage of computational resources (usually running time or memory) Complexity measure in software engineering should measure complexity from the viewpoint of human developers Computer time is cheap; human time is expensive

Desirable Properties of Complexity Metrics Monotonicity: adding responsibilities to a module cannot decrease its complexity If a responsibility is added to a module, the modified module will exhibit a complexity value that is the same as or higher than the complexity value of the original module Ordering (“representation condition” of measurement theory): Metric produces the same ordering of values as intuition would Cognitively more difficult should be measured as greater complexity Discriminative power (sensitivity): modifying responsibilities should change the complexity Discriminability is expected to increase as: 1) the number of distinct complexity values increases and 2) the number of classes with equal complexity values decreases Normalization: allows for easy comparison of the complexity of different classes

Cyclomatic Complexity Invented by Thomas McCabe (1974) to measure the complexity of a program’s conditional logic Counts the number of decisions in the program, under the assumption that decisions are difficult for people Makes assumptions about decision-counting rules and linear dependence of the total count to complexity Cyclomatic complexity of graph G equals #edges - #nodes + 2 V(G) = e – n + 2 Also corresponds to the number of linearly independent paths in a program (described later)

Converting Code to Graph FLOWCHART GRAPH if expression1 then statement2 else statement3 end if statement4 T expr1 ? F n1 For a strongly connected graph: Create a virtual edge to connect the END node to the BEGIN node (a) statm2 statm3 n2 n3 statm4 n4 switch expr1 case 1: statement2 case 2: statm3 case 3: statm4 end switch statm5 1 expr1 ? 3 n1 n2 n4 n5 n3 2 (b) statm2 statm3 statm4 statm5 n1 n2 n3 statm1 do statement1 while expr2 end do statement3 (c) T expr2 ? F statm3

Paths in Graphs (1) A graph is strongly connected if for any two nodes x, y there is a path from x to y and vice versa A path is represented as an n-element vector where n is the number of edges <, , …, > The i-th position in the vector is the number of occurrences of edge i in the path

Example Paths e1 e2 e3 e4 e5 e6 e7 e8 e9 e10 Paths: e10 e1 n1 if expression1 then statement2 end if do statement3 while expr4 end do if expression5 then statement6 statement7 P1 = e1, e2, e4, e6, e7, e8 P2 = e1, e2, e4, e5, e4, e6, e7, e8 P3 = e3, e4, e6, e7, e8, e10 P4 = e6, e7, e8, e10, e3, e4 P5 = e1, e2, e4, e6, e9, e10 P6 = e4, e5 P7 = e3, e4, e6, e9, e10 P8 = e1, e2, e4, e5, e4, e6, e9, e10 1, 1, 0, 1, 0, 1, 1, 1, 0, 0 1, 1, 0, 2, 1, 1, 1, 1, 0, 0 0, 0, 1, 1, 0, 1, 1, 1, 0, 1 1, 1, 0, 1, 0, 1, 0, 0, 1, 1 0, 0, 0, 1, 1, 0, 0, 0, 0, 0 0, 0, 1, 1, 0, 1, 0, 0, 1, 1 1, 1, 0, 2, 1, 1, 0, 0, 1, 1 n2 e3 e2 n3 e4 e5 n4 e6 e7 n5 n6 e9 e8 n7 Paths P3 and P4 are the same, but with different start and endpoints NOTE: A path does not need to start in node n1 and does not need to begin and end at the same node. E.g.,  Path P4 starts (and ends) at node n4  Path P1 starts at node n1 and ends at node n7

Paths in Graphs (2) A circuit is a path that begins and ends at the same node e.g., P3 = <e3, e4, e6, e7, e8, e10> begins and ends at node n1 P6 = <e4, e5> begins and ends at node n3 A cycle is a circuit with no node (other than the starting node) included more than once

Example Circuits & Cycles n1 if expression1 then statement2 end if do statement3 while expr4 end do if expression5 then statement6 statement7 P3 = e3, e4, e6, e7, e8, e10 P4 = e6, e7, e8, e10, e3, e4 P5 = e1, e2, e4, e6, e9, e10 P6 = e4, e5 P7 = e3, e4, e6, e9, 10 P8 = e1, e2, e4, e5, e4, e6, e9, e10 P9 = e3, e4, e5, e4, e6, e9, 10 0, 0, 1, 1, 0, 1, 1, 1, 0, 1 1, 1, 0, 1, 0, 1, 0, 0, 1, 1 0, 0, 0, 1, 1, 0, 0, 0, 0, 0 0, 0, 1, 1, 0, 1, 0, 0, 1, 1 1, 1, 0, 2, 1, 1, 0, 0, 1, 1 0, 0, 1, 2, 1, 1, 0, 0, 1, 1 n2 e3 e2 n3 e4 e5 n4 e6 e7 n5 n6 e9 e8 n7 This equation means to concatenate paths 2 and 3 together to form the path A, B, C, B, C, G, A, B, E, F, G and then remove the four nodes that appear in path 1, resulting in the required path 6. Cycles: P3 = e3, e4, e6, e7, e8, e10 P5 = e1, e2, e4, e6, e9, e10 P6 = e4, e5 P7 = e3, e4, e6, e9, 10 P4, P8, P9 are not cycles

Linearly Independent Paths A path p is said to be a linear combination of paths p1, …, pn if there are integers a1, …, an such that p = aipi (ai could be negative, zero, or positive) A set of paths in a strongly connected graph is linearly independent if no path in the set is a linear combination of any other paths in the set A linearly independent path is any path through the program (“complete path”) that introduces at least one new edge that is not included in any other linearly independent paths. A path that is subpath of another path is not considered to be a linearly independent path. A basis set of cycles is a maximal linearly independent set of cycles In a graph with e edges and n nodes, the basis has e  n + 1 cycles +1 is for the virtual edge, introduced to obtain a strongly connected graph Every path is a linear combination of basis cycles http://www.ironiacorp.com/apps/wiki/testing/Linearly_independent_path

Baseline method for finding the basis set of cycles Start at the source node (the first statement of the program/module) Follow the leftmost path until the sink node is reached Repeatedly retrace this path from the source node, but change decisions at every node with out-degree ≥2, starting with the decision node earliest in the path T.J. McCabe & A.H. Watson, Structured Testing: A Testing Methodology Using the Cyclomatic Complexity Metric, NIST Special Publication 500-235, 1996.

Linearly Independent Paths (1) Example paths: n1 e10 e1 if expression1 then statement2 end if do statement3 while expr4 end do if expression5 then statement6 statement7 P1 = e1, e2, e4, e6, e7, e8 P2 = e1, e2, e4, e5, e4, e6, e7, e8 P3 = e3, e4, e6, e7, e8, e10 P4 = e6, e7, e8, e10, e3, e4 P5 = e1, e2, e4, e6, e9, e10 P6 = e4, e5 P7 = e3, e4, e6, e9, 10 P8 = e1, e2, e4, e5, e4, e6, e9, e10 1, 1, 0, 1, 0, 1, 1, 1, 0, 0 1, 1, 0, 2, 1, 1, 1, 1, 0, 0 0, 0, 1, 1, 0, 1, 1, 1, 0, 1 1, 1, 0, 1, 0, 1, 0, 0, 1, 1 0, 0, 0, 1, 1, 0, 0, 0, 0, 0 0, 0, 1, 1, 0, 1, 0, 0, 1, 1 1, 1, 0, 2, 1, 1, 0, 0, 1, 1 n2 e3 e2 n3 e4 e5 n4 e6 e7 n5 n6 e9 e8 n7 V(G) = e – n + 2 = 9 – 7 + 2 = 4 This equation means to concatenate paths 2 and 3 together to form the path A, B, C, B, C, G, A, B, E, F, G and then remove the four nodes that appear in path 1, resulting in the required path 6. Or, if we count e10, then e – n + 1 = 10 – 7 + 1 = 4 EXAMPLE #1: P5 + P6 = P8 EXAMPLE #2: 2P3 – P5 + P6 = Cycles: P5 {1, 1, 0, 1, 0, 1, 0, 0, 1, 1} + P6 {0, 0, 0, 1, 1, 0, 0, 0, 0, 0} = P8 {1, 1, 0, 2, 1, 1, 0, 0, 1, 1} 2P3 { 0, 0, 2, 2, 0, 2, 2, 2, 0, 2} – P5 { 1, 1, 0, 1, 0, 1, 0, 0, 1, 1} ___ {-1,-1, 2, 1, 0, 1, 2, 2,-1, 1} + P6 { 0, 0, 0, 1, 1, 0, 0, 0, 0, 0} = P? {-1,-1, 2, 2, 1, 1, 2, 2,-1, 1} P3 = e3, e4, e6, e7, e8, e10 P5 = e1, e2, e4, e6, e9, e10 P6 = e4, e5 P7 = e3, e4, e6, e9, 10  Problem: The arithmetic doesn’t work for any paths — it works always only for linearly independent paths!

Linearly Independent Paths (2) (by enumeration) P1' = e1, e2, e4, e6, e7, e8, e10 P2' = e1, e2, e4, e5, e4, e6, e7, e8, e10 P3' = e3, e4, e6, e7, e8, e10 P4' = e1, e2, e4, e6, e9, e10 P1 = P2 = P3 = P4 = P5 = P6 = P7 = P8 = 1, 1, 0, 1, 0, 1, 1, 1, 0, 0 1, 1, 0, 2, 1, 1, 1, 1, 0, 0 0, 0, 1, 1, 0, 1, 1, 1, 0, 1 1, 1, 0, 1, 0, 1, 0, 0, 1, 1 0, 0, 0, 1, 1, 0, 0, 0, 0, 0 0, 0, 1, 1, 0, 1, 0, 0, 1, 1 1, 1, 0, 2, 1, 1, 0, 0, 1, 1 (P4 same as P3) V(G) = e – n + 2 = 9 – 7 + 2 = 4 EXAMPLE #3: P6 = P2' – P1' EXAMPLE #4: P7 = P3' + P4' – P1' EXAMPLE #5: P8 = P2' – P1' + P4' P2' {1, 1, 0, 2, 1, 1, 1, 1, 0, 0} – P1' {1, 1, 0, 1, 0, 1, 1, 1, 0, 0} = P6 {0, 0, 0, 1, 1, 0, 0, 0, 0, 0} P3' {0, 0, 1, 1, 0, 1, 1, 1, 0, 1} + P4' {0, 0, 1, 1, 0, 1, 1, 1, 0, 1} – P1' {1, 1, 0, 1, 0, 1, 1, 1, 0, 0} = P7 {0, 0, 1, 1, 0, 1, 0, 0, 1, 1} P2' {1, 1, 0, 2, 1, 1, 1, 1, 0, 0} – P1' {1, 1, 0, 1, 0, 1, 1, 1, 0, 0} + P4' {0, 0, 1, 1, 0, 1, 1, 1, 0, 1} = P8 {1, 1, 0, 2, 1, 1, 0, 0, 1, 1} Q: Note that P2' = P1' + P6, so why not use P1' and P6 instead of P2'? A: Because P6 is not a “complete path”, so it cannot be a linearly independent path

Unit Testing: Path Coverage Finds the number of distinct paths through the program to be traversed at least once Minimum number of tests necessary to cover all edges is equal to the number of independent paths through the control-flow graph (Recall the lecture on Unit Testing)

Issues (1) = CC = Single statement: Two (or more) statements: Cyclomatic complexity (CC) remains the same for a linear sequence of statements regardless of the sequence length —insensitive to complexity contributed by the multitude of statements (Recall that discriminative power (sensitivity) is a desirable property of a metric)

Issues (2) = CC = Optional action: Alternative choices: expr ? T expr ? F = CC = F Optional action versus alternative choices — the latter is psychologically more difficult

Issues (3) = CC = Simple condition: Compound condition: if (A OR B) then D; if (A) then D; T A ? T A || D ? = CC = D F F BUT, compound condition can be written as a nested IF: T A ? F D B ? if (A) then D; else if (B) then D; T D F

Issues (4) = CC = Switch/Case statement: N1 predicates: expr ? N T expr=1 ? F 2 = CC = statm1 T expr=2 ? F statm1 statm2 statmN statm2 T expr=N ? statmN Counting a switch statement: —as a single decision proposed by W. J. Hansen, “Measurement of program complexity by the pair (cyclomatic number, operator count),” SIGPLAN Notices, vol.13, no.3, pp.29-33, March 1978. —as log2(N) relationship proposed by V. Basili and R. Reiter, “Evaluating automatable measures for software development,” Proceedings of the IEEE Workshop on Quantitative Software Models for Reliability, Complexity and Cost, pp.107-116, October 1979.

Issues (5) = CC = Two sequential decisions: Two nested decisions: expr1 ? F T expr1 ? F = CC = T expr2 ? F T expr2 ? F But, it is known that people find nested decisions more difficult …

CC for Modular Programs (1) Adding a sequential node does not change CC: V = e – n + 2 = 12 – 11 + 2 = 3 n0 n0 n1 n3 n1 n3 n9' n2 n4 n5 n2 n4 n5 n9" n7 n6 n7 n6 Suppose that we decide to “modularize” the program and make the shaded region into a subroutine n8 n8

CC for Modular Programs (2) Intuitive expectation: Modularization should not increase complexity V = e – n + 2 = 12 – 11 + 2 = 3 V = e – n + 2p = 10 – 10 + 2 x 2 = 4 n0 n0 A: n1 n3 n1 n3 CALL A n2 n4 n5 n2 n9 n4 n5 n7 n6 n7 n6 n8 n8

Modified CC Measures Given p connected components of a graph: V(G) = e – n + 2p (1) VLI(G) = e – n + p + 1 (2) Eq. (2) is known as linearly-independent cyclomatic complexity VLI does not change when program is modularized into p modules

CC for Modular Programs (3) Intuitive expectation: Modularization should not increase complexity V = e – n + 2 = 12 – 11 + 2 = 3 V = e – n + 2p = 10 – 10 + 2 x 2 = 4 n0 n0 A: n1 n3 n1 n3 CALL A n2 n4 n5 n2 n9 n4 n5 n7 n6 n7 n6 n8 VLI = e – n + p + 1 = 12 – 11 + 1 + 1 = 3 n8 VLI = e – n + p + 1 = 10 – 10 + 2 + 1 = 3

Practical SW Quality Issues (1) No program module should exceed a cyclomatic complexity of 10 Originally suggested by McCabe P. Jorgensen, Software Testing: A Craftman’s Approach, 2nd Edition, CRC Press Inc., pp.137-156, 2002 . Software refactorings are aimed at reducing the complexity of a program’s conditional logic ♦ Refactoring: Improving the Design of Existing Code by Martin Fowler, et al.; Addison-Wesley Professional, 1999. ♦ Effective Java (2nd Edition) by Joshua Bloch; Addison-Wesley, 2008.

Practical SW Quality Issues (2) Cyclomatic complexity is a screening method, to check for potentially problematic code. As any screening method, it may turn false positives and false negatives Will learn about more screening methods (cohesion, coupling, …)