How does the complexity of the flow of control affect test cases? Testing and Flow How does the complexity of the flow of control affect test cases? This should follow the Flow Graph presentation Copyright © 2001- 2017 Curt Hill

Introduction This considers again white box testing Since the focus is on testing flow of control we have to digress into several topics Flow Charts Flow Graphs Graph theory We are going to have some fun! Copyright © 2001- 2017 Curt Hill

Cyclomatic Complexity AKA McCabe essential complexity A metric that measures flow complexity Gives an upper bound on independent paths in a set the covers the code The formula for Cyclomatic Complexity is defined as: M = E – N + 2P where E is number of edges (arrows) N is number of nodes P is number of connected components Copyright © 2001- 2017 Curt Hill

Connectivity If the item in question is a function or a program, then it is connected and P=1 Formula become M = E – N + 2 If the item in question is a class, then all of its methods may be disconnected with P>1 More likely some methods will call others We will see some disconnected components and some weakly connected components Copyright © 2001- 2017 Curt Hill

Notes This metric only works on structured programs Not usually a problem today Besides the usual such as if and while, do not forget the stealth flow statements: Return Break Continue These have no predicates Compound conditions with short circuit evaluation may add others Copyright © 2001- 2017 Curt Hill

Consider this code int f1, f2, total=0, counter=1; while(!infile){ cin >> f1 >> f2 if(f1==0){ total+=f1; counter++; } else if(f2 == 0){ cout << total<<counter; counter = 0; } else total -= f2; cout << f1 << f2; } // end while cout << counter; Copyright © 2001- 2017 Curt Hill

Flow Graph N = 9 E = 11 9 1 P = 1 M = E – N + 2P M = 11 – 9 + 2 M = 4 3 Independent Paths 1, 9 1, 2, 3, 8, 1, 9 1, 2, 4, 5, 7, 8, 1, 9 1, 2, 4, 6, 7, 8, 1, 9 The first could be left out, it is covered by the others 5 6 From Pressman 7 8 Copyright © 2001- 2017 Curt Hill

Basis Path Testing The Cyclomatic Complexity number gives an upper bound of linearly independent paths Notice that we could have left the first path out Basis Path Testing uses this upper bound to find a set of test cases that covers every node and every edge Copyright © 2001- 2017 Curt Hill

First Alternative Calculation It is possible to count regions in the graph to compute Cyclomatic Complexity as well A region is any enclosed by edges This includes the surrounding region as well Copyright © 2001- 2017 Curt Hill

Region Counting 11 1 Four regions, Cylomatic Complexity must be four B 2,3 B A 6 4,5 7 D 8 C From Pressman 9 10 Copyright © 2001- 2017 Curt Hill

Second Alternative Calculation Predicate nodes plus one M = π + 1 A predicate node is any node with outdegree greater than 1 A compound condition, such as (X || Y) adds an additional predicate node If X is true, we do not evaluate Y Similarly with False AND Y With a switch the case is the predicate Copyright © 2001- 2017 Curt Hill

Predicate Nodes 11 1 Three predicate nodes, Cylomatic Complexity must be four 2,3 6 4,5 7 8 From Pressman 9 10 Copyright © 2001- 2017 Curt Hill

Commentary The counting of predicate node is particularly easy for programs to compute Scan the source and look only for flow constructs No need to construct the flow graph Copyright © 2001- 2017 Curt Hill

Compound Conditions The process dictates that we consider a compound predicate as two: (x>y || x == 5) At the machine level this makes sense The real problem is if there are function calls that may or may not be executed (x>z(y+1) && boolfun(y)) Copyright © 2001- 2017 Curt Hill

Non-Structured Constructs A function or program with multiple exits violates the structured programming tenets Although it is not always a bad thing We may adjust the formula for multiple exits, s M = π – S + 2 Copyright © 2001- 2017 Curt Hill

Generating Test Cases Simple, but not necessarily easy, procedure Create a flow graph Determine the Cyclomatic Complexity Determine the basis set of paths Prepare one test case for each path Copyright © 2001- 2017 Curt Hill

Black ≠ White Black box testing and white box testing are not the same Duh! The upper bound for the number of test cases might not satisfy the demands of black box testing Or it might White box testing cannot detect a left out case Copyright © 2001- 2017 Curt Hill

Development We usually connect Cyclomatic Complexity with testing We may also use it as an aid to development Routines which have high Cyclomatic Complexity measures are candidates for refactoring There may be a group policy mandating dividing complicated routines Copyright © 2001- 2017 Curt Hill

Paths This graph has M = 3 Two paths provide a basis path set 1, 2, 4, 5, 7 1, 3, 4, 6, 7 Yet if 3 and 5 have some kind of interaction, this may not be enough There are four distinct paths through this 1 3 2 4 6 5 7 Copyright © 2001- 2017 Curt Hill

Cyclomatic number again We know that the Cyclomatic Complexity meaure provides an upper bound for every statement testing It also provides a lower bound for every path testing Thus every stmt ≤ cyclomatic ≤ every path Copyright © 2001- 2017 Curt Hill

Problems Not everyone is happy with this metric Some claim that it is no better than LOC for any prediction Almost everyone agrees that if two codes with the same function have approximately the same LOC and much different Cyclomatic Complexity (M), then the smaller M code is preferred In general we expect M and LOC to correlate Copyright © 2001- 2017 Curt Hill

False Complexity What we expect is that higher value will occur with higher complexity This implies lower readability and higher costs to maintain Switches tend to be easy to read but add one for each case Thus larger values even though readability/maintainability does not suffer Copyright © 2001- 2017 Curt Hill

Nesting We generally agree that nesting is less desirable with respect to complexity Two loops nested is harder to understand than if they are sequentially encountered Yet they both have the same Cyclomatic Complexity Copyright © 2001- 2017 Curt Hill

Automatic Metrics There is some disagreement between automated measures These are often in the area of whether to increase the number for a compound condition or to treat compound conditions with side effects differently than those without Moral: only use one automated measure and then determine what is the threshold of a too complex Copyright © 2001- 2017 Curt Hill

Results The conclusion is that this and every other metric is imperfect We take its value as advice, but not as law We still believe that a lower metric is better But we also believe that the opinion of a real developer is better still Copyright © 2001- 2017 Curt Hill

An Exercise On the next two screens is a function and then a method We will examine these We will: Construct a flow graph Count regions Count predicates Compute Cyclomatic Complexity in several ways Copyright © 2001- 2017 Curt Hill

Insertion Sort void insertion(int ar[], unsigned int max) { unsigned int cur; int temp,i; for(cur=0;cur<max;cur++) { temp = ar[cur]; // cur is now free for(i=cur-1;i>=0 && ar[i] > temp;i--) ar[i+1]=ar[i]; ar[i+1] = temp; } return; } // end of insertion M = E – N + 2P M = π + 1 Regions Predicates Copyright © 2001- 2017 Curt Hill

Cache::IsPresent bool Cache::IsPresent(int k, char * & vp){ int i; reads++; clock++; vp = NULL; for(i=0;i<size;i++) { if(k == item[i].key) { if(item[i].ptr == NULL) return false; if(lru) item[i].value = clock; else item[i].value++; vp = item[i].ptr; hits++; return true; } // end of if } // end of for } M = E – N + 2P M = π + 1 Regions Predicates Copyright © 2001- 2017 Curt Hill

Finally Ready to try this yourself? Let’s do a couple of assignments to make this more familiar Copyright © 2001- 2017 Curt Hill