-- Finishing Design Unit -- Intro to Glass Box Testing ECEN5543 / CSCI 5548 SW Eng of Standalone Programs University of Colorado, Boulder

Many Examples Pure Fabrication – an idea to group together some common behavior Many OO design patterns are examples Adapter Strategy Command... Traditional cohesion – top two strongest informational functional

Introduction to Glass Box Testing Testing Software Engineering of Standalone Programs

What is Glass Box Testing? Sometimes called Structure Tests Exercise the internal logic of a program and traverse particular execution paths (Functional, performance, and stress testing are collectively referred to as black box testing.)

What are major activities? Decide which paths need to be exercised. Derive test data to exercise those paths. Determine the test coverage criterion. Execute the test cases. Measure the test coverage achieved.

Why determine test coverage criterion? Why not just test it all? What does “all” mean? Program units (classes, clusters of classes) usually contain too many paths to permit exhaustive testing. For example, loops introduce combinatorial numbers of execution paths and make exhaustive testing impossible.

What if you could test all paths? Would the program be correct then? The program might be missing paths Code for branch alternative not there More likely detect missing paths how? Might have computational errors not revealed in the test data shown, even on the path where the error occurs We call that coincidental correctness For example, suppose code has x = A + A where it should have A*A but you test with A = 2.

When are you done with unit testing? Decide on criterion Statement coverage Branch coverage Logical path coverage

Statement Coverage Find a set of test cases to ensure that every statement is executed at least once

Branch coverage Find a set of test cases to ensure that each branching statement is taken in each direction at least once How is that different from statement coverage?

Logical Path Coverage Acknowledges that the order in which branches are taken can be significant Ensure that every path is executed at least once A path is some feasible combination of statements and branches Not in your life time...

Branch Testing -- May not accomplish what you think Consider the following pseudocode: if (X > 3 and then Y = 2) then Z := 1 endif; if (X = 4 or else Z > 3) then Z := Z + 1; endif; What happens with test case x=5, y=2, z=3 and x=4, y=3, z=3?

Common rule of thumb re unit test completion 85 to 90 % of branch coverage that is, 85 to 90% of all branch alternatives have been traversed by the set of unit test cases. Strikes a balance between excessive number of test cases and leaving test completion to the intuition of each programmer According to studies, functional + performance + stress tests based on functional requirements and programmer’s intuition achieve....

Limitations May not be possible to reach 100% branch coverage because it may be hard to find input data that will exercise a nested branching statement in a particular way. Beware of trend plots # errors discovered per unit of time vs. time Expect buildup to maximum value and then trail off Problems with this?

Control Flow Diagrams Provide visual representation of flow of control IF While For “Do until” Case

Control flow examples - 1

Control flow examples - 2

Control flow examples - 3

Control flow examples - 4

Why look at control flow? Easy to spot complexity Easy to develop basis set for statement coverage

Complexity leads to faults What do you find makes code hard to understand? What does the control flow diagram look like for that? McCabe’s Cyclomatic Complexity What is being measured? Number of branches Number of closed regions What is NOT being measured? Beware: you can analyze YOUR error data against your modules and determine what characteristics are fault-prone in your application area.

Basis set for statement coverage testing Draw the flow graph Count the regions including the “outside” N is the number of linearly independent paths through the program control structure A linearly independent path includes one node of the control flow graph not contained in any other Create a basis set of linearly independent paths Prepare test cases to force execution of each path in the basis set. Execution of these test cases yields ______ coverage

Code sample /* Copyright (c) 1995 Sun Microsystems, Inc. All Rights Reserved. * Permission to use, copy, modify, and distribute this software and its documentation for * NON-COMMERCIAL purposes and without fee is hereby granted provided that this * copyright notice appears in all copies. Please refer to the file “copyright.html” for * further important copyright and licensing information.

* SUN MAKES NO REPRESENTATION OR WARRANTIES ABOUT THE * SUITABILITY OF THE SOFTWARE, EITHER EXPRESS OR IMPLIED, * INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON- * INFRINGEMENT. SUN SHALL NOT BE LIABLE FOR ANY DAMAGES * SUFFERED BY LINCENSEE AS A RESULT OF USING, MODIFYING OR * DISTRIBUTING THIS SOFTWARE OR ITS DERIVATIVES.

import java.io.*; import java.util.vector; class ListOfNumbers { private Vector victor; final int size = 10; public ListOfNumbers () { int i; victor = new Vector(size); for (i = 0; i < size; i++) victor.addElement(new Interger(i)); }

public void writeList() { PrintStream pStr = null; System.out.println(“Entering try statement”); int i; pStr = new PrintStream( new BufferedOutputStream( new FileOutputStream(“OutFile.txt”))); for (i = 0; i < size; i++) pStr.println(“Value at: “ + i + “ = “ + victor.elementAt(i)); pStr.close() }