1. © 2000 Morgan Kaufman Overheads for Computers as Components Program design and analysis zProgram validation and testing.

2. © 2000 Morgan Kaufman Overheads for Computers as Components Goals zMake sure software works as intended. yWe will concentrate on functional testing--- performance testing is harder. zWhat tests are required to adequately test the program? yWhat is “adequate”?

3. © 2000 Morgan Kaufman Overheads for Computers as Components Basic testing procedure zProvide the program with inputs. zExecute the program. zCompare the outputs to expected results.

4. © 2000 Morgan Kaufman Overheads for Computers as Components Types of software testing zBlack-box: tests are generated without knowledge of program internals. zClear-box (white-box): tests are generated from the program structure.

5. © 2000 Morgan Kaufman Overheads for Computers as Components Clear-box testing zGenerate tests based on the structure of the program. yIs a given block of code executed when we think it should be executed? yDoes a variable receive the value we think it should get?

6. © 2000 Morgan Kaufman Overheads for Computers as Components Controllability and observability zControllability: must be able to cause a particular internal condition to occur. zObservability: must be able to see the effects of a state from the outside.

7. © 2000 Morgan Kaufman Overheads for Computers as Components Example: FIR filter zCode: for (firout = 0.0, j =0; j < N; j++) firout += buff[j] * c[j]; if (firout > 100.0) firout = 100.0; if (firout < -100.0) firout = -100.0; zControllability: to test range checks for firout, must first load circular buffer. zObservability: how do we observe values of buff, firout?

8. © 2000 Morgan Kaufman Overheads for Computers as Components Path-based testing zClear-box testing generally tests selected program paths: ycontrol program to exercise a path; yobserve program to determine if path was properly executed. zMay look at whether location on path was reached (control), whether variable on path was set (data).

9. © 2000 Morgan Kaufman Overheads for Computers as Components Points of view zSeveral ways to look at control coverage: ystatement coverage; ybasis sets; ycyclomatic complexity; ybranch testing; ydomain testing.

10. © 2000 Morgan Kaufman Overheads for Computers as Components Example: choosing paths zTwo possible criteria for selecting a set of paths: yExecute every statement at least once. yExecute every direction of a branch at least once.  Equivalent for structured programs, but not for programs with goto s.

11. © 2000 Morgan Kaufman Overheads for Computers as Components Path example Covers all statements +/+ Covers all branches

12. © 2000 Morgan Kaufman Overheads for Computers as Components Basis paths zHow many distinct paths are in a program? zAn undirected graph has a basis set of edges: ya linear combination of basis edges (xor together sets of edges) gives any possible subset of edges in the graph. zCDFG is directed, but basis set is an approximation.

13. © 2000 Morgan Kaufman Overheads for Computers as Components Basis set example ab c ed a b c d e a0 0 1 0 0 b0 0 1 0 1 c1 1 0 1 0 d0 0 1 0 1 e0 1 0 1 0 incidence matrix a0 0 1 0 0 b0 0 1 0 1 c1 1 0 1 0 d0 0 1 0 1 e0 1 0 1 0 basis set

14. © 2000 Morgan Kaufman Overheads for Computers as Components Cyclomatic complexity zProvides an upper bound on the control complexity of a program: ye = # edges in control graph; yn = # nodes in control graph; yp = # graph components. zCyclomatic complexity: yM = e - n + 2p. yStructured program: # binary decisions + 1.

15. © 2000 Morgan Kaufman Overheads for Computers as Components Branch testing strategy zExercise the elements of a conditional, not just one true and one false case. zDevise a test for every simple condition in a Boolean expression.

16. © 2000 Morgan Kaufman Overheads for Computers as Components Example: branch testing zMeant to write: if (a || (b >= c)) { printf(“OK\n”); } zActually wrote: if (a && (b >= c)) { printf(“OK\n”); } zBranch testing strategy: yOne test is a=F, (b >= c) = T: a=0, b=3, c=2. yProduces different answers.

17. © 2000 Morgan Kaufman Overheads for Computers as Components Another branch testing example zMeant to write: if ((x == good_pointer) && (x->field1 == 3))... zActually wrote: if ((x = good_pointer) && (x->field1 == 3))... zBranch testing strategy: yIf we use only field1 value to exercise branch, we may miss pointer problem.

18. © 2000 Morgan Kaufman Overheads for Computers as Components Domain testing zConcentrates on linear inequalities. zExample: j <= i + 1. zTest two cases on boundary, one outside boundary. correct incorrect

19. © 2000 Morgan Kaufman Overheads for Computers as Components Data flow testing zDef-use analysis: match variable definitions (assignments) and uses. zExample: x = 5; … if (x > 0)... zDoes assignment get to the use? def p-use

20. © 2000 Morgan Kaufman Overheads for Computers as Components Loop testing zCommon, specialized structure--- specialized tests can help. zUseful test cases: yskip loop entirely; yone iteration; ytwo iterations; ymid-range of iterations; yn-1, n, n+1 iterations.

21. © 2000 Morgan Kaufman Overheads for Computers as Components Black-box testing zBlack-box tests are made from the specifications, not the code. zBlack-box testing complements clear-box. yMay test unusual cases better.

22. © 2000 Morgan Kaufman Overheads for Computers as Components Types of black-box tests zSpecified inputs/outputs: yselect inputs from spec, determine requried outputs. zRandom: ygenerate random tests, determine appropriate output. zRegression: ytests used in previous versions of system.

23. © 2000 Morgan Kaufman Overheads for Computers as Components Evaluating tests zHow good are your tests? yKeep track of bugs found, compare to historical trends. zError injection: add bugs to copy of code, run tests on modified code.