Software Testing (Part 2)

White Box Testing White box testing (a.k.a. clear box testing, glass box testing, transparent box testing, or structural testing) uses an internal perspective of the system to design test cases based on internal structure. It requires programming skills to identify all paths through the software. The tester chooses test case inputs to exercise paths through the code and determines the appropriate outputs

White Box Testing Since the tests are based on the actual implementation, if the implementation changes, the tests will need to change. While white box testing is applicable at the unit, integration and system levels of the software testing process, it is typically applied to the unit. Normally tests paths are within a unit, but test paths can be between units during integration, and between subsystems during a system level test.

White Box Testing Typical white box test design techniques include: Control flow testing Data flow testing Branch testing Path testing

Black-Box Testing Black box testing takes an external perspective of the test object to derive test cases. These tests can be functional or non-functional, though usually functional. The test designer selects valid and invalid inputs and determines the correct output. There is no knowledge of the test object's internal structure.

Black-Box Testing This method of test design is applicable to all levels of software testing: unit, integration, functional testing, system and acceptance. The higher the level, and hence the bigger and more complex the box, the more one is forced to use black box testing to simplify. While this method can uncover unimplemented parts of the specification, one cannot be sure that all existent paths are tested.

Data Flow Testing Data flow testing is testing based on the flow of data through a program. Data-flow testing looks at the lifecycle of a particular piece of data (i.e. a variable) in an application. By looking for patterns of data usage, risky areas of code can be found and more test cases can be applied.

Data Flow Testing Data flows from where it is defined to where it is used. A definition of data or def is when a value is assigned to a variable.

Data Flow Testing Computation use or c-use is when the value of a variable is referenced. Predicate use or p-use is when a variable appears in the condition of a decision statement.

Data Flow Testing A c-use occurs on the right hand side of assignment statements, when printing variables, when passing a parameter into a function or procedure by value, etc. A p-use is assigned to both branches out of a decision statement.

Data Flow Testing Control Flow Graph Node Source Line A Read a, b, c B Type=”scalene” C if (a==b||b==c||a==c) D Type=”isosceles” E if (a===b && b==c) F Type=”equilateral” G If (a>=b+c ||b>=a+c||c>=a+b) H Type=”not a triangle” I if (a<=0||b<=0||c<=0) J Type=”bad inputs” K Print type Control Flow Graph

Data Flow Testing A definition free path or a def-free path is a path from the definition of a variable to a use of that variable (either c-use or p-use) that does not contain another definition of that variable.

Data Flow Testing Find the def-free paths for each variable in this graph.

Data Flow Testing Some data flow testing criteria: dcu – a def-free path exists from every definition to a c-use. dpu – a def-free path exists from every definition to a p-use. du – a def-free path exists from every definition to a possible use. all du paths – a def-free path exists from every definition to every possible use.

Data Flow Testing Which criteria does this graph specify?

Random Testing Advantages: Fast Eliminates the bias of the testers Random testing is accomplished by randomly selecting the test cases. Advantages: Fast Eliminates the bias of the testers Easier to make statistical inferences

Random Testing For the triangle problem: Use a pseudo- random number generator to generate three numbers within a given range. What would be some of the problems with testing this way?

Operational Profiles An operational profile is a specification of types and the probability those types will be encountered in normal operation. # Description Probability 1 Equilateral 15% 2 Isosceles – right 10% 3 Isosceles – acute 20% 4 Isosceles – obtuse 5% 5 Scalene – right 6 Scalene – acute 25% 7 Scalene – obtuse Random numbers could then be generated to match the probability of occurrence for each type.

Statistical Inference From Testing If random testing has been done by randomly selecting test cases according to an operational profile, then the behavior of the software during testing should be the same as its behavior in the operational environment.

Statistical Inference From Testing For example: If in 1000 test cases randomly selected there were three errors, then the error rate of the software could be inferred as .3% In fact, since the error rate is small, the number of errors can be inferred to have a Poisson distribution with mean 3 and variance 3.

Boundary Testing Often errors happen at boundaries between domains. Consider the statement if (X < 1) {blah blah} An on test for X is is value of X that makes the statement true An off test is a value of X that makes the statement false.

Boundary Testing The trick is to have test values in each region determined by the boundaries

Boundary Testing Boundary testing chooses values that lie on or near the boundary. Boundaries can lie in either the false domain or the true domain The boundary for X<1 is X=1 and this boundary lies in the false domain. The boundary for X<=1 is still X=1, but this boundary lies in the true domain.

Boundary Testing For the triangle example, the primitive conditions a>b+c and b>a+c and c>a+b determine a boundary of the intersection of three planes in 3-space. Show the boundaries for these inequalities graphically and choose appropriate values for boundary testing. http://www.livephysics.com/ptools/online-3d-function-grapher.php