CS 217 Software Verification and Validation Week 7, Summer 2014 Instructor: Dong Si

REVIEW OF BB-testing

Black-box Testing 1. Input Space Partitioning 2. Boundary Value Analysis 3

Example 1: compare two numbers – p50 of week3 n Function ‘Compare (x, y)’ n Inputs: Two numbers – x and y n Outputs: A larger number between x and y 4 Compare (x, y) = z (x, y) z

– p51 of week3 5 Equivalence Classes: { (x, y) | x < y } { (x, y) | x > y } { (x, y) | x = y } { input other than a pair of numbers, “as&%dfget^$(&w” } Valid inputs Invalid inputs

6 Valid (x, y) Input Space x = y x < y x > y Three test cases: (1, 2) (8, 8) (100, 30) Plus one test cases: (^&%*) --- ERROR – p52 of week3

Example 2: Loan application - p53 of week3 7 Customer Name Account number Loan amount requested Term of loan Monthly repayment Term: Repayment: Interest rate: Total paid back: 6 digits, 1st non-zero $500 to $ to 30 years Minimum $ chars. Choosing (or defining) partitions seems easy, but is easy to get wrong…

8 Customer name Number of characters: invalidvalidinvalid 1 Valid characters: Any other A-Z a-z -’ space

9 Loan amount invalidvalidinvalid 499

Black-box Testing 1. Input Space Partitioning 2. Boundary Value Analysis 10

Example 3 - Bonus n Consider a program IsAnOddNumber(x) which output “Yes” or “No” to the user to tell if the input integer x is a odd number or not. If you are planning to classify all possible user inputs into two classes - valid and invalid inputs 1) How do you partition the valid input space (the integer number space)? 2) Can you give an example of invalid test data? 11

Example 3 - Bonus n If your software manager asks the program MUST ONLY take input real number x as: -100<x<100 1) How do you partition all possible user inputs into two classes - valid and invalid inputs now? 2) If the software manager further asks you to do some Boundary Value Analysis, then what do you plan to do for the BVA analysis? 12

Example 4 - Bonus n Consider a program Absolute(x) which calculate the absolute value of a input real number x. If you are planning to classify all possible user inputs into two classes - valid and invalid inputs 1) How do you partition the valid input space (the real number space)? 2) Can you give an example of invalid test data? 13

Example 4 - Bonus n If your software manager asks the program MUST ONLY take input real number x within the range of: -321<x<-123, 123<x<321 1) How do you partition all possible user inputs into two classes - valid and invalid inputs now? 2) If the software manager further asks you to do some Boundary Value Analysis, then what do you plan to do for the BVA analysis? 14

Today’s topic WHITE-BOX TESTING

Styles of Testing n Testing traditionally can be conducted in two styles n Black-Box testing Try to choose "smart" tests based on the requirements, without looking at the code. n White-Box testing Try to choose "smart" tests based on the structure of the code, with minimal reference to the requirements. 16

Definition of White-Box Testing n Testing based on analysis of internal logic (design, code, etc.). (But expected results still come from requirements.) n Use different techniques to check every visible path of the source code to minimize errors. n It is the ability to know which line of the code is being executed and being able to identify what the correct output should be. 17

Characters of WB testing n Exploits the structure, control or data flow of programs in order to design test cases. n Typically performed during coding. n Allows to test parts of the program, since you know the structure. (with black ‐ box is not possible) n Allows to cover systematically every part of the program. 18

WB Testing Techniques n Logic coverage: (learned in previous classes)  Statement coverage  Branch coverage  Condition coverage  … n Dataflow based testing / Path coverage: all program paths have been traversed at least once 19

Pseudo code & Control/Dataflow Graphs 20 “nodes” “edges” Input output Absolute (x) { IF (x>=0) THEN y = x; ELSE IF (x<0) THEN y = -x; Output y; } x IF (x>=0)ELSE IF (x<0) y = x; y = -x; y

Can you draw a graph? 21 Absolute (x) { IF (x>0) THEN y = x; ELSE IF (x==0) THEN y = x; ELSE IF (x<0) THEN y = -x; Output y; }

Example 22 Fun (x, y) { z = 0; IF ((x>z) OR (y>z)) THEN z = 2; ELSE z = 3; Output z; } Test cases: 1.Fun (0, 0) 2.Fun (2, 0) 3.Fun (0, 2) 4.Fun (2, 2) 5.Fun (8, 9)

Control/Dataflow of the example 23 Fun (x, y) { z = 1; IF ((x>z) OR (y>z)) THEN z = 2; ELSE z = 3; Output z; } Input x, y z = 1; IF ((x>z) OR (y>z)) z = 2; ELSE z = 3; Output z; Test cases: 1.Fun (0, 0) 2.Fun (2, 0) 3.Fun (0, 2) 4.Fun (2, 2) 5.Fun (8, 9)

Statement coverage n Has each statement in the program been executed? 24 √ √ √

Branch coverage n Has each branch of each control structure been executed? n For example, given an if statement, have both the T and F branches been executed? n Another way of saying this is, has every edge in the Control/Dataflow graph been executed? 25

Condition coverage n Has each Boolean sub-expression evaluated both to true (T) and false (F) ? 26 Test cases: 1.Fun (0, 0) 2.Fun (2, 0) 3.Fun (0, 2) 4.Fun (2, 2) 5.Fun (8, 9) X>1 Y>1 T, F ? Fun (x, y) { z = 1; IF ((x>z) OR (y>z)) THEN z = 2; ELSE z = 3; Output z; }

Exercise n Consider a program ‘Compare’ which compare the value between two numbers. The inputs are two real numbers x and y, the program outputs the larger number between x and y. 27 Compare (x, y) { IF (x>y) THEN z = x; ELSE IF (x==y) THEN z = x; ELSE IF (x<y) THEN z = y; Output z; }

Exercise - Control/Data Flow Graphs n I have provided you the Pseudo code, n Can you draw a Control/Dataflow Graphs? n Try it! This is a bonus question with +5 points towards to FINAL!! 28

Exercise - Statement Coverage n Statement Coverage requires that each statement will have been executed at least once. What is the minimum number of test cases required to achieve statement coverage for the program Compare (x, y) ? 29

Exercise - Branch Coverage n Branch Coverage requires that each branch will have been traversed at least once. What is the minimum number of test cases required to achieve branch coverage for the program Compare (x, y) ? 30

Condition Coverage n Condition Coverage requires that each condition (sub-expression) will have been True at least once and False at least once. n What is the relationship between Branch and Condition Coverage?

Condition / branch coverage IF ( AND ) THEN … ELSE … 32 X>0 Y>0 T F T, F ?

Condition Coverage example – Combination ! IF A or B THEN s1; ELSE s2; ABBranch test 1TFtrue test 2FFfalse test 3TTtrue test 4FTtrue Combinations of condition values: TT, TF, FT, FF

Dataflow based testing -> Path coverage n All program paths have been traversed at least once. 34 Two paths ! Input x, y z = 1; IF ((x>z) OR (y>z)) z = 2; ELSE z = 3; Output z;

Path coverage - Example 1 35 if a then s1 else if b then s2 else if c then s3 else s4 end_if_then_else # Paths: ___ # Branches: ___

WB Testing Techniques n Logic coverage: (learned in previous class)  Statement coverage  Branch coverage  Condition coverage  … n Dataflow based testing / Path coverage: all program paths have been traversed at least once 36

Coming Up Next week… Lab 2: White-box Testing n Preparation:  Go over the slides of WB testing  Understand “Code coverage” & “Data flow / Path”