1. Problem Analysis and Program Design
IPC144
Session 5
Problem Analysis and Program Design
Continued 1

2. Synthesize a Test Plan from test data
Objectives
Design good test data
Synthesize a Test Plan from test data
Apply a rigorous approach to testing flowcharts using Walkthroughs 2

3. Test Data Test Data From the PDC:
PDC: Test the Algorithm for Correctness
One of the most important steps
Identifies major logic errors
Walkthroughs are a technique for testing
Test plans are used to document and plan the tests necessary to prove your algorithm performs as required 3

4. Test Data, continued Test Data, continued
There could exist an infinite number of possible values that a program can accept as input.
For example, write a program that accepts three numbers, adds them together and displays their total and average.
There are an infinite number of possible input values- does this mean that we must try all possible values to ensure that the program works correctly?
There must be a reasonable set of test data we can use to prove the algorithm works empirically. 4

5. Test Data, continued Test Data, continued
There are two general types of testing:
Black Box Testing
White Box testing 5

6. Test Data, continued Black box testing
The system, or program, is tested without knowing what is inside it
Tests are developed based solely on the requirements statement - based on the inputs, what are the expected outputs
Used to test the system or a multi-module program as a whole
Can be used to help clarify the requirements statement as you find test data that does not appear to be accounted for in the requirements 6

7. Test Data, continued White box testing
The algorithm is tested knowing what is inside it
It can only be developed by a programmer (typically the programmer that created the software)‏
Everything about the program is known- therefore everything is tested for
You must test every possible program instruction and every possible situation. 7

8. Test Data, continued White box testing, continued
Start identifying test data during the Design Phase
As you build your flowchart & pseudocode look for test cases, document them as you find them
Look for unusual conditions / situations to be added to your testing
As you find conditions that are not handled by your algorithm- enhance your algorithm and add the test to your Test Plan.
What is good test data for both Black Box Testing and White Box Testing? What do we look for? 8

9. Test Data, continued Test Data Black Box Look at requirements
Get test data and expected results from the user.
What if user ends program as their first action?
Unexpected values (user types words instead of a number)‏
White Box
Look at the Program and how it was written
Define data that will result in all branches of all decisions being tested
Ensure that ranges are tested for their minimum and maximum values
Look for ways that a divide by zero may happen
Simulate all possible error conditions (even if you have to alter your program to ensure they occur)‏ 9

10. Test Plans Test Plans Why all the trouble?
User groups and Audit departments look for them
Provides a concise description of WHAT was tested
Provides a concise description of HOW it was tested
Provides a concise description of the ENVIRONMENT it was tested in
A well design and documented test plan can break the 'finger pointing' 10

11. Test Plans, continued Test Plans, continued
Provides a concise description of WHAT was tested
At some time software comes under scrutiny:
The system has failed - was this scenario tested?
Somebody (maybe you) months later asks the question was ______ tested? 11

12. Test Plans, continued Test Plans, continued
Provides a concise description of HOW it was tested
What were the test cases?
What data was used?
Was certain sequence of events tested for?
Again, a system failure may precipitate the question: Was this sequence of events tested for? 12

13. Test Plans, continued Test Plans, continued
Provides a concise description of the ENVIRONMENT is was tested in
What other programs where part of the package?
What was the version of these other programs?
What hardware was it tested on?
What was the version of the operating system?
It "NEVER" happens that after you have certified a system somebody changes a component and doesn't tell you! 13

14. Test Plans, continued Test Plans, continued
An example of the parts of a Test Plan:
Cover Page
Environment setup
Black Box Testing
White Box Testing (can include “negative tests” to ensure the program does not crash or have undesired side-effects) 14

15. Test Plans, continued Test Plans - Cover Page
A brief description of what the program or system does.
Not too technical - just enough that somebody non-technical can understand what the program does
An extract of the requirements statement may form part of the cover page 15

16. Test Plans, continued Test Plans - Environmental Setup
Optional – depending on nature of your work environment
More technical description
What hardware is being used (as specific as necessary)‏
What version of the operating system is being used
What is the version of YOUR program that is being tested
The files that are used to makeup your program
What are the versions of the other programs that interface with yours
Are there versions of test data files? 16

17. Test Plans, continued Test Plans - Black Box Testing
These tests are to ensure that the program meets the requirements statement
Predominately positive testing is performed
Hopefully the Business Analyst has provided test cases that can be incorporated into the Test Plan
Concentrate on obtaining the correct outputs for the test inputs
Each test has two parts: the test, the expected results 17

18. Test Plans, continued Test Plans - White Box Testing
These tests are to ensure that the program handles all expected and unexpected data
Every path within the program must (at least in this course) be tested.
Can become very complex with the inter-relationships between variables
One philosophy behind White Box Testing is that "it would be nice that whenever a program prompts a user for a number, the users always enter a legitimate value that the program can handle"
May require modifications to the code to simulate error conditions (be sure to perform these modifications and tests on a COPY of the program)‏
Each test has two parts: the test, the expected results 18

19. Test Plans, continued Test Plans - Negative Testing
The intent is to maliciously attack your code - to see where (if) is breaks
Can be an easy task or frustrating task depending on how well constructed the code is
You will normally have built into your code a means to detect error conditions
The goal of your program is to shut down gracefully (return an error code or error message) as opposed to crashing or affecting other programs or systems 19

20. Test Plans, continued Example
Write a program to help a local Elementary School's library with the overdue fines on books. The librarians don't wish to always charge students for overdue books, however sometimes the number of books that are overdue is too much. When this happens the librarians enter into the program the student's name, the name of the overdue book they borrowed, and the date it was signed out (the librarian reads these fields from the sign-out card). The colour of the card is also entered, as this indicates what the normal sign-out period is for that book. White cards are due back in 14 days, Blue cards in 7 days, Green cards the next day, and Orange cards in 28 days. The colour code of the card is entered as a single character into the program (W, B, G and O). When this information has been entered, the program determines by how much the book is overdue (based on the date the program is run) and the overdue charge, ($0.05 per day). A list is produced that will show the student's name, the name of the book, the number of days the book is overdue and the fine that is payable. 20

21. Test Plans, continued Library Fines Test Plan
The program is a tool that will allow librarians to assess fines on overdue library books. The fine is calculated as $0.05 / day.
The fines are based on the colour code of the sign-out cards, where:
Orange Cards - 28 day sign-out period
White Cards - 14 day sign-out period
Blue Cards - 7 day sign-out period
Green Cards - 1 day sign-out period
The librarian is responsible for entering the student name, book name and the sign-out date. 21

22. Test Plans, continued Environmental Setup Program and Version:
library.c Version 1.0
Hardware:
IBM RS/6000
O/S:
AIX 4.3
Compiler: cc
Other files:
no other files are required 22

23. Test Plans, continued Black Box Testing
1) Enter 'Stu1', 'Book1', 'O', today's date - 30
Expected result: overdue = 2, fine = $0.10
2) Enter 'Stu2', 'Book2', 'W', today's date - 30
Expected result: overdue = 16, fine = $0.80
3) Enter 'Stu3', 'Book3', 'B', today's date - 30
Expected result: overdue = 23, fine = $1.15
4) Enter 'Stu4', 'Book4', 'G', today's date - 30
Expected result: overdue = 29, fine = $1.45
5) Enter 'Stu5', 'Book5', 'G', today's date
Expected result: overdue = 0, fine = $0.00 23

24. Test Plans, continued White Box Testing
1) Enter a colour code other than 'O', 'W', 'B', 'G'.
Expected Result: error message “Invalid card colour code entered”. Program to prompt for the colour code again.
2) Enter a future date (i.e. the book is not yet overdue)
Expected Result: error message “Invalid Future date entered”. Program to prompt for the date again. 24

25. Walkthroughs Walkthroughs
Walkthroughs are a means to test the algorithm BEFORE it has been coded
Walkthroughs are a tool to test that your algorithm behaves correctly
Because the computer is a mindless automaton - in order to check the program YOU must become the mindless automaton!
You 'play computer' - turn OFF your mind and follow the instructions EXACTLY as written
Remember to follow the instructions and perform what is asked for, not what you want to happen 25

26. Walkthroughs, continued
It is hard to keep track of the many variables in a program, so take a sheet a paper: List the variables, and their values. Keep track of the new values of a program as you execute the instructions
For output, reserve a space on your paper to track any information that is displayed to the user.
This can be very tough, as it is very easy at this stage to execute the instructions as you WANT them to be executed.
When we talk about pseudo code, walkthroughs can be used to test the INTEGRITY of the code after it has been translated from flowcharts to pseudo code. 26

27. Walkthroughs, continued
start
Walkthroughs, continued
num = 1
var = 0
var = num+ var
DISPLAY var
num = num + 2
num <= 7
TRUE
FALSE
stop 27

28. Walkthroughs, continued
Start
i > 0
i = 5
Stop
FALSE
TRUE
f = f * j
j = i
j > 1
j = j - 1
i = i - 1
f = 1
DISPLAY “Done”
DISPLAY f
Walkthroughs, continued 28