1. Program Design Invasion Percolation: Testing
Copyright © Software Carpentry 2010
This work is licensed under the Creative Commons Attribution License
See for more information.

2. Program Design Invasion Percolation: Testing
Copyright © Software Carpentry 2010
This work is licensed under the Creative Commons Attribution License
See for more information. 2

3. We found one bug 3

4. How many others haven't we found?
We found one bug
How many others haven't we found? 4

5. How many others haven't we found?
We found one bug
How many others haven't we found?
How do we validate and verify this program? 5

6. How many others haven't we found?
We found one bug
How many others haven't we found?
How do we verify and validate this program?
- Verification: is our program free of bugs? 6

7. How many others haven't we found?
We found one bug
How many others haven't we found?
How do we verify and validate this program?
- Verification: is our program free of bugs?
- Validation: are we using a good model? 7

8. How many others haven't we found?
We found one bug
How many others haven't we found?
How do we verify and validate this program?
- Verification: is our program free of bugs?
- Validation: are we using a good model?
The second question is a question for scientists... 8

9. How many others haven't we found?
We found one bug
How many others haven't we found?
How do we verify and validate this program?
- Verification: is our program free of bugs?
- Validation: are we using a good model?
The second question is a question for scientists...
...so we'll concentrate on testing our program 9

10. How many others haven't we found?
We found one bug
How many others haven't we found?
How do we verify and validate this program?
- Verification: is our program free of bugs?
- Validation: are we using a good model?
The second question is a question for scientists...
...so we'll concentrate on testing our program
making our program testable 10

11. This grid... 2 1 11

12. This grid... 2 1 12

13. ...should fill in like this
This grid... 2 1
...should fill in like this 2 -1 13

14. ...should fill in like this If it doesn't, it should be
This grid... 2 1
...should fill in like this
If it doesn't, it should be
easy to figure out why not 2 -1 14

15. Overall program structure
'''doc string'''
def fail(...): ...
def create_random_grid(N, Z): ...
def mark_filled(grid, x, y): ...
def is_candidate(grid, x, y): ...
def find_candidates(grid): ...
def fill_grid(grid): ...
if __name__ == '__main__':
...

16. Overall program structure
'''doc string'''
def fail(...): ...
def create_random_grid(N, Z): ...
def mark_filled(grid, x, y): ...
def is_candidate(grid, x, y): ...
def find_candidates(grid): ...
def fill_grid(grid): ...
if __name__ == '__main__':
...
This is what we
want to test

17. Overall program structure
'''doc string'''
def fail(...): ...
def create_random_grid(N, Z): ...
def mark_filled(grid, x, y): ...
def is_candidate(grid, x, y): ...
def find_candidates(grid): ...
def fill_grid(grid): ...
if __name__ == '__main__':
...
This is what we
want to test
Let's reorganize
the code so that
it's easy to create
specific grids

18. Old structure ... def create_random_grid(N, Z): ...
if __name__ == '__main__':
grid = create_random_grid(grid_size, value_range)

19. New structure ... def create_grid(N): ...
def fill_grid_random(grid, Z): ...
if __name__ == '__main__':
grid = create_grid(grid_size)
fill_grid_random(grid, value_range)

20. New structure Create an N×N grid of zeroes ... def create_grid(N): ...
def fill_grid_random(grid, Z): ...
if __name__ == '__main__':
grid = create_grid(grid_size)
fill_grid_random(grid, value_range)

21. New structure Create an N×N grid of zeroes Overwrite cells with random
values in 1..Z
...
def create_grid(N): ...
def fill_grid_random(grid, Z): ...
if __name__ == '__main__':
grid = create_grid(grid_size)
fill_grid_random(grid, value_range)

22. New structure Create an N×N grid of zeroes Overwrite cells with random
values in 1..Z
We'll call
something else
when testing
...
def create_grid(N): ...
def fill_grid_random(grid, Z): ...
if __name__ == '__main__':
grid = create_grid(grid_size)
fill_grid_random(grid, value_range)

23. Old structure ... if __name__ == '__main__':
# Get parameters from command line.
arguments = sys.argv[1:]
try:
grid_size = int(arguments[0])
value_range = int(arguments[1])
rand_seed = int(arguments[2])
except ...:

24. New structure ... if __name__ == '__main__':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[1:])

25. Newer structure ... if __name__ == '__main__': scenario = sys.argv[1]
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
if scenario == 'random':

26. Newer structure ... if __name__ == '__main__': scenario = sys.argv[1]
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
if scenario == 'random':
else:
fail('Unknown scenario %s' % scenario)

27. We aren't going to need random numbers when
we fill the grid manually for testing 27

28. We aren't going to need random numbers when
we fill the grid manually for testing
We're also not going to need the value range 28

29. We aren't going to need random numbers
in cases where we fill the grid manually for testing
We're also not going to need the value range
Or the grid size 29

30. We aren't going to need random numbers
in cases where we fill the grid manually for testing
We're also not going to need the value range
Or the grid size
So move argument handling and RNG seeding
into the random scenario 30

31. The revised structure ... if __name__ == '__main__':
scenario = sys.argv[1]
if scenario == 'random':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
random.seed(rand_seed)
else:
fail('Unknown scenario %s' % scenario)

32. The revised structure ... if __name__ == '__main__':
scenario = sys.argv[1]
if scenario == 'random':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
random.seed(rand_seed)
else:
fail('Unknown scenario %s' % scenario)

33. The revised structure ... if __name__ == '__main__':
scenario = sys.argv[1]
if scenario == 'random':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
random.seed(rand_seed)
else:
fail('Unknown scenario %s' % scenario)

34. The revised structure ... if __name__ == '__main__':
scenario = sys.argv[1]
if scenario == 'random':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
random.seed(rand_seed)
else:
fail('Unknown scenario %s' % scenario)

35. The revised structure ... if __name__ == '__main__':
scenario = sys.argv[1]
if scenario == 'random':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
random.seed(rand_seed)
else:
fail('Unknown scenario %s' % scenario)

36. A closer look if scenario == 'random':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
random.seed(rand_seed)
grid = create_grid(grid_size)
fill_grid_random(grid, value_range)
mark_filled(grid, grid_size/2, grid_size/2)
num_filled_cells = fill_grid(grid) + 1
print '%d cells filled' % num_filled_cells

37. A closer look if scenario == 'random':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
random.seed(rand_seed)
grid = create_grid(grid_size)
fill_grid_random(grid, value_range)
mark_filled(grid, grid_size/2, grid_size/2)
num_filled_cells = fill_grid(grid) + 1
print '%d cells filled' % num_filled_cells

38. A closer look if scenario == 'random':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
random.seed(rand_seed)
grid = create_grid(grid_size)
fill_grid_random(grid, value_range)
mark_filled(grid, grid_size/2, grid_size/2)
num_filled_cells = fill_grid(grid) + 1
print '%d cells filled' % num_filled_cells

39. A closer look if scenario == 'random':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
random.seed(rand_seed)
grid = create_grid(grid_size)
fill_grid_random(grid, value_range)
mark_filled(grid, grid_size/2, grid_size/2)
num_filled_cells = fill_grid(grid) + 1
print '%d cells filled' % num_filled_cells

40. A closer look if scenario == 'random':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
random.seed(rand_seed)
grid = create_grid(grid_size)
fill_grid_random(grid, value_range)
mark_filled(grid, grid_size/2, grid_size/2)
num_filled_cells = fill_grid(grid) + 1
print '%d cells filled' % num_filled_cells

41. A closer look if scenario == 'random':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
random.seed(rand_seed)
grid = create_grid(grid_size)
fill_grid_random(grid, value_range)
mark_filled(grid, grid_size/2, grid_size/2)
num_filled_cells = fill_grid(grid) + 1
print '%d cells filled' % num_filled_cells

42. A closer look if scenario == 'random':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
random.seed(rand_seed)
grid = create_grid(grid_size)
fill_grid_random(grid, value_range)
mark_filled(grid, grid_size/2, grid_size/2)
num_filled_cells = fill_grid(grid) + 1
print '%d cells filled' % num_filled_cells

43. A closer look if scenario == 'random':
grid_size, value_range, rand_seed = \
parse_arguments(sys.argv[2:])
random.seed(rand_seed)
grid = create_grid(grid_size)
fill_grid_random(grid, value_range)
mark_filled(grid, grid_size/2, grid_size/2)
num_filled_cells = fill_grid(grid) + 1
print '%d cells filled' % num_filled_cells

44. These function names are too similar if scenario == 'random':
...get arguments...
random.seed(rand_seed)
grid = create_grid(grid_size)
fill_grid_random(grid, value_range)
mark_filled(grid, grid_size/2, grid_size/2)
num_filled_cells = fill_grid(grid) + 1
print '%d cells filled' % num_filled_cells

45. So rename if scenario == 'random': ...get arguments...
random.seed(rand_seed)
grid = create_grid(grid_size)
init_grid_random(grid, value_range)
mark_filled(grid, grid_size/2, grid_size/2)
num_filled_cells = fill_grid(grid) + 1
print '%d cells filled' % num_filled_cells

46. And since we do this in all scenarios... if scenario == 'random':
...get arguments...
random.seed(rand_seed)
grid = create_grid(grid_size)
init_grid_random(grid, value_range)
mark_filled(grid, grid_size/2, grid_size/2)
num_filled_cells = fill_grid(grid) + 1
print '%d cells filled' % num_filled_cells
And since we do this
in all scenarios...

47. ...move it into fill_grid if scenario == 'random': ...get arguments...
random.seed(rand_seed)
grid = create_grid(grid_size)
init_grid_random(grid, value_range)
num_filled_cells = fill_grid(grid)
print '%d cells filled' % num_filled_cells
...move it into
fill_grid

48. Reminder of revised program structure
'''doc string'''
def fail(...): ...
def create_grid(N): ...
def init_grid_random(grid, Z): ...
def mark_filled(grid, x, y): ...
def is_candidate(grid, x, y): ...
def find_candidates(grid): ...
def fill_grid(grid): ...
def parse_arguments(arguments): ...
if __name__ == '__main__':
...

49. Created by splitting a function that was doing two things
'''doc string'''
def fail(...): ...
def create_grid(N): ...
def init_grid_random(grid, Z): ...
def mark_filled(grid, x, y): ...
def is_candidate(grid, x, y): ...
def find_candidates(grid): ...
def fill_grid(grid): ...
def parse_arguments(arguments): ...
if __name__ == '__main__':
...
Created by
splitting a function
that was doing
two things

50. Fills middle cell at the start, and returns count of all filled cells
'''doc string'''
def fail(...): ...
def create_grid(N): ...
def init_grid_random(grid, Z): ...
def mark_filled(grid, x, y): ...
def is_candidate(grid, x, y): ...
def find_candidates(grid): ...
def fill_grid(grid): ...
def parse_arguments(arguments): ...
if __name__ == '__main__':
...
Fills middle cell
at the start, and
returns count of
all filled cells

51. Handle arguments '''doc string''' def fail(...): ...
def create_grid(N): ...
def init_grid_random(grid, Z): ...
def mark_filled(grid, x, y): ...
def is_candidate(grid, x, y): ...
def find_candidates(grid): ...
def fill_grid(grid): ...
def parse_arguments(arguments): ...
if __name__ == '__main__':
...
Handle arguments

52. Handle arguments for random case '''doc string''' def fail(...): ...
def create_grid(N): ...
def init_grid_random(grid, Z): ...
def mark_filled(grid, x, y): ...
def is_candidate(grid, x, y): ...
def find_candidates(grid): ...
def fill_grid(grid): ...
def parse_arguments(arguments): ...
if __name__ == '__main__':
...
Handle arguments
for random case

53. Handle arguments for random case We should rename it to make that
'''doc string'''
def fail(...): ...
def create_grid(N): ...
def init_grid_random(grid, Z): ...
def mark_filled(grid, x, y): ...
def is_candidate(grid, x, y): ...
def find_candidates(grid): ...
def fill_grid(grid): ...
def parse_arguments(arguments): ...
if __name__ == '__main__':
...
Handle arguments
for random case
We should rename
it to make that
clear...

54. '''doc string'''
def fail(...): ...
def create_grid(N): ...
def init_grid_random(grid, Z): ...
def mark_filled(grid, x, y): ...
def is_candidate(grid, x, y): ...
def find_candidates(grid): ...
def fill_grid(grid): ...
def parse_arguments_random(arguments): ...
if __name__ == '__main__':
...

55. We set out to test our program...

56. We set out to test our program...
...but found we had to reorganize it first

57. We set out to test our program...
...but found we had to reorganize it first
refactor

58. We set out to test our program...
...but found we had to reorganize it first
refactor
Refactoring: changing a program's structure
without modifying its behavior or functionality
in order to improve its quality

59. There are whole books about how to do this
systematically

60. There are whole books about how to do this
systematically
The original

61. There are whole books about how to do this
systematically
The original
Mostly a catalog of refactorings for
object-oriented programs

62. There are whole books about how to do this
systematically
The original
Mostly a catalog of refactorings for
object-oriented programs
How to refactor legacy programs to
make them more testable

63. There are whole books about how to do this
systematically
The original
Mostly a catalog of refactorings for
object-oriented programs
How to refactor legacy programs to
make them more testable
Examples drawn from many languages

64. And now we can start testing our program64

65. Greg Wilson created by May 2010 Copyright © Software Carpentry 2010
This work is licensed under the Creative Commons Attribution License
See for more information.