1. The Taxidermy Studio of Alfred Giles
Scenario 1
The Taxidermy Studio of Alfred Giles
Amadori Courses: Delivering an Effective Test Process

2. Scenario Alfred Giles runs a taxidermy studio
The cost of stuffing a dead creature varies according a set of predefined rules
An application has been written to automatically calculate and store the price that is to be charged for each commission
Our task is to devise the set of tests which will confirm that the correct amount is calculated in all cases
Amadori Courses: Delivering an Effective Test Process

3. Key Questions for you to answer
How many tests will it take to fully test the application?
How would you check to ensure that important cases have not been omitted?
If there a smaller subset of testcases which provides reasonable coverage?
If so what is it?
How will you document your testcases?
How long will the work take?
Amadori Courses: Delivering an Effective Test Process

4. A Worked Solution
Step 1: examine each individual in isolation and then recombine
Makes the process less overwhelming
Makes it easier to spot if you miss anything out at any stage in the process
Step 2: test positive and negative tests separately
Step 3: establish a subset of tests which quickly provides high level of all areas
Step 4: establish a quick and efficient method for identifying and describing your tests
Amadori Courses: Delivering an Effective Test Process

5. Step 1:
What test scenarios are required to test the POSITIVE logic relating to
Size
Legs
Eyes
Coat
Habitat
In isolation ?
(We will worry about combining them later)
Amadori Courses: Delivering an Effective Test Process

6. Amadori Courses: Delivering an Effective Test Process
Positive Scenarios
What test scenarios are required to test the POSITIVE logic
Size 3 scenarios (Small/Medium/Large)
Legs 9 scenarios (0,2,4,6,8,10,12,14,110)
Eyes 9 scenarios (0,1,2,3,4,7,10,11,27)
Coat 3 scenarios (Fur/Scales/Feathers)
Habitat 3 scenarios (Water/Land/Trees)
Amadori Courses: Delivering an Effective Test Process

7. Unnatural Creatures
Creatures with odd number of legs (3 cases here 1, 17, 27)
Tree living creature with zero legs
Tree living creature with more than 10 legs
Feathered animal with zero legs
Feathered animal with more than 2 legs (3 cases here 3,8,27)
Tree living creature with zero eyes
Sea living creature with more than 4 eyes (5,13,23)
Any creature with more than 27 eyes (3 cases here 28,37,112)
Any of the above should lead to the order being rejected
Amadori Courses: Delivering an Effective Test Process

8. Positive and Negative Scenarios Combined
What test scenarios are required to POSITIVE test logic relating to
Size 3 scenarios (Small/Medium/Large)
Legs 14 scenarios (0,2,3,4,6,8,10,11,12,14,17,27,29,110)
Eyes 15 scenarios (0,1,2,3,4,5,7,10,11,13,23,27,28,37,112)
Coat 3 scenarios (Fur/Scales/Feathers)
Habitat 3 scenarios (Water/Land/Trees)
Amadori Courses: Delivering an Effective Test Process

9. Amadori Courses: Delivering an Effective Test Process
How many test cases?
Stage 1: work out how many testcases in total can be derived from your set of scenarios
3*14*14*15*3*3 = 5265 testcases
Stage 2: separate out your tests into positive and negative scenarios as you will test these separately
Negative scenarios: 3150
Positive scenarios: 2115
That’s still a lot of tests however…
Amadori Courses: Delivering an Effective Test Process

10. More Questions….
How are we going to script such a large number of tests?
How are we going to execute so many tests?
Do we really need to run so many tests?
If we don’t why are we writing tests we may not execute?
Amadori Courses: Delivering an Effective Test Process

11. How do we script so many tests?
Not as a big a job as it sounds if you adopt the correct approach…
Standard view of test scripting…
Every test must include
Objective
Detailed test steps
Expected results
This works well when each test is performed end to end one after another
Here one set of output can provide us with all of the output we need
Therefore no need to repeat the instructions 5265 times
Amadori Courses: Delivering an Effective Test Process

12. What should our tests look like?
Actually only require 3 elements
A unique ID
An explanation of what makes this test different from the other 5264
The expected result
We need a format which can provide us with this information with a minimum of effort
Amadori Courses: Delivering an Effective Test Process

13. Suggested Test Format
Each scenario is a unique combination of the following 5 factors
Number of eyes
Number of legs
Coat
Habitat
Size
So a typical test can merely state
“Request a quotation for stuffing a large furry land dwelling animal with 4 legs and 2 eyes”
And it should be possible for you to work out an expected result for each case by applying the same logic as the application has been asked to
Amadori Courses: Delivering an Effective Test Process

14. How do you build the tests quickly?
Let concatenation be your friend
Using either MS Excel or MS Access build a formula which applies the following to the underlying scenario parameters
“Request a quotation for stuffing a “&[SIZE]&” “&[COAT]&[HABITAT]” dwelling animal with “&[EYES]&”eyes and “&”[LEGS]&” legs.
Items in square brackets reference the underlying data in your database table or spreadsheet
You don’t need to add text or spaces but it does make the test easier for others to follow
You will end up with some cases which read “1 eyes” or “1 legs”
But this is a test script not a novel! You don’t get bonus points for grammatical correctness…..
Amadori Courses: Delivering an Effective Test Process

15. Examples
Test Case 1: “Request a quotation for stuffing a “&[b2]&” “&[c2]&[d2]” animal with “&[e2]&”eyes and “&”[f2]&” legs.
Test Case 4: “Request a quotation for stuffing a “&[b5]&” “&[c5]&[d5]” animal with “&[e5]&”eyes and “&”[f5]&” legs.
Amadori Courses: Delivering an Effective Test Process

16. Amadori Courses: Delivering an Effective Test Process
Example Test Output
ID 1. Positive Test. Request a quotation for stuffing a Fur covered Water dwelling animal with 0 eyes and 0 legs
ID 2. Positive Test. Request a quotation for stuffing a Scales covered Water dwelling animal with 0 eyes and 0 legs
ID 4. Positive Test. Request a quotation for stuffing a Fur covered Water dwelling animal with 0 eyes and 0 legs
ID 5. Positive Test. Request a quotation for stuffing a Scales covered Water dwelling animal with 0 eyes and 0 legs
ID 7. Positive Test. Request a quotation for stuffing a Fur covered Water dwelling animal with 0 eyes and 0 legs
ID 8. Positive Test. Request a quotation for stuffing a Scales covered Water dwelling animal with 0 eyes and 0 legs
ID 10. Positive Test. Request a quotation for stuffing a Fur covered Land dwelling animal with 0 eyes and 0 legs
ID 11. Positive Test. Request a quotation for stuffing a Scales covered Land dwelling animal with 0 eyes and 0 legs
ID 13. Positive Test. Request a quotation for stuffing a Fur covered Land dwelling animal with 0 eyes and 0 legs
ID 14. Positive Test. Request a quotation for stuffing a Scales covered Land dwelling animal with 0 eyes and 0 legs
ID 16. Positive Test. Request a quotation for stuffing a Fur covered Land dwelling animal with 0 eyes and 0 legs
ID 17. Positive Test. Request a quotation for stuffing a Scales covered Land dwelling animal with 0 eyes and 0 legs
ID 28. Positive Test. Request a quotation for stuffing a Fur covered Water dwelling animal with 0 eyes and 2 legs
ID 29. Positive Test. Request a quotation for stuffing a Scales covered Water dwelling animal with 0 eyes and 2 legs
ID 31. Positive Test. Request a quotation for stuffing a Fur covered Water dwelling animal with 0 eyes and 2 legs
ID 32. Positive Test. Request a quotation for stuffing a Scales covered Water dwelling animal with 0 eyes and 2 legs
ID 34. Positive Test. Request a quotation for stuffing a Fur covered Water dwelling animal with 0 eyes and 2 legs
ID 35. Positive Test. Request a quotation for stuffing a Scales covered Water dwelling animal with 0 eyes and 2 legs
ID 37. Positive Test. Request a quotation for stuffing a Fur covered Land dwelling animal with 0 eyes and 2 legs
ID 38. Positive Test. Request a quotation for stuffing a Scales covered Land dwelling animal with 0 eyes and 2 legs
ID 40. Positive Test. Request a quotation for stuffing a Fur covered Land dwelling animal with 0 eyes and 2 legs
ID 41. Positive Test. Request a quotation for stuffing a Scales covered Land dwelling animal with 0 eyes and 2 legs
ID 43. Positive Test. Request a quotation for stuffing a Fur covered Land dwelling animal with 0 eyes and 2 legs
ID 44. Positive Test. Request a quotation for stuffing a Scales covered Land dwelling animal with 0 eyes and 2 legs
ID 55. Positive Test. Request a quotation for stuffing a Fur covered Water dwelling animal with 0 eyes and 4 legs
Amadori Courses: Delivering an Effective Test Process

17. Why do we separate positive and negative tests?
Makes testing more efficient and easier to manage
Allows an iterative approach to test execution
Begin by testing a representative sample of positive tests
Increase the depth of positive testing if time and resources allow focussing in particular on areas where the initial tests identified errors
Also test a sample of negative tests ensuring that
You test each of different reasons for making a commission invalid
You include additional test in areas concentrating on areas where positive testing has identified issues
Amadori Courses: Delivering an Effective Test Process

18. Identifying a representative sample
Should include an example of each different value for each of the 5 parameters which drive the total cost
Should include additional tests of eye and leg logic as this is the most complex
A random sample of additional tests can also add a lot of value here
Prevents self selection of tests by the tester
Can focus on areas which might not have been selected otherwise
Identifying a representative sample
Amadori Courses: Delivering an Effective Test Process

19. Selecting a random sample
If you have given each test a unique id the Rand() function in Excel can be very useful here
Decide how many cases you want in your sample
(a sample size of 1% is probably a good place to start)
Perform the following formula once for every case you want in your sample (for if you want 50 cases run the formula 50 times)
=INT(RAND()*[X])
Where X is the total number of testcases in this set
This gives each of your test scenarios an equal chance of being selected
If the same number comes up more than once in your sample, accept it and just run 1 less test
Amadori Courses: Delivering an Effective Test Process

20. Efficient Test Execution
The use of samples and the adoption of an iterative approach helps ensure that you can make the best use of whatever time and resource is available
But is there a way all of the tests you have written in less than 2 hours?
Efficient Test Execution
Amadori Courses: Delivering an Effective Test Process

21. How to quickly execute all of your tests
Preparation is the key here
To be successful you need
A means of bulk loading test data into the application
A pre-prepared set of expected for every test scenario you upload
An efficient method of matching system output against the correct expected result
ideally you need a way of mapping the test id within the test data you upload (even if only within a comments field) so that the mapping of actual and expected results can be automated
Amadori Courses: Delivering an Effective Test Process

22. The Benefits of Such an Approach
This approach requires some work up front but means that the process of test execution becomes simple and quick and can easily be repeated again and gain
Quite possible that you could
Upload test data
Export actual results
Compare actual and expected results AUTOMATICALLY using a comparision spreadsheet/database
Highlight errors and automatically produce execution metrics
All within 2 hours
5265 tests in just 2 hours…….
Amadori Courses: Delivering an Effective Test Process

23. Conclusion This session has shown how to
Break down and analyse complex logic
Confirm that you have full test coverage of the requirements
Show some techniques to allow you to
Write large numbers of similar tests very quickly
Execute large numbers of tests very quickly
Neither would be possible if
Every test had to be written using a standard test template
If we hadn’t come up with an approach based on the particular requirements of this task
Conclusion
Amadori Courses: Delivering an Effective Test Process