1. 17/03/2016"How not to performance test"1 How not to Performance Test Gordon McKeown Alan Gordon

2. 17/03/2016 "How not to performance test" 2 Introduction  Who are we? And questions for you  Six ways to guarantee failure  30 minutes from us, then discussion  Forum not lecture – interrupt at will!  Last 5 minutes – future forum topics

3. 17/03/2016 "How not to performance test" 3 Encourage unrealistic expectations

4. 17/03/2016 "How not to performance test" 4 Great Expectations  Performance testing is a challenging discipline.  The difficulties are routinely underestimated by management.  Expect the unexpected – there will be issues.  Performance testing is where the technical chickens come home to roost.  Realism & good working relationships are crucial.

5. 17/03/2016 "How not to performance test" 5 Get team selection wrong

6. 17/03/2016 "How not to performance test" 6 The “wrong” team  Temporary transfer from the functional test team.  Limited technical knowledge.  Lack of experience.  No technical skills?  Lack of communication skills.  People who are just talent challenged...

7. 17/03/2016 "How not to performance test" 7 Create an environmental crisis!

8. 17/03/2016 "How not to performance test" 8 Some environmental issues  Test lab? Pros and Cons  Live versus test configuration dilemma.  Security  Test tools share many characteristics with hacker tools!  External system management is very inefficient.

9. 17/03/2016 "How not to performance test" 9 Don’t test on production hardware

10. 17/03/2016 "How not to performance test" 10 Testing on Live Configuration  Not always possible  Adds risk if not done  Suggested approach:  Test as early as possible on any environment  Use test lab approach if possible  Once system is optimised, repeat the test against the production hardware

11. 17/03/2016 "How not to performance test" 11 Ignore test data issues

12. 17/03/2016 "How not to performance test" 12 Test data pitfalls  Not enough.  Test user accounts.  Under populated database.  Naive use of live data.  The “data shape problem”.  Legal issues: should the data be “anonymised”?  Embed test data within (anonymised) live data?  Restore, test cycle.

13. 17/03/2016 "How not to performance test" 13 Only test for expected situations

14. 17/03/2016 "How not to performance test" 14 Test the Unexpected  Not enough to test “normal” and “peak”.  Test “peak of peaks” and spikes  Test situations that have never occurred before  Unusual world events – market crash?  Combination of unusual events  Load balancing situation – technical issues

15. 17/03/2016 "How not to performance test" 15 We say… Don’t do it like this! 1.Encourage unrealistic expectations 2.Get team selection wrong 3.Neglect your environments 4.Ignore test data issues 5.Only test for expected situations 6.… This list is not complete… What are YOUR do’s and don’ts?

16. 17/03/2016 "How not to performance test" 16 Future Topics What would you like to discuss at future forums?