1. Nov 20, Fall 2006IAT 4101 Play Testing Software Testing Play Testing Team Structures

2. Nov 20, Fall 2006IAT 4102 Software Testing  Goal: “Prove” that the software works –Matches the functional specification –Matches the performance specification  Real Proof: NP-Hard problem –Simulate all the possible runs –Exponential growth in testing time –Each branch point multiplies possible program state by 2

3. Nov 20, Fall 2006IAT 4103 Testing Method  Run the program using a suite of data –Data suite exercises as much of the program’s code as possible –Each data set is accompanied by expected results –Where the results are unexpected, there is a bug!

4. Nov 20, Fall 2006IAT 4104 Generating Test Suites  Black Box Testing –Generate test data based purely on interpretation of the specifications –Don’t look at the code! –Code is a Black Box

5. Nov 20, Fall 2006IAT 4105 Generating Test Suites  White Box Testing –Use the code to generate tests with the purpose of exercising all blocks of code  Central concept: Coverage  Cover all of the parts of the programs code –Ensure that all lines of code run at least once

6. Nov 20, Fall 2006IAT 4106 Combine Both Approaches  Black Box + White Box test suites –Complementary roles –Hard to tell from the data what approach was used –Aims to generate that ever-elusive “proof”  Test suites: –Great for testing Compilers –Perhaps not so great for interaction and games

7. Nov 20, Fall 2006IAT 4107 Random Testing  Some applications are hard to test:  Random test –Give up the idea of formal coverage –Hope for stochastic coverage!  Sometimes this depends on actual human interaction –Need to get lots of people –Beta testing by customers

8. Nov 20, Fall 2006IAT 4108 Play Testing  Play testing follows software testing –Overlap of phases  Software Testing + More –Tuning game play –Tuning that Flow experience!  Discover Intellectual Space –Does the player Get it?? –Less important in established genres

9. Nov 20, Fall 2006IAT 4109 Team Structure  Fundamental issue: Willful blindness  Don’t fall in love with your code!  Having testing team members be different from your coders is vital  A Software Testing team  A Play Testing Team

10. Nov 20, Fall 2006IAT 41010 Play Testing Team  Internal team –Manages play-testing people –Does own play tests –Recruits outsiders for a week or two  External team –A population sample –Varying skills –Unfamiliar with product –Not in love with your product! –Not yet bored with the product

11. Nov 20, Fall 2006IAT 41011 External Testers  Do they like the game? –They may lie to you –Politeness effect If they can’t say why they like it, you have a problem  Do they get frustrated? –What common areas? –Often skill-based Internal team judges player’s skill, also

12. Nov 20, Fall 2006IAT 41012 External Testers  FAQ –Be prepared to deal with the frequent questions in the final game

13. Nov 20, Fall 2006IAT 41013 External Testers: Half-Life  People near Valve’s offices who sent in registration cards  Designers sit quietly while player struggles –Designer takes notes  Typical 2-hour session –100 action items  First 20-30 sessions absolutely vital –Learn what was fun –200 sessions total

14. Nov 20, Fall 2006IAT 41014 Half-Life: Fine tuning  Add instrumentation –Player position, time, health, weapons –Activities: Game save, dying, being hurt, solving a puzzle, fighting a monster…  Graph series of sessions together –Spot too long with no encounters –Spot long periods of too much health –Spot long periods of too little health

15. Nov 20, Fall 2006IAT 41015 Half-Life  Most important playtesting outcome:  Clearly identified good and bad ideas  Allowed people to…

16. Nov 20, Fall 2006IAT 41016 Half-Life  Most important playtesting outcome:  Clearly identified good and bad ideas  Allowed people to Abandon Bad Ideas  Playtesting provides the evidence needed

17. Nov 20, Fall 2006IAT 41017 Advice  Don’t get defensive! –The tester opinion is important  Testers: Stick to your findings! –Respectfully point out problems  Mix up the hardware –Be honest about the specs on the box

18. Nov 20, Fall 2006IAT 41018 More Advice  Discourage Legacies –Don’t let bad decisions live forever! –MS-DOS!