1. March 24, Spring 2004CS44551 Play Testing Software Testing Play Testing Team Structures

2. March 24, Spring 2006CS 44552 Software Testing Goal: “Prove” that the software works –Matches the functional specification –Matches the performance specification Real Proof: NP-Hard problem –Simulate all the possible runs –Exponential growth in testing time –Each branch point multiplies possible program state by 2

3. March 24, Spring 2006CS 44553 Testing Method Run the program using a suite of data –Data suite exercises as much of the program’s code as possible –Each data set is accompanied by expected results –Where the results are unexpected, there is a bug!

4. March 24, Spring 2006CS 44554 Generating Test Suites Black Box Testing –Generate test data based purely on interpretation of the specifications –Don’t look at the code! –Code is a Black Box

5. March 24, Spring 2006CS 44555 Generating Test Suites White Box Testing –Use the code to generate tests with the purpose of exercising all blocks of code Central concept: Coverage Cover all of the parts of the programs code –Ensure that all lines of code run at least once

6. March 24, Spring 2006CS 44556 Combine Both Approaches Black Box + White Box test suites –Complementary roles –Hard to tell from the data what approach was used –Aims to generate that ever-elusive “proof” Test suites: –Great for testing Compilers –Perhaps not so great for interaction and games

7. March 24, Spring 2006CS 44557 Random Testing Some applications are hard to test: Random test –Give up the idea of formal coverage –Hope for stochastic coverage! Sometimes this depends on actual human interaction –Need to get lots of people –Beta testing by customers

8. March 24, Spring 2006CS 44558 Play Testing What is it?

9. March 24, Spring 2006CS 44559 Play Testing Play testing follows software testing –Overlap of phases Software Testing + More –Tuning game play –Tuning that Flow experience! Discover Intellectual Space –Does the player Get it?? –Less important in established genres

10. March 24, Spring 2006CS 445510 Team Structure Fundamental issue: Willful blindness Don’t fall in love with your code! Having testing team members be different from your coders is vital A Software Testing team A Play Testing Team

11. March 24, Spring 2006CS 445511 Play Testing Team Internal team –Manages play-testing people –Does own play tests –Recruits outsiders for a week or two External team –A population sample –Varying skills –Unfamiliar with product –Not in love with your product! –Not yet bored with the product

12. March 24, Spring 2006CS 445512 External Testers Do they like the game? –They may lie to you –Politeness effect If they can’t say why they like it, you have a problem Do they get frustrated? –What common areas? –Often skill-based Internal team judges player’s skill, also

13. March 24, Spring 2006CS 445513 External Testers FAQ –Be prepared to deal with the frequent questions in the final game

14. March 24, Spring 2006CS 445514 External Testers: Half-Life People near Valve’s offices who sent in registration cards Designers sit quietly while player struggles –Designer takes notes Typical 2-hour session –100 action items First 20-30 sessions absolutely vital –Learn what was fun –200 sessions total

15. March 24, Spring 2006CS 445515 Half-Life: Fine tuning Add instrumentation –Player position, time, health, weapons –Activities: Game save, dying, being hurt, solving a puzzle, fighting a monster… Graph series of sessions together –Spot too long with no encounters –Spot long periods of too much health –Spot long periods of too little health

16. March 24, Spring 2006CS 445516 Half-Life Most important playtesting outcome: Clearly identified good and bad ideas Allowed people to…

17. March 24, Spring 2006CS 445517 Half-Life Most important playtesting outcome: Clearly identified good and bad ideas Allowed people to Abandon Bad Ideas Playtesting provides the evidence needed

18. March 24, Spring 2006CS 445518 Advice Don’t get defensive! –The tester opinion is important Testers: Stick to your findings! –Respectfully point out problems Mix up the hardware –Be honest about the specs on the box

19. March 24, Spring 2006CS 445519 More Advice Discourage Legacies –Don’t let bad decisions live forever! –MS-DOS!