1. Queensland University of Technology CRICOS No. 000213J ITB/ITN751 Games Production Lecture 6 Quality Assurance Ross Brown

2. CRICOS No. 000213J a university for the world real R Lecture Contents Game Programming Guest Lecture – Scott Cameron, Halfbrick

3. CRICOS No. 000213J a university for the world real R Readings Textbook Chapters: –Chapter 11 Websites: –http://www.igda.org/qa/index.php - IGDA QA Special Interest Grouphttp://www.igda.org/qa/index.php –http://www.sloperama.com/advice/lesson5.htm - some practical advice on game testing employmenthttp://www.sloperama.com/advice/lesson5.htm –http://www.gamasutra.com/php- bin/news_index.php?story=13338 – some thoughts on QAhttp://www.gamasutra.com/php- bin/news_index.php?story=13338

4. CRICOS No. 000213J a university for the world real R Quality and Quantity “Those who speak most of progress measure it by quantity and not by quality” – George Santayana

5. CRICOS No. 000213J a university for the world real R Infinity Ward Studio Tour Quality Assurance http://youtube.com/watch?v=2e2xPRDQRB8

6. CRICOS No. 000213J a university for the world real R What is Quality Assurance? The step of the game development life cycle that ensures a game is: –Bug free –Functional –Usable –Fun Is similar to the usability testing utilised within other ICT industry sectors – testing cycles

7. CRICOS No. 000213J a university for the world real R Why so important? To make sure the game is bug free Improves acceptance of game Makes sure that game concept is going to sell – compare to screenings for movies

8. CRICOS No. 000213J a university for the world real R And… Builds rapport with your future audience via the use of bug testing groups Games are very complex pieces of software, that will be run on many forms of hardware While having a word processor crashing as a piece of software is bad

9. CRICOS No. 000213J a university for the world real R No Flow Having a game crash is simply destructive of the immersion experience Thus giving your game players a bad experience of the game

10. CRICOS No. 000213J a university for the world real R Piracy Prevents piracy…most cracking requires a bug exploit Prevents cheating – again, unless the codes are easter eggs, you can prevent cheats from misusing your game… Check: http://videogames.lovetoknow. com/wiki/Category:Video_Gam e_Cheat_Codes http://en.wikipedia.org/wiki/Image:Jolly-roger.svg

11. CRICOS No. 000213J a university for the world real R Processes Alpha Testing Focus Group Testing Beta Testing Ongoing Bug Reports Competitive Analysis Publisher Testing

12. CRICOS No. 000213J a university for the world real R Pre-Alpha Testing Pre-alpha code is not feature complete and is often full of bugs Thus no testing pre-alpha by Q/A as pointless Pre-alpha bug fixing is handled by development team as matter of course

13. CRICOS No. 000213J a university for the world real R Alpha Testing Alpha is typically [Textbook] defined as code that is content and feature complete, but may have bugs The bugs in this case may crash the system This testing is done in house by Q/A team and ad hoc development meetings Part of the overall development process defined by the timetable of the producer

14. CRICOS No. 000213J a university for the world real R Closed Beta Testing Closed Beta – has all content features and assets complete No crashing bugs Usually used in focus groups with players, but not released to public

15. CRICOS No. 000213J a university for the world real R Focus Group Testing Gather a group of people to play a beta of the game Concentrates on the fun aspects of the game May video the players expressions to correlate gameplay with emotional state of players

16. CRICOS No. 000213J a university for the world real R Open Beta Halo 3 Anyone? Beta is released to closed subscribers for play testing Can report on discovered bugs Can track information about the gameplay via network links to the home server http://www.clubskill.com/Game_News/3779

17. CRICOS No. 000213J a university for the world real R Bugs vs Gameplay It should be noted here that the testing processes is divided up into bug discovery in code, and gameplay issues. A program may be bug free and not crash But it may not be an effective game

18. CRICOS No. 000213J a university for the world real R Software Bug Testing Looking for software bugs that cause a number of effects: Write specific test plans to look for bugs Use regression testing to keep changes within test limits Can use automated testing for these purposes, ie. little test programs which use the new code

19. CRICOS No. 000213J a university for the world real R Software Bug Testing Seek to replicate circumstances the cause game to crash, and or crash the system. Usual technical issues of memory problems, bad code logic, OBOEs – Out By One Errors Game Mechanics Visual and Audio Effects – game runs fine but the appearance is affected, eg. Holes in objects, textures destroyed, passing through walls etc.

20. CRICOS No. 000213J a university for the world real R Gameplay Testing Designers/Balancers and other development team members have interests in getting the gameplay right Thus Q/A is also about assessing the Quality of the gameplay as well as the Correctness of software code Game Development Team is usually interested in a number of gameplay factors

21. CRICOS No. 000213J a university for the world real R Some Gameplay Factors Assessed Mission – uptake and completion Content Usage – reaction to, amongst other things, rewards, weapons, economy Player vs Player – balancing factors, are the players able to fight fairly Difficulty – levels and puzzles balance of challenge and ease of completion

22. CRICOS No. 000213J a university for the world real R Some Gameplay Factors Assessed Social Factors – chat usage, social locations Emergent Gameplay – social hangouts, novel use of weapons User Interface – How players configure their games

23. CRICOS No. 000213J a university for the world real R Publisher/Manufacturer Testing Console companies have VERY stringent testing regimes regarding the release of games Final CD/DVD of game is submitted to publisher/manufacturer to test May not pass on first round, and requires resubmission

24. CRICOS No. 000213J a university for the world real R Technology Typical collaboration tools: wikis, spreadsheets, or web tools for communication of bug reports Bug tracking software Integrated into engine/game build processes within the development teams – use of live and development versions of games

25. CRICOS No. 000213J a university for the world real R Technology Video camera for player focus testing Benchmarking tools – to test the game to a predefined standard of frame rates on defined hardware profiles Integrated Version Control Systems –version control systems (eg. Subversion, Alienware) allow for the tracking of versions of documents, especially content and software –also known as Digital Asset Management Systems

26. CRICOS No. 000213J a university for the world real R Technology Debug builds of game engines for logging of data Bug reporting tools as spreadsheets or web- based tools for reporting bugs to development team Visualisation software for test data to see patterns

27. CRICOS No. 000213J a university for the world real R Skills Is often the entrance point for people into the game industry Game producers often come from a QA background Does not require specific training - we don’t have a QA minor in this degree ;-) Design/Culture/Business units in Games Degree are probably the best point to gain knowledge Also play games and volunteer for game testing focus groups

28. CRICOS No. 000213J a university for the world real R Skills Enthusiasm – checking holes in walls is boring after all Deep understanding of gameplay and what is fun Thorough approach to playing and checking game environments – playing and checking are two different things

29. CRICOS No. 000213J a university for the world real R Skills People skills for communicating bugs to other development team members – polite positive criticism People skills for receiving criticism from focus groups and beta testers – some criticism is not going to be nice

30. CRICOS No. 000213J a university for the world real R Roles Lead QA – manages the complete testing team and testing/QA processes under supervision of the producer and is part of the development team Assistant QA – Manages testing of components of game Testers – who do the wall crawling etc. Producer/Designer/Lead Developer – interface to development team

31. CRICOS No. 000213J a university for the world real R Roles and then some… Roles change depending on at what level the testing is performed With the latest SCRUM and Agile software methodologies, there is an emphasis on unit testing as a way of ensuring quality code Each unit of code of only accepted after its accessed as being complete

32. CRICOS No. 000213J a university for the world real R Assignment 2: Sim Game Studio Sim Game Studio Team Assignment Gain an understanding of setting up and running a new studio for three years

33. CRICOS No. 000213J a university for the world real R Assignment 2: Sim Game Studio Team worker set up You will lose marks if not in team by IMPORTANT: Form teams NOW and have regular meetings – like right this minute during the break!

34. CRICOS No. 000213J a university for the world real R Guest Lecturer Scott Cameron – Quality Assurance Halfbrick Studios

35. CRICOS No. 000213J a university for the world real R References Secrets of the game business / edited by François Dominic Laramée, Hingham, MA: Charles River Media, c2005 – has a section on Quality Assurance in QUT Library Dasso, Aristides Verification, Validation and Testing in Software Engineering, Hershey, PA: Idea Group, Inc., 2006 – online book via QUT library on software testing, as there are no actual books on game QA proper