Game Optimization through Large-Scale Experimentation Erik Andersen Ph.D. student Center for Game Science University of Washington Computer Science Dept.

Slides:

Advertisements

Similar presentations

What do you want to be? What are you good at? What do you like?

Advertisements

Jack Buckley Commissioner National Center for Education Statistics May 10, 2012.

EGR 141 Computer Problem Solving in Engineering and Computer Science

AB 11 22 33 44 55 66 77 88 99 10  20  19  18  17  16  15  14  13  12  11  21  22  23  24  25  26  27  28.

Tic Tac Toe Game Design Using OOP

 Title: QWOP  Developer: Bennett Foddy  Genre: Casual Skill Game  Price: Free  Requirements: A computer with internet and flash.

For the ultimate in entertainment, play with 2 or more people, individually or as a team Players alternate turns Each player picks an “answer” and must.

Multimedia Authoring 2 Game Concept Snakes & Ladders 3D Sjaak Wouters Wouter den Boer.

A Service Platform for On-Line Games DebanJan Saha, Dambit Sahu, Anees Shaikh (IBM TJ Watson Research Center, NY) Presented by Gary Huang March 17, 2004.

Gameplay Analysis through State Projection Erik Andersen 1, Yun-En Liu 1, Ethan Apter 1, François Boucher-Genesse 2, Zoran Popović 1 1 Center for Game.

CIS 587 Game Review: Madden 2004 for PlayStation 2.

Project Pitch Date: Friday, June 16th. Introduction Present game to independent panel. Get them to care about your game –Ex: Publishers (deep pockets)

Monopoly Information for Assignment. Game Management Subsystem Game manager Deregister player Edit game View games Set up game > View game state Start.

Game Budgets. Costs Developers Software Computers Marketing Legal.

My free time. Piano In my free time I like playing the piano.I am graduate of Basic art school. When I studied at Basic art school I learnt classical.

University of Louisiana at Monroe 702 Cole Avenue Monroe, LA Emily Williamson, Director

Aquaphobia Aaron Crutchfield. Why I Made This Game.

Lists. Lists store information in a certain order. You can add, remove, or rearrange items in a list. You can also pick out information from a list.

LAUP AM Class Schedule 8:30 am - 9:00 am Breakfast 9:05 am - 9:20 am Circle Time ( Greeting, Calendars, Song/Poem 9:25 am - 10:25 am Open Centers ( Art,

Bekir Mugayitoglu, MBA Learn to code, code to learn.

1 Learning from Scratch Pete Benson U of Michigan School of Ed, Secondary MAC Woodrow Wilson Teaching Fellow.

Art and Science. Art Goal: Communicate Human perception and aesthetics Grounded in Science Admits Qualitative and Subjective elements ConcreteAbstract.

Last Fan Standing TURN YOUR FAN ON!. What is Last Fan Standing? Mobile sports trivia application The quickest, most accurate fan(s) wins the game Level.

Click here to add a picture of your country’s flag Click here to add your country’s name.

Resource Sheet 8.5. What do you like to do after school?

What do you like doing ? I like cooking. What do you like doing ? I like reading.

Appendix 6 College Comparisons. Mean Total Score by College (Possible Score Range 400 to 500) SSD = Total Scores for Colleges of Business, Education,

2010 FLVS AP QUALIFYING RESULTS FLVS AP COURSENATIONALFLORIDAFLVS AP Art History 61%46%40% AP Biology 49%32%31% AP Calculus AB 55%47%45% AP Calculus BC.

Clue-less Human Computer Interface Synergy. Login Screen Drop down menu for character selection. Selected characters will be grayed out These are players.

Done by: Mihailova A.-9”b” form Scientific leader: Andreeva T. N.- a teacher of English Chagasy MUSIC IN OUR LIFE Сhagasy secondary school Kanash.

IB Computer Science – Logic

CULTURE. SCIENCE/HEA LTH PHYSICAL EDUCATION Use Prezi as an interesting way of introducing how science is used in different cultures Use pedometers as.

Culture Fav Sources Goals Hobbies My name is Juan I was born in Mexico.Sins I was 4 I have not gone to Mexico but in December I am going to go for the.

ST Math Small Group Ideas. Warm-Up…JiJi Style WHO: Small groups of 3-5 students WHAT: Engaged math lesson warm-up using ST Math Test Drive WHEN: During.

Hebrew in the Online Learning Center Student & Family Information.

From Pure Math to Data Science: My Perspective Paul Raff Principal Data Scientist, Analysis and Experimentation, Microsoft October 1, 2015 My

Sleeping Elephants Game. subject your.

Activity3. Play “Pass the phone game” 1. Pass the two phones. Green and blue phones. 2. Listening to the music, Pass the two phones to others. 3. When.

The children are moving to music, played on a CD player in a PE lesson.

Smart Player Piano Student: Jason Pawlak Advisor: Dr. Anca Ralescu.

Department of educational leadership instructional technology program Games for Learning: Using Game Play and Game Design to Facilitate Learning Dr. Wayne.

Let the Games Begin Debbie Sharp and Sue Smith GAMING IN INFORMATION LITERACY.

ViralFlux and Logo Start New Game Load Save Game Start Multiplayer Game Change Settings Exit Game ViralFlux Flavor Text.

CISC Machine Learning for Solving Systems Problems Microarchitecture Design Space Exploration Lecture 4 John Cavazos Dept of Computer & Information.

Brent M. Dingle Game Design and Development Program Mathematics, Statistics and Computer Science University of Wisconsin - Stout.

GAME DESIGN Elements of Engagement. Objectives Know the relationship between immersion and engagement Detail the elements needed to create audience –

Adding Sounds Games Programming in Scratch. Games Programming in Scratch L6 Adding Sounds Learning Objectives Learn how to add sound to a Scratch game.

Baseball Boss Run the Show Play with all of your favorite 2011 MLB players.

Last Fan Standing TURN YOUR FAN ON!. What is Last Fan Standing? Mobile sports trivia application The quickest, most accurate fan(s) wins the game Level.

Name Homeroom How to Play For all EVEN numbered slides write a question in the large black box In the small black boxes on those same slides provide.

Making Learning Fun Again Sarah Kim Education and Computers

Verb “To Be” practice Let’s meet my friends !....

Game Monetisation Fundamental Strategies in use Today.

Music Learning Game Sourcecode SELLMYSOURCECODE. INTRODUCTION Do your kids love music? If yes then let’s your child learn music and play on phone or tablet.

Www3.informatik.uni-wuerzburg.de Institute of Computer Science Chair of Communication Networks Prof. Dr.-Ing. P. Tran-Gia Simulation Framework for Live.

Can you learn in the classroom?. Learn in the classroom, learn in the classroom. Can you learn in the classroom? Yes, … yes, you can.

By Aric Krause Ethan Young.  Home audio system owners spend time setting up their speakers for the optimal audio experience, but this is rendered useless.

PYSS Lower Secondary Maths Games Design Project. What are Maths Games? Interactive games to pique student’s interest in mathematics An alternative approach.

A Holistic Approach to A/B Testing Jesse Hull Executive Director Technology, GSN Games.

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1 In-Jeong Chung Intelligent Information System lab. Department.

Being developed by Kris Seque and Morgan Pickford.

CHANGES  Fixed dug dirt bug  Background Color + Glow  Random Level Generation  Point Value Display  New sounds.

Science of Nurture Quiz Session 5 Module TESTING AND PERFORMANCE OPTIMIZATION QUIZ.

Introduction of Scratch (1/4) You can find Scratch on the Web:

Hello Educational presentation.

What do we have at school?

Year 7 Homework Timetable History Geography Maths English

ضرورت آموزش و یادگیری الکترونیکی در مدارس

Reading: ‘Washington, DC’

Presentation transcript:

Game Optimization through Large-Scale Experimentation Erik Andersen Ph.D. student Center for Game Science University of Washington Computer Science Dept.

Music Gameplay Sound effects Art

engagement retention revenue

A/B Testing

AB

Metrics ● Levels completed ● Time played ● Return rate

Aesthetics Secondary Objectives Tutorials

Foldit RefractionHello Worlds

Refraction

Rating: 3.8 / 5 400,000 plays

Hello Worlds!

Rating: 4.2 / 5 1,385,000 plays

Foldit

240,000 players

Player Tracking ● Flash cache / login name ● New players only

Statistical significance 95% confidence level (p<0.05)

Experiment #1: Audio Sound Effects Music

Result: Music and sound effects did not matter

Experiment #2: Animations

Result: Animations improved engagement

No Animations Refraction

No Animations Refraction

No Animations Refraction

No Animations Hello Worlds

No Animations Hello Worlds

Experiment #3: Secondary Objectives (Super Mario Bros., Nintendo 1985)

Levels % of players 100 0

Levels % of players With Coins Hypothesis

Levels % of players With Coins Reality

Time % of players What about time played?

Time % of players With Coins Hypothesis

Time % of players Hypothesis With Coins

Time % of players With Coins Reality: Many Players Quit Sooner

Time % of players 100 0

Result: secondary objectives harmed engagement

(Assassin’s Creed, Ubisoft 2007)

Maybe easier is better? (Super Mario Bros., Nintendo 1985)

Time % of players Off-path Coins

Time % of players On-path Coins

Result: secondary objectives were good if they supported the primary objectives

Experiment #4: Tutorials

No Animations Foldit

No Animations Foldit

Refraction and Hello Worlds: no effect

Result: text tutorials helped only in the most complex game

Number of concepts

Context-sensitivity Context-sensitiveContext-insensitive

Result: context-sensitive help was better (if tutorials helped at all)

No Animations Foldit

No Animations Foldit

Let’s gather more data ● Let us A/B test your games!

Coauthors Yun-En Liu Richard Snider Roy Szeto Seth Cooper Eleanor O’Rourke Jeff Lowdermilk David Truong Zoran Popović

Acknowledgements Kongregate: August Brown and Anthony Pecorella Logging: Kefan Xu Refraction Team: Erik Andersen, Yun-En Liu, Marianne Lee, Eric Butler, Brian Britigan, Stephen Sievers, Roy Szeto, Mai Dang, Christian Lee, Ethan Apter, Emma Lynch, Happy Dong, Zorah Lea Fung, Justin Irwen, Seth Cooper, François Boucher-Genesse, Zoran Popović Hello Worlds Team: Rich Snider, Michael Eng, Marianne Lee, Blake Thompson, Jeff Flatten Foldit Team: Seth Cooper, Adrien Treuille, Firas Khatib, Janos Barbaros, Joshua Snyder, Alex Cho Snyder, Jeff Flatten, Jeff Lowdermilk, Dun-Yu Hsiao, Jeehyung Lee, David Salesin, David Baker, Zoran Popović