METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center.

Slides:



Advertisements
Similar presentations
CMSC 471 Spring 2014 Class #9 Tue 2/25/14 Game Playing Professor Marie desJardins,
Advertisements

Game Playing CS 63 Chapter 6
Adversarial Search Chapter 6 Section 1 – 4. Types of Games.
10/29/01Reinforcement Learning in Games 1 Colin Cherry Oct 29/01.
CS 484 – Artificial Intelligence
Adversarial Search Chapter 5.
COMP-4640: Intelligent & Interactive Systems Game Playing A game can be formally defined as a search problem with: -An initial state -a set of operators.
Application of Artificial intelligence to Chess Playing Capstone Design Project 2004 Jason Cook Bitboards  Bitboards are 64 bit unsigned integers, with.
Adversarial Search CSE 473 University of Washington.
MINIMAX SEARCH AND ALPHA- BETA PRUNING: PLAYER 1 VS. PLAYER 2.
1 Adversarial Search Chapter 6 Section 1 – 4 The Master vs Machine: A Video.
Computers Playing Games Arif Zaman CS 101. Acknowledgements Portions of this are taken from MIT’s open-courseware
An Alternative Approach for Playing Complex Games like Chess. 1Alternative Game Playing Approach Jan Lemeire May 19 th 2008.
Minimax and Alpha-Beta Reduction Borrows from Spring 2006 CS 440 Lecture Slides.
An Integration of Program Analysis and Automated Theorem Proving Bill J. Ellis & Andrew Ireland School of Mathematical & Computer Sciences Heriot-Watt.
Game Playing CSC361 AI CSC361: Game Playing.
Learning Shape in Computer Go David Silver. A brief introduction to Go Black and white take turns to place down stones Once played, a stone cannot move.
UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering CSCE 580 Artificial Intelligence Ch.6: Adversarial Search Fall 2008 Marco Valtorta.
1 search CS 331/531 Dr M M Awais A* Examples:. 2 search CS 331/531 Dr M M Awais 8-Puzzle f(N) = g(N) + h(N)
Monte Carlo Go Has a Way to Go Haruhiro Yoshimoto (*1) Kazuki Yoshizoe (*1) Tomoyuki Kaneko (*1) Akihiro Kishimoto (*2) Kenjiro Taura (*1) (*1)University.
Othello Sean Farrell June 29, Othello Two-player game played on 8x8 board All pieces have one white side and one black side Initial board setup.
Reinforcement Learning of Local Shape in the Game of Atari-Go David Silver.
The National Curriculum The national curriculum for physical education aims to ensure that all pupils: develop competence to excel in a broad range of.
Game Playing: Adversarial Search Chapter 6. Why study games Fun Clear criteria for success Interesting, hard problems which require minimal “initial structure”
GoogolHex CS4701 Final Presentation Anand Bheemarajaiah Chet Mancini Felipe Osterling.
Ocober 10, 2012Introduction to Artificial Intelligence Lecture 9: Machine Evolution 1 The Alpha-Beta Procedure Example: max.
1 Adversary Search Ref: Chapter 5. 2 Games & A.I. Easy to measure success Easy to represent states Small number of operators Comparison against humans.
Hex Combinatorial Search in Game Strategy by Brandon Risberg May 2006Menlo School.
Ch1 AI: History and Applications Dr. Bernard Chen Ph.D. University of Central Arkansas Spring 2011.
Game Playing.
Senior Project – Computer Science – 2015 Modelling Opponents in Board Games Julian Jocque Advisor – Prof. Rieffel Abstract Modelling opponents in a game.
Game Playing Chapter 5. Game playing §Search applied to a problem against an adversary l some actions are not under the control of the problem-solver.
Evaluation Function in Game Playing Programs M1 Yasubumi Nozawa Chikayama & Taura Lab.
Game-playing AIs Part 1 CIS 391 Fall CSE Intro to AI 2 Games: Outline of Unit Part I (this set of slides)  Motivation  Game Trees  Evaluation.
Adversarial Search Chapter 6 Section 1 – 4. Outline Optimal decisions α-β pruning Imperfect, real-time decisions.
Introduction to Chess Club
Application of Artificial Intelligence to Chess Playing Jason Cook Capstone Project.
Memory and Analogy in Game-Playing Agents Jonathan Rubin & Ian Watson University of Auckland Game AI Group
Games. Adversaries Consider the process of reasoning when an adversary is trying to defeat our efforts In game playing situations one searches down the.
CHECKERS: TD(Λ) LEARNING APPLIED FOR DETERMINISTIC GAME Presented By: Presented To: Amna Khan Mis Saleha Raza.
PYIWIT'021 Threat Analysis to Reduce the Effects of the Horizon Problem in Shogi Reijer Grimbergen Department of Information Science Saga University.
GAME PLAYING 1. There were two reasons that games appeared to be a good domain in which to explore machine intelligence: 1.They provide a structured task.
Outline Intro to Representation and Heuristic Search Machine Learning (Clustering) and My Research.
Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 9 of 42 Wednesday, 14.
Artificial Intelligence and Searching CPSC 315 – Programming Studio Spring 2013 Project 2, Lecture 1 Adapted from slides of Yoonsuck Choe.
Game Playing Revision Mini-Max search Alpha-Beta pruning General concerns on games.
Definitions of AI There are as many definitions as there are practitioners. How would you define it? What is important for a system to be intelligent?
Course Overview  What is AI?  What are the Major Challenges?  What are the Main Techniques?  Where are we failing, and why?  Step back and look at.
Game tree search Chapter 6 (6.1 to 6.3 and 6.6) cover games. 6.6 covers state of the art game players in particular. 6.5 covers games that involve uncertainty.
ARTIFICIAL INTELLIGENCE (CS 461D) Princess Nora University Faculty of Computer & Information Systems.
Computers and Games Board Maps and Hill-Climbing for Opening and Middle Game Play in Shogi Reijer Grimbergen (Saga University) Jeff Rollason (Oxford.
ITEC 1010 Information and Organizations Chapter V Expert Systems.
February 25, 2016Introduction to Artificial Intelligence Lecture 10: Two-Player Games II 1 The Alpha-Beta Procedure Can we estimate the efficiency benefit.
Adversarial Search Chapter 5 Sections 1 – 4. AI & Expert Systems© Dr. Khalid Kaabneh, AAU Outline Optimal decisions α-β pruning Imperfect, real-time decisions.
ADVERSARIAL SEARCH Chapter 6 Section 1 – 4. OUTLINE Optimal decisions α-β pruning Imperfect, real-time decisions.
Understanding AI of 2 Player Games. Motivation Not much experience in AI (first AI project) and no specific interests/passion that I wanted to explore.
The Game Development Process: Artificial Intelligence.
Game Playing Why do AI researchers study game playing?
By Kevin Madison and Emma Drobina
Done Done Course Overview What is AI? What are the Major Challenges?
Adversarial Search Chapter 5.
Artificial Intelligence and Searching
The Alpha-Beta Procedure
Mini-Max search Alpha-Beta pruning General concerns on games
Search.
Search.
Adversarial Search CS 171/271 (Chapter 6)
Artificial Intelligence and Searching
Goals for Learning in the Geosciences
Artificial Intelligence and Searching
Presentation transcript:

METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

OUTLINE OF TALK METAGAME Chess-Like Games and Generation METAGAMER Performance Related Work Implications for learning and reasoning in games Conclusion

Knight-Zone Chess

META-GAME PLAYING Diverse Class of Games Automated Game Designer Uniform Representation Programs Must Analyze Rules No Existing Experts Evaluation by Metagame Tournament Increase challenge by extending class over time

TOURNAMENT FORMAT Accept Rules Initial Analysis Individual Contests Post-Mortem Analysis Time Limits No Programmer Modification Winner

Computer Game-Playing Research Class of games General game knowledge Resource bounds Game rules Competitive context Player Specific game knowledge Opponent Player (Minimal use today) (Heavy use today)

Computer Game-Playing Research Class of games General game knowledge Resource bounds Game rules Competitive context Player Specific game knowledge Opponent Player

Computer Game-Playing Research Class of games General game knowledge Resource bounds Game rules Competitive context Player Specific game knowledge Opponent Player

Game rules Game generator Metagamer Opponent Metagamer Meta-Game-Playing Research Class of games General game knowledge Resource bounds Game rules Competitive context Player Specific game knowledge Opponent Player

Class and Generator Symmetric Chess-Like Games –Global Symmetry –Board Pieces Initial Setup Goals – Includes many known games of varying complexity Game Generator – Stochastic Context-Free Generation – Controllable Parameters –Generates some interesting games

METAGAMER Class and Strategy in General Representation Game-Specializer: Compiles to Improve Efficiency Game-Analyzer: Produces Specialized Analysis Tables Advisors: Use Analysis Tables to Evaluate Position Weights –Relative Importance of General Advisors –Tuned by experiments –Values not as crucial as for base-level Search Engine: Alpha-Beta Minimax

Advisors for Chess-Like Games Mobility –dynamic-mobility –static-mobility –capturing-mobility –eventual-mobility Threats and Capturing –global-threats – potent-threats –possession Goals and Step Functions –Vital –arrival-distance –promote-distance

Results in Competition Checkers –Stronger than Greedy-Material –1-man handicap ==> draws strong opponent –Strong if 1-man handicap Chess –Stronger than Greedy-Material –Can Defeat Human Novices –Good Positional Play, Weak Tactics Other games –Chinese chess, Japanese chess, Chess variations: Sensible play Generated Games (w/o human assistance) –All Advisors ==> won Tourney –No Version was best on every game –Knowledge outperforms Search (so far!) "Rediscovers" Known Strategies Long-range strategic capabilities with limited search Learning Gives Improvement

Related work Other work in learning and planning games –Forks, abstraction, parameter-learning, feature-learning and generation Metagamer works on unknown games Does not rely on strong opponents Benefits from Rules Plays Entire Game

Implications for learning and planning in general games Game analysis like scientific investigation Intellectual development –Discipline for perceiving, searching, reacting, time mgmt –Practice and training –Progression of skills Multi-strategy approaches –Constraint-based design –Theorems and lemmas –Analogies –Theory-driven experiments –Exploration and Trial and error Cultural –Transfer of knowledge –Authorship and history

CONCLUSION Metagame reveals wide open problems Attractive properties as evaluation testbed –Competitive performance criteria –Quantifiable demonstration of generality –Requires learning and reasoning on integrated problems –Humans have high competence, so impressive if programs could play well –Increasing challenges over time More general classes of problems (eg chess + go) Larger scale problems (bigger boards, more pieces) More complex domain attributes (eg multi-player, incomplete information, chance) Chess-Like Games is a good start –Existence proof that something is possible here –Hard problem (little improvement in 10 years!) –Workbench makes development easy Similar ideas could be applied to other challenges –Eg. planning, categorization, robotics competitions Key to any of these –Quantify claims of generality to the information available to humans in system –Removing information forces new challenges for agents