METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

OUTLINE OF TALK METAGAME Chess-Like Games and Generation METAGAMER Performance Related Work Implications for learning and reasoning in games Conclusion

Knight-Zone Chess

META-GAME PLAYING Diverse Class of Games Automated Game Designer Uniform Representation Programs Must Analyze Rules No Existing Experts Evaluation by Metagame Tournament Increase challenge by extending class over time

TOURNAMENT FORMAT Accept Rules Initial Analysis Individual Contests Post-Mortem Analysis Time Limits No Programmer Modification Winner

Computer Game-Playing Research Class of games General game knowledge Resource bounds Game rules Competitive context Player Specific game knowledge Opponent Player (Minimal use today) (Heavy use today)

Computer Game-Playing Research Class of games General game knowledge Resource bounds Game rules Competitive context Player Specific game knowledge Opponent Player

Computer Game-Playing Research Class of games General game knowledge Resource bounds Game rules Competitive context Player Specific game knowledge Opponent Player

Game rules Game generator Metagamer Opponent Metagamer Meta-Game-Playing Research Class of games General game knowledge Resource bounds Game rules Competitive context Player Specific game knowledge Opponent Player

Class and Generator Symmetric Chess-Like Games –Global Symmetry –Board Pieces Initial Setup Goals – Includes many known games of varying complexity Game Generator – Stochastic Context-Free Generation – Controllable Parameters –Generates some interesting games

METAGAMER Class and Strategy in General Representation Game-Specializer: Compiles to Improve Efficiency Game-Analyzer: Produces Specialized Analysis Tables Advisors: Use Analysis Tables to Evaluate Position Weights –Relative Importance of General Advisors –Tuned by experiments –Values not as crucial as for base-level Search Engine: Alpha-Beta Minimax

Advisors for Chess-Like Games Mobility –dynamic-mobility –static-mobility –capturing-mobility –eventual-mobility Threats and Capturing –global-threats – potent-threats –possession Goals and Step Functions –Vital –arrival-distance –promote-distance

Results in Competition Checkers –Stronger than Greedy-Material –1-man handicap ==> draws strong opponent –Strong if 1-man handicap Chess –Stronger than Greedy-Material –Can Defeat Human Novices –Good Positional Play, Weak Tactics Other games –Chinese chess, Japanese chess, Chess variations: Sensible play Generated Games (w/o human assistance) –All Advisors ==> won Tourney –No Version was best on every game –Knowledge outperforms Search (so far!) "Rediscovers" Known Strategies Long-range strategic capabilities with limited search Learning Gives Improvement

Related work Other work in learning and planning games –Forks, abstraction, parameter-learning, feature-learning and generation Metagamer works on unknown games Does not rely on strong opponents Benefits from Rules Plays Entire Game

Implications for learning and planning in general games Game analysis like scientific investigation Intellectual development –Discipline for perceiving, searching, reacting, time mgmt –Practice and training –Progression of skills Multi-strategy approaches –Constraint-based design –Theorems and lemmas –Analogies –Theory-driven experiments –Exploration and Trial and error Cultural –Transfer of knowledge –Authorship and history

CONCLUSION Metagame reveals wide open problems Attractive properties as evaluation testbed –Competitive performance criteria –Quantifiable demonstration of generality –Requires learning and reasoning on integrated problems –Humans have high competence, so impressive if programs could play well –Increasing challenges over time More general classes of problems (eg chess + go) Larger scale problems (bigger boards, more pieces) More complex domain attributes (eg multi-player, incomplete information, chance) Chess-Like Games is a good start –Existence proof that something is possible here –Hard problem (little improvement in 10 years!) –Workbench makes development easy Similar ideas could be applied to other challenges –Eg. planning, categorization, robotics competitions Key to any of these –Quantify claims of generality to the information available to humans in system –Removing information forces new challenges for agents