Today’s Topics Playing Deterministic (no Dice, etc) Games –Mini-max –  -  pruning –ML and games? 1997: Computer Chess Player (IBM’s Deep Blue) Beat Human Chess Champ (Kasparov) 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 51

Mini-Max Algorithm Generate Search Tree of ALL Possible Games Score Leaves (Assume Higher Better for ‘You’) eg, +1 you win, -1 you lose, 0 if a draw (ie, a tie) Assume Opponent Scores ‘Board Configurations’ the Same Way You Do (more on this later) Propagate Info from Leaves to Root of Search Tree, then Choose Best Move –Your Turn: choose MAX score –Opponent’s Turn: choose MIN score (which is best for opponent) Only Choose NEXT Move; When Your Turn Again, Repeat –opponent might not have done what you expected 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 5 2

Tic-Tac-Toe ( 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 5 3

What If Game More Complex? Chess: approx 35 moves per term and 50 moves/player/game, so O ( ) possible board configurations! Better estimate: en.wikipedia.org/wiki/Shannon_number en.wikipedia.org/wiki/Shannon_number Number of atoms in the Earth: Solution: project ahead as far as possible, then use a ‘static board evaluator’ (SBE) to score in-progress board configurations 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 5 4 (Common) Abuse of big-O notation

SBE’s No Perfect SBE Exists for Chess (otherwise we’d know if black can always win, etc) But a lot of Domain K can be put in the SBE You Might Have Learned “Piece Values” in Chess SBE = 1  (#myPawns - #theirPawns) + … + 10  (#myQueens - #theirQueens) + points for ‘center of board’ control + etc Can Run Mini-Max with SBE 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 5 5

Saving Unnecessary Calculations:  -  Pruning Can Prune Away Subtrees by Realizing no Matter their Score, they won’t Impact Choice We Won’t Cover the Full  -  Pruning Algorithm, Since its Applicability Across AI is Limited But We’ll Cover some Simple Cases of the Two Types of Pruning Ask: Would it Matter if this Node had Value  or Value -  ? If Decision Same in Both Cases, No Need to Compute Node’s SBE Value 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 5 6

An  -Cutoff Example (view in Slide Show mode to see animation) 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 5 7 Us (max) Them (min) big subtree ?  2 2  1 Tip: Always fully compute the left-most subtree of root If good for Us, Opponent will not go here. If bad for Us, we will take left branch from root. move

A  -Cutoff Example (view in Slide Show mode to see animation) 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 5 8 Us (max) Them (min) big subtree ?  7 7  9 Recall: Always compute the left-most subtree of root If good for Us, Opponent will take left branch from root. If bad for Us, we will take left branch. move

Note We project ahead as far as we can in the time we have, but only choose our next move Next time it is our turn, we repeat  -  calc (possibly repeating some work, but too much to store) 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 5 9

Performance Improvement Best case:  -  can double search depth considered is fixed amount of time (doubling depth exponentially increases boards considered!) Worst case: no savings Note:  -  and mini-max return SAME answer (  -  simply avoids wasted work) 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 5 10

Some Built-in Assumptions Projecting Ahead k Moves will Lead to Better SBE Estimates than Simply Applying SBE to the Legal Next Boards –seems reasonable, but isn’t guaranteed Our Opponent Thinks the Same Way We Do –no consideration of ‘traps’ and tricking opponent 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 5 11

Horizon Effect Defn: Since We Usually Stop before Game Ends, ‘Danger’ might be Lurking just Beyond our Search Depth One (Partial) Soln: Once Best Move Chosen, Look Ahead a Few Moves More if turns out bad, spend more time searching this turn 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 5 12

Chess: The ‘E. coli’ of AI? Chess Originally Thought to be a Great Testbed for Advancing AI Has had Moderate Impact on AI Progress Not Much ‘Technology Transfer’ to AI Tasks Played Minor Role in ‘ML Revolution’ 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 5 13

ML and Games ML Led to World-Class Backgammon Player Decades Ago ML Produced World-Class Poker Player Over Last Few Years Had Less Success at Chess and Go, but Recent Promise Shown with ‘Deep [Neural] Networks’ Deep Networks Recently Learned to Play Well without being given Games’ Rules 10/6/15CS Fall 2015 (Shavlik©), Lecture 12, Week 5 14