CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6 7/6/2018 Today’s Topics Playing Deterministic (no Dice, etc) Games Mini-max - pruning ML and games? 1997: Computer Chess Player (IBM’s Deep Blue) Beat Human Chess Champ (Kasparov) 2015: World-Class Computer GO Player 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6 Mini-Max Algorithm Generate Search Tree of ALL Possible Games Score Leaves (Assume Higher Better for ‘You’) eg, +1 you win, -1 you lose, 0 if a draw (ie, a tie) Assume Opponent Scores ‘Board Configurations’ the Same Way You Do (more on this later) Propagate Info from Leaves to Root of Search Tree, then Choose Best Move Your Turn: choose MAX score Opponent’s Turn: choose MIN score (which is best for opponent) Only Choose NEXT Move; When Your Turn Again, Repeat opponent might not have done what you expected 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

Tic-Tac-Toe (https://en.wikipedia.org/wiki/Tic-tac-toe) 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

What If Game More Complex? CS 540 Fall 2015 (Shavlik) 7/6/2018 What If Game More Complex? Chess: approx 35 moves per turn and 50 moves/player/game, so O(35100) possible board configurations! Better estimate: 1047 en.wikipedia.org/wiki/Shannon_number Number of atoms in the Earth: 1050 www.fnal.gov/pub/science/inquiring/questions/atoms.html Solution: project ahead as far as possible, then use a ‘static board evaluator’ (SBE) to score in-progress board configurations (Common) Abuse of big-O notation 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6 SBE’s No Perfect SBE Exists for Chess (otherwise we’d know if black can always win, etc) But a lot of Domain K can be Put in the SBE You Might Have Learned “Piece Values” in Chess SBE = 1  (#myPawns - #theirPawns) + … + 10  (#myQueens - #theirQueens) + points for ‘center of board’ control + etc Can Run Mini-Max with SBE 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

Saving Unnecessary Calculations: - Pruning Can Prune Away Subtrees by Realizing no Matter their Score, they won’t Impact Choice We Won’t Cover the Full - Pruning Algorithm, Since its Applicability Across AI is Limited But We’ll Cover some Simple Cases of the Two Types of Pruning Ask: Would it Matter if this Node had Value  or Value -? If Decision Same in Both Cases, No Need to Compute Node’s SBE Value 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

An -Cutoff Example (view in Slide Show mode to see animation) move Tip: Always fully compute the left-most subtree of root  2 Us (max) 2  1 Them (min) 2 7 1 big subtree ? If good for Us, Opponent will not go here. If bad for Us, we will take left branch from root. 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

A -Cutoff Example (view in Slide Show mode to see animation) move Recall: Always compute the left-most subtree of root  7 Them (min) 7  9 Us (max) 2 7 9 big subtree ? If good for Us, Opponent will take left branch from root. If bad for Us, we will take left branch. 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6 Note We project ahead as far as we can in the time we have, but only choose our next move Next time it is our turn, we repeat - calc (possibly repeating some work, but too much to store) 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

Performance Improvement Best case: - can double search depth considered is fixed amount of time (doubling depth exponentially increases boards considered!) Worst case: no savings Note: - and mini-max return SAME answer (- simply avoids wasted work) 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

Some Built-in Assumptions Projecting Ahead k Moves will Lead to Better SBE Estimates than Simply Applying SBE to the Legal Next Boards seems reasonable, but isn’t guaranteed Our Opponent Thinks the Same Way We Do no consideration of ‘traps’ and tricking opponent 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6 Horizon Effect Defn: Since We Usually Stop before Game Ends, ‘Danger’ might be Lurking just Beyond our Search Depth One (Partial) Soln: Once Best Move Chosen, Look Ahead a Few Moves More if turns out bad, spend more time searching this turn 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

Perils of Not-Looking Ahead 10/11/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

Chess: The ‘E. coli’ of AI? Chess Originally Thought to be a Great Testbed for Advancing AI Has had Moderate Impact on AI Progress Not Much ‘Technology Transfer’ to AI Tasks Played Minor Role in ‘ML Revolution’ 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6

CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6 ML and Games ML Led to World-Class Backgammon Player Decades Ago ML Produced World-Class Poker Player over Last Few Years World-class GO Player via ‘Deep [Neural] Networks’ (2015) Deep Networks Recently Learned to Play Well without being given Games’ Rules 10/13/16 CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6