1. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 1 Algorithms & Complexity 5 Games Mark D Dunlop Mark.Dunlop@cis.strath.ac.uk 14.6

2. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 2 How computers play games Chess, draughts and naughts and crosses We'll do naughts and crosses –its easiest

3. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 3 Naughts and crosses Three X or thee O in a row to win Can guarantee not to lose Can grab a win when available

4. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 4 Minimax strategy Use an evaluation strategy to quantify the goodness of a position Terminal cases: A win for the computer gives +1, a draw 0, a loss -1 Non-terminal: calculated by recursively assuming best play by both computer and human Minimax - human tries to minimise the computer goodness function, computer tries to maximise it and we take turns

5. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 5 Basic Strategy "If I (computer) move there, then my opponent will probably move there, then I can move there,..., and win!"

6. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 6 Next position The successor position of P is any position P’ reachable from P in one move –Computer move: look at all next positions and calculate their score (recursively) pick next move with maximum score –Guess of human move: look at all next positions and calculate their score assume human plays move with minimum score

7. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 7 Some Psuedo Code public moveInfo findComputerMove() //computer plays X, human O if board is full return new moveInfo(DRAW) else if can_win_in_one_move return new moveInfo(winmove,WIN) else maxvalue = LOSE for i=1..9 if square i is empty place (i, 'X') tryvalue = findHumanMove().value emptycell (i) if tryvalue > maxvalue maxvalue = tryvalue; bestmove = i return new moveInfo(bestmove, maxvalue) findHumanMove is very similar

8. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 8 Problems All this says is –if I (computer) move there then I can win if user plays best as I expect –If I move there the best I can do is draw –If I move there I can only lose Better to return some function of the probability of winning, i.e. –if I move here then for 1/6 human moves I can win or –If I move here then for 5/6 human moves I can win

9. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 9 Complexity Main loop cycles round 9 squares For each square - calculate all possible future values from that move, e.g first move –computer has 9 choices –human then has 8 –computer then has 7 –human then has 6 –computer then has 5 and might win or not, etc... 9x8x7x6x5x4x3x2x1 = 362 880 combinations Of which 97 162 are possible (stop after a win)

10. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 10 Complexity If the computer moves first it has 97 162 positions to evaluate If the human then picks the centre square, the computers next turn has 5 185 evaluations (9 761 for a corner, 13 233 for a side) For chess it is estimated there are 10^100 positions to be examined to decide the best first move Standard openings help but in general, this is still too bad...

11. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 11 Complexity reducers Only look so far ahead –requires some function to evaluate the strength of a current board –in chess these functions can be very complex –still the ply or number of levels look ahead is still the big performance factor

12. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 12 Complexity reducers Remember already calculated positions –Use a transposition table - almost always a hash table –When you calculate a position you store it in the table –Next time you face a position, see if it is there and just read off the value (note: there are many routes to one position) Only look ahead where it looks good

13. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 13 Sample game tree Note: no tree is actually created - this is the tree of recursive calls, or choices we can make.

14. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 14 Alpha-Beta Pruning Another algorithmic design pattern We can ignore large parts of the tree because we can't do better than already found, e.g.

15. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 15 How to beat a chess program Most chess programs (and real games) have a time limit per move The programs optimise their "thinking" using alpha-beta pruning to maximise the depth in places where they are likely to be asked to play, i.e. they assume the human plays well

16. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 16 How to beat a chess program If you play very badly for one move, the computer will run out of time on its next shot since it will not have calculated that whole chunk of the game tree Grand masters didn't take long to work that one out...

17. Mark Dunlop, Computer and Information Sciences, Strathclyde University http://www.cis.strath.ac.uk/~mdd/ 17 Summary MiniMax strategy for making best choice for computer, and best from human too Look ahead level, ply, & evaluation function are both important ply usually dominates Transposition tables remember positions ALPHA-BETA pruning reduce number of nodes to calculate based on best/worst already seen You throw a chess program by playing badly (once...)