1. 1 Heuristic Programming CIS 487/587 Bruce R. Maxim UM-Dearborn

2. 2 State Space Search Many AI problems can be conceptualized as searching a well defined set of related problem states –Puzzle solving –Game playing –Generate and test –Genetic algorithms –Scheduling algorithms

3. 3 Water Jug Problem Problem –You have a four gallon jug and a three gallon jug and the goal is to come up with exactly two gallons of water. Operations –Dump bucket contents on the ground –Dump bucket contents in other bucket –Fill bucket to top with water

4. 4 Tree Representation (0,0) (4,0) (0,3) (0,3) (0,0) (4,3) (1,3) (3,0) (4,3) (0,0)

5. 5 Digraph Representation (0,0) (4,0) (0,3) (1,3) (4,3) (3,0)

6. 6 Adjacency List (0 0)  ((4 0) (0 3)) (4 0)  (( 4 3) (0 0) (1 3) (0 3)) (0 3)  ((4 3) (3 0) (0 0)) (1 3)  ((1 0) (4 3) (4 0) (0 3))

7. 7 Good Knowledge Representations Important things made clear /explicit Expose natural constraints Must be complete Are concise Transparent (easily understood) Information can be retrieved & stored quickly Detail suppressed (can be found as needed) Computable using existing procedures

8. 8 River Puzzle Problem –There are four items a farmer, wolf, goose, and corn. The farmer can only take one item across the river at a time. Constraints –Wolf will eat the goose if left alone with it –Goose will eat the corn if left alone with it

9. 9 Problem Analysis Each of the 4 problem elements can be considered a Boolean variable based on its bank location There are 2 4 or 16 possible states 6 of these states fail the “no eat” test That leaves 10 states to consider which could be linked in C(10,2) = 45 ways However only 10 of these links can really be used to connect states

10. 10 F=Farmer W=Wolf G=Goose C=Corn ~=River W F ~ W F G F W F G ~ G F ~ W C W C C ~ G F G ~ C F ~ W C F ~ C F W W G ~ G G ~ G C C C F C W ~ G W

11. 11 Solution Once graph is constructed finding solution is easy (simply find a path) AI programs would rarely construct the entire graph explicitly before searching AI programs would generate nodes as needed and restrict the path search to the nodes generated May use forward reasoning (initial to goal) or backward reasoning (goal to initial)

12. 12 Rules Many time these state transitions are written as rules: If ((Wolf Corn) (Farmer Goose)) Then ((Wolf Corn Farmer) (Goose)) Depending on which direction you are reasoning you match either the left (if) or right (then) part

13. 13 Potential Problems If you have more than one state to move to then you need a procedure to choose one Exact matches often do not occur in the real world so you may need to measure how “close” you are to an exact match

14. 14 Control Strategies A good control strategy causes motion (ideally toward the goal state) The control strategy should be systematic and not allow repeated use of the same rule sequences (to avoid infinite loops)

15. 15 Heuristics AI programming often relies on the use of heuristics Heuristics are techniques that improves efficiency by trading “speed” for “completeness”

16. 16 Search Considerations Can the search algorithm work for the problem space? Is the search algorithm efficient? Is it easy to implement? Is search the best approach or is more knowledge better?

17. 17 Example Graph S A B C F

18. 18 Adjacency List (setf (get 's 'children) '(a b)) (setf (get 'a 'children) '(s b f)) (setf (get 'b 'children) '(s a c)) (setf (get 'c 'children) '(b f)) (setf (get 'f 'children) '(a c))

19. 19 “Any Path” Search Algorithms Depth First Search Hill Climbing Best First Search Breadth First Search Beam Search

20. 20 Depth First Search Add root to queue of partial paths Until queue is empty or goal is attained If first queue element equals the goal then do nothing Else remove the first queue element add its children to the front of the queue of the partial paths If goal is attained then announce success

21. 21 Depth First Output > (depth 's 'f) ((s)) ((a s) (b s)) ((b a s) (f a s) (b s)) ((c b a s) (f a s) (b s)) ((f c b a s) (f a s) (b s)) (s a b c f)

22. 22 Depth First Weakness Depth first is not good for “tall” trees when it is possible to over commit to a “bad” path early In our example depth first missed a complete solution because it focused on checking the first partial path and did not test the others in the queue

23. 23 Improving Search We will need to add knowledge to our algorithms to make them perform better We will need to add some distance estimates, even using “best guesses” would help We will need to begin sorting part of the the queue of partial paths

24. 24 Example Graph S A B C F 2 1 1 0 2

25. 25 British Museum Search If enough monkeys had enough type writers and enough time then they could recreate all the knowledge housed in the British Museum So we could compute every path by trial and error then pick the shortest

26. 26 Branch and Bound An optimal depth first search Requires the ability to measure the distance traveled “so far” using an ordinal scale (does not require exact distances) Entire queue of partial path is sorted after it is modified

27. 27 Branch and Bound Add root to queue of partial paths Until queue is empty or goal is attained If first queue element equals the goal then do nothing Else remove the first queue element add its children to the front of the queue of the partial paths sort the queue of partial paths by distance traveled If goal is attained then announce success

28. 28 Branch and Bound Output > (branch 's 'f) ((s)) ((a s) (b s)) ((b s) (b a s) (f a s)) ((a b s) (c b s) (b a s) (f a s)) ((c b s) (b a s) (f a s) (f a b s)) ((b a s) (f a s) (f c b s) (f a b s)) ((f a s) (c b a s) (f c b s) (f a b s)) (s a f)

29. 29 A* Makes use of both a cost measure and remaining distance underestimator Removes redundant paths from the queue of partial paths

30. 30 A* Add root to queue of partial paths Until queue is empty or goal is attained If first queue element equals the goal then do nothing Else remove the first queue element add its children to the front of the queue of the partial paths sort the queue of partial paths by distance traveled plus the estimate of distance to goal remove redundant paths If goal is attained then announce success

31. 31 A* > (a* 's 'f) ((s)) ((a s) (b s)) ((f a s) (b s)) (s a f)

32. 32 A* Weaknesses Still requires the use of a “natural” distance measure Underestimator is a guess as best Sorting takes time Removing paths takes time

33. 33 Generate and Test Search can be viewed generate and test procedures Testing for a complete path is performed after varying amount of work has been done by the generator At one extreme the generator generates a complete path which is evaluated At the other extreme each move is tested by the evaluator as it is proposed by the generator

34. 34 Improving Search-Based Problem Solving Two options 1.Improve “generator” to only generate good moves or paths 2.Improve “tester” so that good moves recognized early and explored first

35. 35 Using Generate and Test Can be used to solve identification problems in small search spaces Can be thought of as being a depth-first search process with backtracking allowed Dendral – expert system for identifying chemical compounds from NMR spectra

36. 36 Dangers Consider a safe cracker trying to use generate a test to crack a safe with a 3 number combination (00-00-00) There are 100 3 possible combinations At 3 attempts/minute it would take 16 weeks of 24/7 work to try each combination in a systematic manner

37. 37 Generator Properties Complete –capable of producing all possible solutions Non-redundant –don’t propose same solution twice Informed –make use of constraints to limit solutions being proposed

38. 38 Dealing with Adversaries Games have fascinated computer scientists for many years Babbage –playing chess on Analytic Engine –designed Tic-Tac-Toe machine Shanon (1950) and Turing (1953) –described chess playing algorithms Samuels (1960) –Built first significant game playing program (checkers)

39. 39 Why games attracted interest of computer scientists? Seemed to be a good domain for work on machine intelligence, because they were thought to: –provide a source of a good structured task in which success or failure is easy to measure –not require much knowledge (this was later found to be untrue)

40. 40 Chess Average branching factor for each position is 35 Each player makes 50 moves in an average game A complete game has 35 100 potential positions to consider Straight forward search of this space would not terminate during either players lifetime

41. 41 Games Can’t simply use search like in “puzzle” solving since you have an opponent Need to have both a good generator and an effective tester Heuristic knowledge will also be helpful to both the generator and tester

42. 42 Ply Some writers use the term “ply” to mean a single move by either player Some insists “ply” is made up of a move and a response I will use the first definition, so “ply” is the same as the “depth - 1” of the decision tree rooted at the current game state

43. 43 Static Evaluation Function Used by the “tester” Similar to “closerp” from our heuristic search work in A* type algorithms In general it will only be applied to the “leaf” node of the game tree

44. 44 Static Evaluation Functions Turing (Chess) sum of white values / sum of black values Samuels (Checkers) linear combination with interaction terms piece advantage capability for advancement control of center threat of fork mobility

45. 45 Role of Learning Initially Samuels did not know how to assign the weights to each term of his static evaluation function Through self-play the weights were adjusted to match the winner’s values c1 * piece advan + c2 * advanc + …

46. 46 Tic Tac Toe

47. 47 Tic Tac Toe 100A + 10B + C – (100D + 10E + F) A = number of lines with 3x’s B = number of lines with 2 x’s C = number of lines with single x D = number of lines with 3 O’s E = number of lines with 2 O’s F = number of lines with a single O

48. 48 Example X X O O X A = 0 B = 0 C = 1 D = 0 E = 1 F = 1 100 (0) + 10(0) + 1 – (100 (0) + 10(1) + 1) = 1 – 11 = -10

49. 49 Weakness All static evaluation functions suffer from two weaknesses –information loss as complete state information mapped to a single number –Minsky’s Credit Assignment problem it is extremely difficult to determine which move in a particular sequence of moves caused a player to win or loss a game (or how much credit to assign to each for end result)

50. 50 What do we need for games? Plausible move generator Good static evaluation functions Some type of search that takes opponent behavior into account for nontrivial games

51. 51 1-ply Minimax If the static evaluation is applied to the leaf nodes we get B = 8 C = 3 D = -2 So best move appears to be B A CBD

52. 52 2-ply Minimax Applying the static evaluation function E = 9 F = -6 G = 0 H = 0 I = -2 J = -4 K = -3 A BCD EFGIHJK

53. 53 Propagating the Values Will depend on the level Assuming that the “minimizer” chooses from the leaf nodes, be would get B = min(9, -6, 0) = -6 C = min(0, -2) = -2 D = min(-4, -3) = -4 The the “maximizer” gets to choose from the minimizers values and selects move C A = max(-6, -2, -4)

54. 54 Minimax Algorithm If (limit of search reached) then compute static value of current position return the result Else If (level is minimizing level) then use Minimax on children of current position report minimum of children’s results Else use Minimax on children of current position report maximum of children’s results

55. 55 Search Limit Has someone won the game? Number of ply explored so far How promising is this path? How much time is left? How stable is this configuration?

56. 56 Criticism of Minimax Goodness of current position translated to a single number without knowing how the number was forced on us Suffers from “horizon effect” –a win or loss might be in the next ply and we would not know it

57. 57 Minimax with Alpha-Beta Pruning Alpha cut-off –whenever a min node descendant receives a value less than the “alpha” known to the min node’s parent, which will be a max node, the final value of min. node can be set to beta Beta cut-off –whenever a max node descendant receives a value greater than “beta” known to the max nodes parent (a min node), the final value of max node can be set to “alpha”

58. 58 Alpha-Beta Assumptions Alpha value initially set to -  and never decreases Beta value initially set to -  and never decreases Alpha value is always current largest backed up value found by any node successor Beta value is always current smallest backed up value found by any node successor

59. 59 Alpha-Beta Pruning

60. 60 Alpha-Beta With perfect ordering more static evaluations are skipped Even without perfect ordering many evaluations can be skipped If worst paths are explored first no cutoffs will occur With perfect ordering alpha-beta lets you exam twice the number of ply that minimax without alpha-beta can examine in the same amount of time

61. 61 Alpha-Beta Algorithm Function Value (P, ,  ) // P is the position in the data structure { // determine successors of P and call them // P(1), P(2),... P(d) if d=0 then return f(p) // call static evaluation function // return as value to parent

62. 62 Alpha-Beta Algorithm else { m =  for i =1 to d do { t = - value (Pi - , - m) if t > m then m = t if =>  then exit loop } return m }

63. 63 Horizon Heuristics Progressive deepening –3 ply search followed by 4 ply, followed by 5 ply, etc. until time runs out Heuristic pruning –order moves based on plausibility and eliminate unlikely possibilities –does not come with “minimax” guarantee Heuristic continuation –extend promising or volatile paths 1 or 2 more steps before committing to choice

64. 64 Horizon Heuristics Futility cut-off –stop exploring when improvements are marginal –does not come with “minimax” guarantee Secondary search –once you pick a path using a 6 ply search continue from leaf node with a 3 ply search to confirm pick Book moves –eliminates search in specialized situations –does not come with “minimax” guarantee