1. CS236501 Introduction to AI Tutorial 5 Automatic Generation of Heuristic Functions Tutorial 5 Automatic Generation of Heuristic Functions

2. Problem Definition Most search algorithms need a heuristic function to guide them in the search space Writing a good heuristic function is hard: –Extensive knowledge about the problem is needed –The distance to the goal may be hard to estimate (especially when variables have nominal values) –It is hard to measure how informative a heuristic function is –We mostly want admissible heuristics We sometimes want a heuristic function for an unknown problem, or a problem that changes with time 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano2

3. Proposed Solution Automatically generate a heuristic function –Should be a problem and domain independent method –Can be state specific –Should produce informative, preferably admissible heuristics –Should be fast in practice and preferably have low theoretical complexity Such heuristics generally use a simplified version of the original problem, the (optimal) solution to which serves as the heuristic value for the original states Most ideas come from the field of automated planning 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano3

4. A useful formal definition of search problems A search problem P= can be defined in the following terms: –S – The search space. S can be defined as a set of variables V={v 1,v 2,…,v n } with value domains D={d 1,d 2,…,d n } such that each variable v i has an associated domain d i, and a state s ∊ S is an element of the space defined by d 1 X d 2 X … X d n (alternatively this can be viewed as a complete assignment for the variables, i.e. s ∊ S={ } i=1..n ). –I – The initial state. I is an element of S. –G – The goal state. G is a partial element of S, i.e. G ⊆ { } i=1..n. When a state s ∊ S contains G, we say that s is in a goal state. 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano4

5. A useful formal definition of search problems –A – The set of allowed actions (or operators). This is a function from S to S. An action } i=1..n,eff ⊆ { } i=1..n > ∊ A takes as input a state s ∊ S, and if it meets the preconditions pre (i.e. the variables that pre specifies have the values that pre specifies), then the effect eff takes place (i.e. the variables that eff specifies are set to the values that eff specifies for them). Variables not specified in pre are “don’t cares” and the values of variables not in eff are copied from the input state to the output state. –The set A*(I) is the feasible state space. –Note: Each action a ∊ A is a symbolic manipulation on the values of the variables of a state s ∊ S. 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano5

6. Relaxation We will consider problem relaxation –A relaxation P’ of problem P is achieved by deleting some constraints present in P –Other methods include: Over constraining, Analogy, and Probabilistic models Example: P = 8-Puzzle –Goal: Order tiles –Action: Swap(x, y) when tile x is H/V-adjacent to tile y, and either x is empty or y is empty 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano6 123 456 78

7. 8-Puzzle – The original problem The optimal solution of the original (non-relaxed) problem is: 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano7 h*=9 24 153 786

8. 8-Puzzle – Relaxation I Tiles can move to any location (whether occupied by another tile or not) –This is the number of misplaced tiles heuristic 17-Dec-15Intro. to AI – Tutorial 7 – By Haggai Toledano8 2 153 786 h=5 4

9. 8-Puzzle – Relaxation II Tiles can move to any adjacent location (whether occupied by another tile or not) –Summing over all tiles, this is Manhattan Distance 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano9 h=7 2 153 786 4

10. 8-Puzzle – Relaxation III Tiles can move to the empty location (even if it is not adjacent to them) –Optimal solution: Sort (place a misplaced target tile in the empty place; prefer the target of that place) 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano10 h=5 2 153 786 4

11. Relaxations – Usage Receive a problem P= Create P’=, the relaxed version of P Run your favorite heuristic algorithm on P –For each s ∊ S, starting with I, get the successors succ s of s by applying all applicable actions from A. –For each t ∊ succ s compute the optimal solution of P’ from t’ to G’. The solution value (length, cost etc.) will be denoted by h(t). –Use h(succ s ) as the heuristic function for the search algorithm. Since P’ is only a symbolic manipulation of P, it is problem and domain independent 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano11

12. h + Relaxation A general method for creating a certain type of relaxations: –State variables accumulate values instead of merely changing their value – never forget achieved values Simplifies the original problem – less constraints: –Allows “jumps” between previously unlinked states –In effect: Adds edges to the states transition graph –Preserves preconditions –Computation: Uninformed, complete search, e.g. BFS Monotonic: The original actions are still applicable to the states they were applicable to beforehand Admissible 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano12

13. 8-Puzzle – h + 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano13 X24 153 786 2X 4 153 786 24X2X4X 153 786 24X12X4X 1X53 786 24X12X34X 1X53X 786 24X 124 X 34X 1X53X 786 0 3 12 54 4

14. 8-Puzzle – h + 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano14 24X 124 X 34X 1X536X 786X 24X 124 X 34X 14X536X 786X 24X 124 X 34X 14X14X536X36X 786X6X h + = 7 h* = 9 67 4

15. h + Relaxation – Other problems Lightz Out 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano15.. On Off On, Off.. On Off P P+P+ Goal!

16. h + Relaxation – Other problems City Map –I = –succ(I) = {, } –succ + (I)= {, } –succ + ( ) = ) 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano16 Boston QuebecNew York Boston Quebec New York

17. h + Relaxation – Other problems Missionaries and Cannibals –State: ∊ NxNxNxNx{right,left}, where N={0,1,2,3} –I = ; G = –A = {CCR = [move 2 cannibals from L to R] = <pre={C L ≥ 2,B=left,M R ≥ C R +2}, eff={C L =C L -2,C R =C R +2,B=right}>, MMR, CML, … } –CCR(I= ) = –CCR + = <pre={C L ≥ 2,B=left,M R ≥ C R +2}, eff={C L =C L U C L -2,C R =C R U C R +2,B=B U right}> –CCR + (I + = )= 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano17

18. h + Relaxation: Complexity Although the + -relaxed problem seems simpler to solve, its complexity might still be too high Proof (Bylander ’94): –Propositional planning is a type of search problem –PLANMIN (k) is the decision problem of determining whether an instance of a propositional planning problem is satisfiable in k or less actions –PLANMIN (k) + is PLANMIN (k) restricted to positive postconditions (can only add facts, can’t delete them) –PLANMIN (k) is PSPACE complete –PLANMIN (k) + is NP complete (reduction from 3SAT ) Solution: Approximate h + 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano18

19. h + Approximation I: h max h max – Find the longest path to any goal value for the relaxed problem –Build a layered planning graph: Start from the initial state At each layer apply all applicable actions –The first layer to hold all of the goal variables in their goal values is the approximated solution length Properties: –Not very informative (too optimistic; no mutex detection) –Admissible 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano19

20. 8-Puzzle – h max 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano20 X24 153 786 245 X 2X4X 15X3 786 245 X 124 5X 234 5X 1X 123 458 X 3X 78X6 123 458 X 124 5X 123 45X 123 457 8X 123 458 X 123 456 8X 7X 123 456 78X 6X 123 458 X 124 5X 123 45X 123 457 8X 123 458 X 123 456 8X 7X7X 123 456 78X 6X6X 012 3 h + = 7 h* = 9 h max = 3 h max = max(2(1), 1(2), 2(3), 3(4), 0(5), 3(6), 0(7), 0(8)) = 3

21. h + Approximation II: h add h add – Sum over the costs to achieve all goal variables values Compute just like h max but sum the costs instead of finding their maximum Assumes total independence of goal variables Properties: –Informative in problems where the goal variables have little interaction with one another –Not admissible – often over estimates 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano21

22. 8-Puzzle – h add 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano22 X24 153 786 245 X 2X4X 15X3 786 012 3 h + = 7 h* = 9 h max = 3 h add = 11 h add = 2(1)+1(2)+2(3)+ 3(4)+0(5)+3(6)+ 0(7)+0(8) = 11 245 X 124 5X 234 5X 1X 123 458 X 3X 78X6 123 458 X 124 5X 123 45X 123 457 8X 123 458 X 123 456 8X 7X 123 456 78X 6X 123 458 X 124 5X 123 45X 123 457 8X 123 458 X 123 456 8X 7X7X 123 456 78X 6X6X

23. h + Approximation III: h FF h FF – Fast Forward: The length of a non-optimal solution to the relaxed problem: –Build the planning graph –Backchain some solution (i.e. necessary actions) from the last (goal) layer to the first (initial state): For each goal in layer i, set its preconditions as goals for layer i-1 Select actions that were applied on layer i-1 and contain the goals from layer i as their combined postconditions Repeat until reaching the initial graph layer –The solution’s length (i.e. the sum of the number of uniquely selected actions for each layer) is the heuristic value for the initial state 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano23

24. h + Approximation III: h FF Properties: –Takes into account positive interactions –Not admissible –Rarely over estimates –Uses the fact that while PLANMIN (k) + is NPC, PLANSAT + (relaxed satisfiablity) is in P (so if the goal cannot be reached – h + is unsolvable; if the goal is reached – h + is solvable and a witness can be constructed) –Note that solving for the minimal number of actions in each layer is NPC (reduction from SC) The original FF algorithm uses enforced hill-climbing (use BFS to escape local minima) 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano24

25. h + and its approximations Problem: –Variables: x ∊ {x 1,x 2,x 3 }, y ∊ {y 1,y 2,y 3 }, z ∊ {z 1,z 2 } –Actions ( ): a 1 =, a 2 =, a 3 =, a 4 =, a 5 =, a 6 = –Initial state: {x=x 1, y=y 1, z=z 1 } –Goal state: {x=x 3, y=y 3, z=z 2 } 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano25 x1x1 y1y1 z1z1 x1x2x1x2 x 1,x 2 y1y1 z 1,z 2 x 1,x 2,x 3 y 1,y 2 z 1,z 2 y2y3y2y3 x 1,x 2,x 3 y 1,y 2,y 3 z 1,z 2 z1z2z1z2 x2x3x2x3 x2y2x2y2 h max =max{2(x),3(y),1(z)}=3 h add =sum{2(x),3(y),1(z)}=6 h FF =2(a 1,a 6 )+2(a 3,a 4 )+1(a 5 )=5 h + =1(a 1 )+1(a 2 )+1(a 5 )+1(a 6 )=4 x 2  x 3,y 2 = h*

26. 8-Puzzle – h FF 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano26 X24 153 786 245 X 2X4X 15X3 786 012 3 h + = 7 h* = 9 h max = 3 h add = 11 h FF = 9 h FF = by a certain choice of actions = 9 245 X 124 5X 234 5X 1X 123 458 X 3X 78X6 123 458 X 124 5X 123 45X 123 457 8X 123 458 X 123 456 8X 7X 123 456 78X 6X 123 458 X 124 5X 123 45X 123 457 8X 123 458 X 123 456 8X 7X7X 123 456 78X 6X6X

27. 8-Puzzle – h FF backchaining 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano27 1(1) 2(2) 3(3) 4(4) 5(5) 6(6) 7(7) 8(8) 1(1) 2(2) 3(3) 4(1) X(4) 5(5) 6(9) X(6) 7(7) 8(8) 1(4) X(1) 2(2) 3(6) X(3) 4(2) X(1) X(5) 5(5) 6(9) 7(7) 8(8) 1(4) 2(1) X(2) 4(3) 5(5) 6(9) 3(6) 7(7) 8(8) Unique non-noop actions: 3+4+2  h FF = 9 bad good noop action 1(4) X(1) 2(2) 3(6) X(3) 4(2) X(1) X(5) 5(5) 6(9) 7(7) 8(8)

28. Pattern Database Relaxation The Pattern Database (PDB) relaxation method maps a problem P to a relaxed problem P’ by ignoring some aspects of the states –Option I: Variables deletion: Use only a subset of the variables from P (but take care to always include a goal variable in this subset) –Option II: Values domain abstraction: Use only a subset of the domain for some variables PDB only clusters states that share some properties (rather than arbitrary states) We now have a set of patterns of states –The new state(+actions) space is homomorphic to the original state space 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano28

29. Pattern Database Relaxation 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano29 Use this pattern

30. Pattern Database Relaxation How to solve using the state patterns: –For each state, optimally solve the abstracted problem and use the solution length as the heuristic value of the original state –Construct a database of solutions for all abstracted states in advance (rather than on the fly) This method guarantees admissibility: All of the abstracted solutions are shorter than the optimal solution of the original problem 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano30

31. Pattern Database Relaxation 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano31 Use this pattern h* = 3 h = 2

32. 8-Puzzle – PDB Relaxation 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano32 24 153 786 placetile 41 12 63 34 55 96 77 88 placetile 41 12 34 96 77 88 24 1 786 21 46 78 Goal h = 6 placetile 41 12 63 34 55 96 77 88 123 456 789 123 456 789 123 456 789 Example action:  = eff Ø 

33. Pattern Database Relaxation Representation is important for understanding the patterned state space –In the last example, the representation was: “Where is tile X located”. Eliminating tiles 3 and 5 adds two extra empty places on the board –Using the representation “What tile does place X hold” deletes places, creating strange actions like “place y holds x ↔ place y holds empty”, allowing multiple instances of a tile and allowing the empty board – all of these do not impact admissibility –However “What tile does place X hold” is a more intuitive representation for values domain abstraction: We could place the empty tile on several placeholders simultaneously 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano33

34. placetile 11 22 33 Pattern Database Relaxation PDB is not necessarily surjective: The new state space often contains states that have no mapping from the original state space (may affect search speed but not admissibility) Example: 3-Puzzle 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano34 This state has no source in the original state space placetile 11 22 33 1 3 2 12 34 1 2 12 34 2 1 12 34

35. Pattern Database Relaxation Very important subject (which we will not discuss): How to choose which variables to delete –Always keep at least one goal variable –Possible strategy: Try to decompose the problem into as many independent subproblems as possible while retaining as many variables as possible 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano35

36. Summary The solution to a relaxed problem can be used as a heuristic for the original problem It is preferable to use an admissible heuristic when possible Some (crucial) aspects of the automated process of heuristic functions generation still need to be taken care of manually 17-Dec-15Intro. to AI – Tutorial 5 – By Haggai Toledano36