1. Origin of Heuristic Functions
Chapter 6
Origin of Heuristic Functions

2. Heuristic from Relaxed Models
A heuristic function returns the exact cost of reaching a goal in a simplified or relaxed version of the original problem.
This means that we remove some of the constraints of the problem.
Removing constraints = adding edges

3. Example: Road navigation
A good heuristic – the straight line.
We remove the constraint that we have to move along the roads
We are allowed to move in a straight line between two points.
We get a relaxation of the original problem.
In fact, we added edges of the complete graph

4. Example 2: - TSP problem
We can describe the problem as a graph with 3 constraints:
1) Our tour covers all the cities.
2) Every node has a degree two
an edge entering the node and
an edge leaving the node.
3) The graph is connected.
If we remove constraint 2 :
We get a spanning graph and the optimal solution to this problem is a MST (Minimum Spanning Tree).
If we remove constraint 3:
Now the graph isn’t connected and the optimal solution to this problem is the solution to the assignment problem.

5. Example 3- Tile Puzzle problem
One of the constraints in this problem is that a tile can only slide into the position occupied by a blank.
If we remove this constraint we allow any tile to be moved horizontally or vertically position.
This is the Manhattan distance to its goal location.

6. The STRIPS Problem formulation
We would like to derive such heuristics automatically.
In order to do that we need a formal description language that is richer than the problem space graph.
One such language is called STRIPS.
In this language we have predicates and operators.
Let’s see a STRIPS representation of the Eight Puzzle Problem

7. STRIPS - Eight Puzzle Example
On(x,y) = tile x is in location y.
Clear(z) = location z is clear.
Adj(y,z) = location y is adjacent to location z.
Move(x,y,z) = move tile x from location y to location z.
In the language we have:
A precondition list - for example to execute move(x,y,z) we
must have: On(x,y)
Clear(z)
Adj(y,z)
An add list - predicates that weren’t true before the operator and now after the operator was executed are true.
A delete list - a subset of the preconditions, that now after the
operator was executed aren’t true anymore.

8. STRIPS - Eight Puzzle Example
Now in order to construct a simplified or relaxed problem we only have to remove some of the preconditions.
For example - by removing Clear(z) we allow tiles to move to adjacent locations.
In general, the hard part is to identify which relaxed problems have the property that their exact solution can be efficiently computed.

9. Admissibility and Consistency
The heuristics that are derived by this method are both admissible and consistent.
Note : The cost of the simplified graph should be as close as possible to the original graph.
Admissibility means that the simplified graph has an equal or lower cost than the lowest - cost path in the original graph.
Consistency means that a heuristic h is consistent for every neighbor n’ of n,
when h(n) is the actual optimal cost of reaching a goal in the graph of the relaxed problem.
h(n)  c(n,n’)+h(n’)

10. Method 2: pattern data base
A different method for abstracting and relaxing to problem to get a simplified problem.
Invented in 1996 by Culberson & Schaeffer

11. optimal path search algorithms
For small graphs: provided explicitly, algorithm such as Dijkstra’s shortest path, Bellman-Ford or Floyd-Warshal. Complexity O(n^2).
For very large graphs , which are implicitly defined, the A* algorithm which is a best-first search algorithm.

12. Best-first search schema
sorts all generated nodes in an OPEN-LIST and chooses the node with the best cost value for expansion.
generate(x): insert x into OPEN_LIST.
expand(x): delete x from OPEN_LIST and generate its children.
BFS depends on its cost (heuristic) function. Different functions cause BFS to expand different nodes.. 30 25 20 30 35 35 35 40
Open-List

13. Best-first search: Cost functions
g(x): Real distance from the initial state to x
h(x): The estimated remained distance from x to the goal state.
Examples:Air distance
Manhattan Dinstance
Different cost combinations of g and h
f(x)=level(x) Breadth-First Search.
f(x)=g(x) Dijkstra’s algorithms.
f(x)=h’(x) Pure Heuristic Search (PHS).
f(x)=g(x)+h’(x) The A* algorithm (1968).

14. A* (and IDA*)
A* is a best-first search algorithm that uses f(n)=g(n)+h(n) as its cost function.
f(x) in A* is an estimation of the shortest path to the goal via x.
A* is admissible, complete and optimally effective. [Pearl 84]
Result: any other optimal search algorithm will expand at least all the nodes expanded by A*
Breadth
First
Search A*

15. Domains 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
15 puzzle
10^13 states
First solved by [Korf 85] with IDA* and Manhattan distance
Takes 53 seconds
24 puzzle
10^24 states
First solved by [Korf 96]
Takes two days 20 21 22 23 24 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

16. Domains Rubik’s cube 10^19 states First solved by [Korf 97]
Takes 2 days to solve

17. (n,k) Pancake Puzzle 1 2 3 4 5 N An array of N tokens (Pancakes)
Operators: Any first k consecutive
tokens can be reversed.
The 17 version has 10^13 states
The 20 version has 10^18 states

18. (n,k) Top Spin Puzzle n tokens arranged in a ring
States: any possible permutation of the tokens
Operators: Any k consecutive tokens can be reversed
The (17,4) version has 10^13 states
The (20,4) version has 10^18 states

19. 4-peg Towers of Hanoi (TOH4)
Harder than the known 3-peg Towers of Hanoi
There is a conjecture about the length of optimal path but it was not proven.
Size 4^k

20. How to improve search? Enhanced algorithms:
Perimeter-search [Delinberg and Nilson 95]
RBFS [Korf 93]
Frontier-search [Korf and Zang 2003]
Breadth-first heuristic search [Zhou and Hansen 04]
 They all try to better explore the search tree.
Better heuristics: more parts of the search tree will be pruned.

21. Better heuristics In the 3rd Millennium we have very large memories.
We can build large tables.
For enhanced algorithms: large open-lists or transposition tables. They store nodes explicitly.
A more intelligent way is to store general knowledge. We can do this with heuristics

22. Subproblems-Abstractions
Many problems can be abstracted into subproblems that must be also solved.
A solution to the subproblem is a lower bound on the entire problem.
Example: Rubik’s cube [Korf 97]
Problem:  3x3x3 Rubik’s cube
Subproblem:  2x2x2 Corner cubies.

23. Pattern Databases heuristics
A pattern database (PDB) is a lookup table that stores solutions to all configurations of the sub-problem (patterns)
This PDB is used as a heuristic during the search
88 Million states
10^19 States
Search
space
Mapping/Projection
Pattern
space

24. Non-additive pattern databases
Fringe pattern database [Culberson & Schaeffer 1996].
Has only 259 Million states.
Improvement of a factor of 100 over Manhattan Distance

25. Example - 15 puzzle 1
4 5 2
13 15
How many moves do we need to move tiles 2,3,6,7 from locations 8,12,13,14 to their goal locations
The solution to this is located in
PDB[8][12][13][14]=18

26. Example - 15 puzzle 2
How many moves do we need to move tiles 2,3,6,7 from locations 8,12,13,14 to their goal locations
The solution to this is located in
PDB[8][12][13][14]=18

27. Disjoint Additive PDBs (DADB)
If you have many PDBS, take their maximum
7-8
Values of disjoint databases can be added and are still admissible [Korf & Felner: AIJ-02,
Felner, Korf & Hanan: JAIR-04]
Additivity can be applied if the cost of a subproblem is composed from costs of objects from corresponding pattern only

28. DADB:Tile puzzles 5-5-5 6-6-3 7-8 6-6-6-6 Memory Time Nodes Value
[Korf, AAAI 2005]
Memory
Time
Nodes
Value
Heuristic
Puzzle
3-tera-bytes
28 days
10^13
Breadth-FS 15
53.424
401,189,630
36.942
Manhattan
3,145
0.541
3,090,405
41.562
5-5-5
33,554
0.163
617,555
42.924
6-6-3
576,575
0.034
36,710
45.632
7-8
242,000
2 days
360,892,479,671 24

29. Heuristics for the TOH
Infinite peg heuristic (INP): Each disk moves to its own temporary peg.
Additive pattern databases
[Felner, Korf & Hanan, JAIR-04]

30. Additive PDBS for TOH4 6 10 Partition the disks into disjoint sets
Store the cost of the complete pattern space of each set in a pattern database.
Add values from these PDBs for the heuristic value.
The n-disk problem contains 4^n states
The largest database that we stored was of 14 disks which needed 4^14=256MB. 6 10

31. Duality  Motivation
What is the relation between state S and states S1 and S2? S 3 1 4 2 5 S1 5 2 4 1 3
Geometrical symmetries!! Reversing and/or rotating S2 1 4 2 5 3
And what is the relation between S and Sd?? Sd 2 4 1 3 5

32. Symmetries in PDBs 7 8 8 7 Symmetric lookups
Tile puzzles: reflect the tiles
about the main diagonal.
Rubik’s cube: rotate the cube
We can take the maximum among the different lookups
These are all geometrical symmetries
We suggest a new type of symmetry!! 7 8 8 7

33. Duality :definition 1 п п Let S be a state.
Let Π be a permutation such that Π(S)=G
Define: =Π(G)
consequences
Π ( )=G
The length of the optimal path from S to G and from S to G is identical S S d п -1 G S d п d d S d
An admissible heuristic for S is also admissible for S

34. Regular and dual representation
Regular representation of a problem:
Variables – objects (tiles, cubies etc,)
Values – locations
Dual representation:
Variables – locations
Values – objects

35. Duality :definition 2
Definition 2: For a state S we flip the roles of variables and objects
Assume a vector <3,1,4,2>
Regular representation:
Dual representation: 3 1 4 2 2 4 1 3

36. Duality Claim: Definition 1 and definition 2 are equivalent
Proof: Assume that in S, object j is in location i and that Π(i)=j.
Applying Π for the first time (on S) will move object j to location j.
Applying Π for the second time (on G) will move object i to location j

37. Using duality
Dual lookup: We can take the heuristic of the dual state and use it for the regular state.
In particular we can perform a PDB lookup for the dual state
Dual Search:
This is a novel search algorithm which can be constructed for any known search algorithm

38. Dual Search
When the search arrives at a state S, we also look at its dual state S.
We might consider to JUMP and continue the search from S towards the goal.
This is a novel version of bidirectional search d d

39. Example п п Traditional Search Bidirectional SearchG G п S 1 S 1 d
(a) No Jumps
(b) One Jump
Traditional Search
Bidirectional Search
Construction of the solution path is possible by applying usual backtracking with some simple modifications.

40. When to jump
At every node, a decision should be made whether to continue the search from S or to jump to S
Jumping Policies:
JIL: Jump if larger
JOR: Jump only at the root
J15,J24: Special jumping policies for the 15 and 24 tile puzzles d

41. Experimental results
Rubik’s cube: 7-edges PDB problem instances.
Heuristic
Search
Policy
Nodes
Time r
IDA* -
90,930,662
28.18 d
8,315,116
3.24
max(r,d)
2,997,539
1.34
DIDA*
JIL
2,697,087
1.16
JOR
2,464,685
1.02

42. Experimental results
16 Pancake problem 9-tiles PDB. 100 problem instances.
Heuristic
Search
Policy
Nodes
Time r
IDA* -
342,308,368,717
284,054 d
14,387,002,121
12,485
max(r,d)
2,478,269,076
3,086
DIDA*
JIL
260,506,693
362

43. Experimental results
15 puzzle 7-8 tiles PDB problem instances from [Korf & Felner 2002]
7-8
Heuristic
Search
Policy
Value
Nodes
Time r
IDA* -
44.75
136,289
0.081
Max(r,r*)
45.63
36,710
0.034
max(r,r*,d,d*)
46.12
18,601
0.022
DIDA*
J15
13,687
0.018

44. Experimental results
24 puzzle tiles PDB. 50 Problem instances from [Korf & Felner 2002]
Heuristic
Search
Policy
Nodes
max(r,r*)
IDA* -
43,454,810,045
max(r,r*,d,d*)
13,549,943,868
max(r,d*)
DIDA*
J24
8,248,769,713
3,948,614,947

45. Conclusions Duality in search spaces Two way to use duality:
1) the dual heuristic
2) the dual search
Improvement in performance

46. Inconsistent heuristics
Inconsistency sounds negative
“It is hard to concoct heuristics that are admissible but are inconsistent”
[AI book Russel and Norvig 2005]
“Almost all admissible heuristics are consistent” [Korf, AAAI-2000]

47. A* Best-first search with a cost function of f(n)=g(n)+h(n)
g(n): actual distance from the initial state to n
h(n): estimated remained distance from n to the goal state.
h(n) is admissible if it is always underestimating
Examples of h:
Air distance,
Manhattan Distance

48. Attributes of A*
If h(n) is admissible then A* is guaranteed to return an optimal (shortest) solution
A* is admissible, complete and optimally effective. [Pearl & Dechter 84]
A* is exponential in memory!!

49. IDA* IDA* [Korf 1985] is a linear-space version of A*.
A series of DFS calls with increasing threshold
Since 1985 it is the default algorithm for many problems!!
IDA* re-expands duplicate nodes !!!

50. Consistent heuristics
A heuristic is consistent if for every two nodes n and m
|h(n)-h(m)| dist(n,m)
Intuition  h cannot change by more than the change of g
h(n)  c(n,m) + h(m)

51. inconsistent heuristics
A heuristic is inconsistent if for some two nodes n and m
|h(n)-h(m)| > dist(n,m)
g=5
h=5
f=10
The child is inconsistent with its parent 1
g=6
h=2
f=8

52. Pathmax
The pathmax (PMX) method corrects inconsistent heuristics. [Mero 84,Marteli 77]
The child inherits its parent f-value if it is larger
g=5
h=5
f=10 1
g=6
h=2 4
f=810

53. Reopening of nodes with A*c a
f=2
f=1 d I 5 G
f=8
f=7
f=0 5
f=3 b
f=6
f=2
Node d is expanded twice with A*!!

54. In the context of A* inconsistency was considered a bad attribute

55. Inconsistency and IDA*
Node re-opening is not a problem with IDA*,because each path to a node is examined anyway!!
No overhead for inconsistent heuristics

56. Bidirectional pathmax (BPMX)
h-values
h-values 2 4
BPMX 5 1 5 3
Bidirectional pathmax: h-values are propagated in both directions decreasing by 1 in each edge.
If the IDA* threshold is 2 then with BPMX the right child will not even be generated!!

57. Achieving inconsistent heuristics
Random selection of heuristics
Dual evaluations [Zahavi et al 2006]
Compressed pattern databases [Felner et al 2004]

58. More than one heuristic
A munber of different PDBs
Symmetric lookups
Tile puzzles: reflect the tiles
about the main diagonal.
Rubik’s cube: rotate the cube 7 8 8 7

59. Using K heuristics Taking the maximum of K heuristics is
Admissible
Consistent
Better than each of them
Drawbacks: Overhead of K heuristics lookups
diminishing return.
Alternatively, we can randomize which heuristic out of K to consult.
Inconsistent
Benefits: Only one look up. BPMX can be activated.

60. Duality [Zahavi et al. AAAI-2006]
Let S be a state.
Let Π be a permutation such that Π(S)=G
Define: =Π(G) S п S d
Requirements G п
Any sequence of operators applicable to S is also applicable to G. S d
If the operators are invertible with same cost: An admissible heuristic for S is also admissible for S d

61. Achieving inconsistent heuristics
Dual evaluations
are inconsistent
[Zahavi et al. AAAI-2006]
3) Compressed pattern databases
In general – any partial heuristic is inconsistent.

62. Inconsistency and duality
Regular states
Dual states A 1 2 3 4 5 6 7 8 9
A^d=A 1 2 3 4 5 6 7 8 9 B 1 2 3 4 5 9 8 7 6
B^d=B 1 2 3 4 5 9 8 7 6
C^d C 1 2 3 8 9 5 4 7 6 1 2 3 7 6 9 8 4 5
Regular lookup for A, A^d, B, B^d : PDB[1,2,3,4,5]=0
Regular lookup for C : PDB[1,2,3,7,6]=1
Dual lookup for C: PDB[1,2,3,8,9]=2
Regular
Dual B C 1 2

63. Consistent Vs. Inconsistent
Assume a threshold of 5
consistent
inconsistent

64. Heuristic value distribution
Notice that all these heuristics have the
same average value

65. 7-edges PDB over 1000 instances of depth 14
Rubik’s cube results
Time
Nodes
Lookups No
28.18
90,930,662
Regular 1
7.38
19,653,386
Dual
3.24
8,315,116
Dual+BPMX
3.30
9,652,138
Random
1.25
3,828,138
Random+BPMX
7.85
13,380,154 2
11.60
10,574,180 4
7-edges PDB over 1000 instances of depth 14

66. 7-8 additive PDB over 1000 instances
Tile puzzle results
Time
Nodes
Lookups No
0.081
136,289
Regular 1
0.139
247,299
Dual+BPMX
0.029
44,829
Random+BPMX
0.034
36,130
Regular+reflected 2
0.025
26,862
2 Random+BPMX
0.026
21,425
3 Random+BPMX 3
0.022
18,601
All 4 4
7-8 additive PDB over 1000 instances