1. Searching for Macro-operators with Automatically Generated Heuristics István T. Hernádvölgyi University of Ottawa istvan@site.uottawa.ca

2. Motivation Time Quality 1 sec 1 week 18 85 Korf 97 Korf 85 50455 Sims 70, Osterlund 95 us

3. Subgoals and Macros 12345

4. 12345

5. 1 2 3 4 1 1 1 2 23 12345

6. 1 1 1 1 1 2 2 2 2 3 3 3 4 4 1 1 1 1 1 1 1 1 1 2 2 2 2 2 3 3 12345

7. 1 1 1 1 1 2 2 2 2 3 3 3 4 4 1 1 1 1 1 1 1 1 1 2 2 2 2 2 3 3 {} {o1} {o2} {o3} {o4} {} {o2,o1,o2} {o3,o2,o3} {o4,o3,o4} {o3,o1,o3} {} {o4,o2,o4} {} {o4,o1,o4} 12345

8. 1 1 1 1 1 2 2 2 2 3 3 3 4 4 1 1 1 1 1 1 1 1 1 2 2 2 2 2 3 3 {} {o1} {o2} {o3} {o4} {} {o2,o1,o2} {o3,o2,o3} {o4,o3,o4} {o3,o1,o3} {} {o4,o2,o4} {} {o4,o1,o4} 41253

9. 1 1 1 1 1 2 2 2 2 3 3 3 4 4 1 1 1 1 1 1 1 1 1 2 2 2 2 2 3 3 {} {o1} {o2} {o3} {o4} {} {o2,o1,o2} {o3,o2,o3} {o4,o3,o4} {o3,o1,o3} {} {o4,o2,o4} {} {o4,o1,o4} 41253 {o1}

10. 1 1 1 1 1 2 2 2 2 3 3 3 4 4 1 1 1 1 1 1 1 1 1 2 2 2 2 2 3 3 {} {o1} {o2} {o3} {o4} {} {o2,o1,o2} {o3,o2,o3} {o4,o3,o4} {o3,o1,o3} {} {o4,o2,o4} {} {o4,o1,o4} 14253 {o1,o2,o1,o2}

11. 1 1 1 1 1 2 2 2 2 3 3 3 4 4 1 1 1 1 1 1 1 1 1 2 2 2 2 2 3 3 {} {o1} {o2} {o3} {o4} {} {o2,o1,o2} {o3,o2,o3} {o4,o3,o4} {o3,o1,o3} {} {o4,o2,o4} {} {o4,o1,o4} 12453 {o1,o2,o1,o2,o4,o2,o4}

12. 1 1 1 1 1 2 2 2 2 3 3 3 4 4 1 1 1 1 1 1 1 1 1 2 2 2 2 2 3 3 {} {o1} {o2} {o3} {o4} {} {o2,o1,o2} {o3,o2,o3} {o4,o3,o4} {o3,o1,o3} {} {o4,o2,o4} {} {o4,o1,o4} 12354 {o1,o2,o1,o2,o4,o2,o4,o4,o1,o4} {o1,o2,o1,o2,o4,o2,o1,o4} {o2,o4,o3,o2}

13. 2 2 2 2 3 3 3 1 1 1 1 1 1 1 2 2 2 {} {o2,o1,o2} {o3,o2,o3} {o4,o3,o4} {o3,o1,o3} {} {o4,o2,o4}

14. 231 {} 231 231 231 231 231 231 231 231 231 231 231 {o3,o1,o3} {o4,o2,o4} {o1,o2,o1} {o1,o3,o2} {o1,o4,o1,o3} {o2,o3,o1} {o3,o2,o3} {o2,o4,o2} {o3,o4,o1} {o2,o3,o4,o2,o3} {o4,o3,o4}

16. Results

17. The cost of merging subgoals #macrosavg/max length States expanded to build table Time to solve 10,000 problems 18 subgoals2585.6 / 13108 million (1 hour) 6 subgoals (8,3,2,2,1) 2902 + 96 million 9.7 / 13 ?

18. Solution: use heuristics to speed up search for each macro For each subgoal, create a heuristic specifically for that subgoal Can combine this with the heuristic for the previous subgoal (because that subgoal is a subset of the current subgoal) Need to create these automatically, they are unintuitive spaces and the heuristics are “throw away”

19. Previous Work Automatically Generated Heuristics Absolver [Prieditis90] Pattern Database [CulbersonSchaeffer96] –15 Puzzle, “don’t care tiles”, Rubik’s Cube [Korf97] –Domain Abstraction [HolteHernadvolgyi99] –Planning (Strips) [Edelkamp2000]

20. Pattern Database 123 456 78 123 6 181,44015,120

21. Pattern Database 634 875 12 63 12

22. Domain Abstraction 123 456 78 181,4405,040

23. Rubik’s Cube ULFUFURF LFRF DLFDFDRF F DLDR DLBDBDRB D ULBUBURB ULURU ULBUL LB DLBDL L URURB RB DRDRB R UBULB LB DBDLB B

24. State Representation Goal state (subscripts indicate orientation) Typical start state

25. Macro Search Space Goal state for the subgoal “fix RF and RB” Typical start state for this subgoal

26. Domain Abstraction “don’t care” about a cubie’s orientation Make 2 (or more) cubies indistinguishable from each other Continue to add abstractions until the abstract space is sufficiently small (2 million entries)

27. Abstraction Example Abstracted goal state (x = orientation doesn’t matter) Abstracted start state (typical)

28. The cost of merging subgoals #macrosavg/max length States expanded to build table 18 subgoals2585.6 / 13108 million (1 hour) 6 subgoals (8,3,2,2,1) 2902 + 96 million 9.7 / 13 2 billion + 250,000 /problem (27 hours + 1 second/problem)

29. Contributions Automatically Generated Heuristics for Finding Macro-operators Merging Subgoals to Obtain Shorter Solutions 44% Improvement for Rubik’s Cube Method is General and Mostly Automatic

30. Using Symmetries

31. URF, URB,ULB,ULF,UF,UL,UR,UB LF,LB,DL RF,RB DF,DB (> DR) DRF,DRB DLF (>DLB) LF,RF,DRRF,RB,DR LB,RB,DL DLF, DLB DRF (> DRB) LF,RBLF,LB LF, RF

32. Kociemba’s Two-phase Algorithm

33. Open Questions Do the improvements carry over to other spaces? Does the order of the subgoals matter?

44. Motivation

45. Time Quality 1 sec 1 week 18 85 Korf 97 Korf 85 Kociemba 92 50 Kloosterman 90 Thistlethwaite 80 455 Sims 70, Osterlund 95 human

46. Merging Consecutive Subgoals Serial Decomposability [Korf83] Idea:

50. Previous Work Macro-Operators Stabilizer Chain [Sims70, Osterlund95] Bi-directional Partial-Match [Korf85]

51. Domain Abstraction 1.select two invariant cubies: c1 and c2 2.toss coin 3.if (heads) // mask orientations 4. if (both orientations masked) 5. goto 1 6. mask orientations of c1 and c2 7.else // mask cubie identity 8. if (mask(c1) == mask(c2) goto 1 9. select new mask A 10. if (!masked(c1) && !masked(c2)) 11. c1 := A, c2 := A 12. else if (masked(c1) && !masked(c2)) 13. all cubies with mask(c1) and c2 := A 14. else if (!masked(c1) && masked(c2)) 15. all cubies with mask(c2) and c1 := A 16. else // both has a mask 17. cubies with mask(c1) and mask(c2) := A 18. H := expand(N) 19. if (H == 0) goto 1 20. return H

52. Search IDA* [Korf85] Maximum of 3 pattern databases (5M entries) 18 subgoals 6 subgoals 10,000 random instances

53. Statistics