1. 1 Integer Programming Approaches for Automated Planning Menkes van den Briel Department of Industrial Engineering Arizona State University menkes@asu.edu http://www.public.asu.edu/~dbvan1/ menkes@asu.edu http://www.public.asu.edu/~dbvan1/

2. 2 What is automated planning? Ordering problem Scheduling is the problem of deciding when to execute a set of actions NP-complete Selection and ordering problem Planning is deciding both what actions need to be done and when to execute them PSPACE-complete SchedulingPlanning

3. 3 What is automated planning? Creating a computer program to produce a plan, a sequence of actions that will transform the world from some given initial state to a desired goal state Initial state s 0  S Goal g  S Plan P =  a 1, …, a n  Action Actions are state transformation functions 12 12

4. 4 What is automated planning? Creating a computer program to produce a plan, a sequence of actions that will transform the world from some given initial state to a desired goal state Initial state s 0  S Goal g  S Plan P =  a 1, …, a n  Action sisi sjsj Actions are state transformation functions

5. 5 Planning applications Autonomous vehicles –Mars rovers –Underwater robotics –Remote agent experiment Games –Bridge Baron –General game playing Others –Manufacturing process planning –Composition of web services –Cyber Security

6. 6 Planning by integer programming Operations research (OR) Scheduling problems typically involve solving hard optimization problems Integer programming (IP), branch-and-bound Artificial intelligence (AI) Planning problems typically involve solving hard feasibility problems Constraint satisfaction, satisfiability (SAT), A* search SchedulingPlanning

7. 7 Planning by integer programming Very little focus on integer programming approaches for planning –[Bylander, 1997] –[Bockmayr and Dimopoulos, 1998, 1999] –[Kautz and Walser, 1999] –[Vossen et al., 1999] –[Dimopoulos, 2001] –[Dimopoulos and Gerevini, 2002]

8. 8 1.IP-based approaches simply don’t work –“Lplan [a linear programming-based heuristic for optimal planning] was often slower than the other algorithms primarily due to the time to evaluate the linear programming heuristic” [Bylander, 1997] 2.SAT-based approaches are much faster –SAT-based planners have successfully participated in IPC1, IPC2, IPC4, and IPC5 3.Traditionally there has been little focus on plan quality –Planning is PSPACE-complete, so finding a feasible plan is already hard enough Why this lack of interest?

9. 9 1.IP-based approaches do work –Optiplan, first IP-based planner to take part in the IPC series –Ranked 2nd in four out of seven domains in IPC4 in the optimal track for propositional domains 2.IP-based approaches can compete with SAT-based approaches –Represent planning as a set of interdependent network flow problems –Generalize the notion of action parallelism 3.Shift in focus towards optimal planning –Applied formulations to partial satisfaction planning problems –Developed a novel framework for optimal planning –Utilized LP relaxations in deriving quality sensitive heuristics Counter arguments

10. 10 1.IP-based approaches do work 2.IP-based approaches can compete with SAT-based approaches 3.Shift in focus towards optimal planning Contributions –[Van den Briel, and Kambhampati. Journal of Artificial Intelligence Research, 2005] –[Van den Briel, Vossen, and Kambhampati. ICAPS, 2005] –[Van den Briel, Vossen, and Kambhampati. Journal of Artificial Intelligence Research, 2008] –[Van den Briel, et al. AAAI, 2004] –[Do, Benton, van den Briel, and Kambhampati. IJCAI, 2007] –[J. Benton, van den Briel, and Kambhampati. ICAPS, 2007] –[Van den Briel, Benton, Kambhampati, and Vossen. CP, 2007]

11. 11 1. IP approaches do work Optiplan –IP-based planner that extends the state change formulation by [Vossen et al., 1999] [van den Briel, and Kambhampati, 2005]

12. 12 Summary of results International planning competition (IPC) –Bi-annual event –Provides data sets (domains) that are used as benchmarks IPC4 –7 competition domains –7 participating planners in the “optimal” track Domains –Pipesworld Control the flow of oil derivatives through a pipeline network, obeying various constraints such as product compatibility and tankage restrictions –Satellite Collect image data with a number of satellites –Philosophers, Optical telegraph Involves finding deadlocks in communication protocols

13. 13 Summary of results

14. 14 2. IP versus SAT approaches 1.Represent planning as a set of interdependent network flow problems –One network flow problem for each state variable in the planning domain –Nodes correspond to the values of the state variables, arcs correspond to the value transitions 2.Generalize the notion of action parallelism –Reduces the plan length of the solution plan (and thus the size of the formulation)

15. 15 Logistics example 12 P T 1 2 Truck Drive(1,2)Drive(2,1) Load(P,T,1) Unload(P,T,1) 1 2 T Package Load(P,T, 1) Load(P,T, 2) unload(P,T, 1) unload(P,T, 2) States are described by state variables

16. 16 Logistics example 12 1 2 Truck Drive(1,2)Drive(2,1) Load(P,T,1) Unload(P,T,1) 1 2 T Package Load(P,T, 1) Load(P,T, 2) unload(P,T, 1) unload(P,T, 2) Actions are state transformation functions Effect Prevail

17. 17 One state change (1SC) Network representation Logistics example 2 1 2 1 2 1 2 1 tt Truck Package h g f h g f h g f t = 1 Prevail Effect Planning involves considering plans of increasing length Plan step

18. 18 One state change (1SC) Network representation Logistics example 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 tttt Drive(1,2) Load(P,T, 1)Unload(P,T, 2)- Load(P,T, 1) Unload(P,T, 2) Truck Package h g f h g f h g f t = 1t = 2t = 3 Prevail Effect

19. 19 1SC formulation Constraints –State changes (network flow), for all c  C  g  C y c f,g,t = 1{f  I}for f  D c  h  C y c g,h,t+1 =  f  C y c g,h,t for f  D c, 1  t < T  f  C y c f,g,T = 1for g  G –Effect implications, for all c  C, 1  t  T  a  A:(f,g)  SC(a) x a,t = y c f,g,t for f, g  D c, f  g x a,t  y c f,f,t for a  A, f  PR(a)

20. 20 Summary of results Experimental setup –Domains from IPC2, IPC3 –Comparing 1SC formulation versus SATPLAN04 (winner of the “optimal“ track IPC4) –2.67GHz CPU with 1.0GB memory Domains –Logistics, Driverlog Involves driving trucks (and flying airplanes) around to deliver packages between locations –Blocksworld Stacking and unstacking towers of blocks –Zenotravel Transporting people around in planes, using different modes of movement: fast and slow

21. 21 Summary of results

22. 22 2. IP versus SAT approaches 1.Represent planning as a set of interdependent network flow problems –One network flow problem for each state variable in the planning domain –Nodes correspond to the values of the state variables, arcs correspond to the value transitions 2.Generalize the notion of action parallelism –Reduces the plan length of the solution plan (and thus the size of the formulation)

23. 23 Generalized one state change (G1SC) Network representation Example 2 1 2 1 2 1 2 1 2 1 2 1 ttt Load(P,T, 1) Drive(1,2) Load(P,T, 1)Unload(P,T, 2) h g h ff g Truck Package h g f t = 1t = 2 Prevail Effect

24. 24 Implied precedences (G1SC) Example A1,A2 A3 A4 A1 A2 A3A4 Implied precendence graph

25. 25 Implied precedences (G1SC) Example Ordering (cycle elimination) constraints ensure a feasible ordering of the actions A1,A2 A3 A4 A1 A2 A3A4 x A1,t + x A3,t + x A4,t  2 Implied precendence graph

26. 26 G1SC formulation Constraints –State changes (network flow), for all c  C  g  C y c f,g,t = 1{f  I}for f  D c  h  C y c g,h,t+1 =  f  C y c g,h,t for f  D c, 1  t  T  f  C y c f,g,T = 1for g  G –Effect implications, for all c  C, 1  t  T  a  A:(f,f)  SC(a) x a,t = y c f,g,t for f, g  D c, f  g, x a,t  y c f,f,t +  g  Dc:f≠g (y c g,f,t + y c f,g,t ) for a  A, f  PR(a) –Ordering (Cycle elimination) constraints  a  V(  ) x a,t  |V(  )| – 1for all cycles  G, 1  t  T

27. 27 Branch-and-cut Initialize LP Node selection LP solver Branching Cut generation START Nodes found? Cuts found? STOP Feasible? Fathom Z_lp < Z*? no yes no yes no Integer? no yes

28. 28 Prevail Effect State change path (PathSC) Network representation Example 2 1 2 1 2 1 2 1 tt Load(P,T, 1) Drive(1,2) Unload(P,T, 2) load(P,T, 1) unload(P,T,2) h g f h g Truck Package h g ff t = 1

29. 29 Summary of results

30. 30 Summary of results [van den Briel, Vossen, and Kambhampati, 2005, 2008]

31. 31 3. Shift towards optimal planning Applied formulations to partial satisfaction planning problems Developed a novel framework for optimal planning Utilized LP relaxations in deriving quality sensitive heuristic search approaches

32. 32 Partial satisfaction planning P LAN L ENGTH is PSPACE-complete –[Bylander, 1994] PSP U TILITY C OST is PSPACE-complete –[Van den Briel, et al., 2004] P LAN E XISTENCE P LAN L ENGTH PSP G OAL L ENGTH PSP G OAL P LAN C OST PSP U TILITY PSP N ET B ENEFIT Total Satisfaction Problems Partial Satisfaction Problems PSP U TILITY C OST

33. 33 Framework for optimal planning For step-based IP formulations optimality is restricted to the length of the plan Plan step 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 tttt Drive(1,2) Load(P,T, 1)Unload(P,T, 2)- Load(P,T, 1) Unload(P,T, 2) Truck Package t = 1t = 2t = 3

34. 34 Framework for optimal planning 12 P T 1 2 Truck Drive(1,2)Drive(2,1) Load(P,T,1) Unload(P,T,1) 1 2 T Package Load(P,T, 1) Load(P,T, 2) unload(P,T, 1) unload(P,T, 2)

35. 35 Action selection formulation Variables –x a  Z +, for a  A ; x a is equal to the number of times action a is executed –y  v(c,a)  Z +, for v  V, a  A, a   – (c) ; y  v(c,a) is equal to the number of times transition  v (c,a) is executed Objective function –MIN  a  A c a x a Constraints –  a  v+(e) y  v(c,a) –  a  v–(e) y  v(c,a)  –  a  v+(e) y  v(c,a) = x a 1 if c  c 0,v, c  g –1 if c = c 0,v, c  g 0 otherwise No time indices No upper bounds

36. 36 Concurrent automata Given a set of state variables V = {v 1, …, v n } For each v  V we define a deterministic automaton G v = (D v, A v,  v,  v, c 0,v, g v ) –D v is a finite set of states corresponding to the domain of state variable v –A v is a finite set of actions associated with the transitions in G v –  v : D v  A  D v is the transition function –  v : D v  2 A is the active action function –c 0,v  S is the initial state of state variable v –g v  S is a set of goal states of state variable v

37. 37 Parallel composition The parallel composition of the two automata G 1 and G 2 is the automaton G 1 ||G 2 := (D 1  D 2, A 1  A 2,  1||2,  1||2, (c 0,1, c 0,2 ), g 1  g 2 ) –  1||2 ((c 1,c 2 ),a) := –  1||2 (c 1,c 2 ) := [  1 (c 1 )  2 (c 2 )]  [  1 (c 1 )\A 2 ]  [  2 (c 2 )\A 1 ] (  1 (c 1,a),  2 (c 2,a) if a   1 (c 1 )  2 (c 2 ) (  1(c 1,a), c 2 ) if a   1 (c 1 )\A 2 (c1,  2(c 2,a)) if a   2 (c 2 )\A 1 undefinedotherwise

38. 38 Logistics example 12 P T 1 2 Truck Drive(1,2)Drive(2,1) Load(P,T,1) Unload(P,T,1) 1 2 T Package Load(P,T, 1) Load(P,T, 2) unload(P,T, 1) unload(P,T, 2)

39. 39 Simple logistics example 1,1 1,T 2,T 2,2 1,2 2,1 Truck || Package Drive(1,2) Drive(2, 1) Load(P, T, 1) Load(P, T, 2) Unload(P, T, 1) Unload(P, T, 2) Drive(1, 2) Drive(2, 1) Drive(1, 2)Drive(2, 1) 12 P T

40. 40 Summary of results Highlighted values equal optimal solution

41. 41 Summary of results

42. 42 Utilize LP in heuristic search [Benton, van den Briel, and Kambhampati, 2007] BBOP-LP planner

43. 43 Summary IP-based approaches do work –Optiplan, first IP-based planner to take part in the IPC series –Ranked 2nd in four out of seven domains in IPC4 in the optimal track for propositional domains IP-based approaches can compete with SAT-based approaches –Represent planning as a set of interdependent network flow problems –Generalize the notion of action parallelism Shift in focus towards optimal planning –Applied formulations to partial satisfaction planning problems –Developed a novel framework for optimal planning –Utilized LP relaxations in deriving quality sensitive heuristics

44. 44 Publications status Journal –[M.H.L. van den Briel, and S. Kambhampati. Optiplan: Unifying IP-based and graph-based planning. Journal of Artificial Intelligence Research, 24:623-635, 2005] –[M.H.L van den Briel, T. Vossen, and S. Kambhampati. Loosely coupled formulation for automated planning: An integer programming perspective. Journal of Artificial Intelligence Research, 31:217-257, 2008] –[(In progress) M.H.L van den Briel, T. Vossen, S. Kambhampati and J. Fowler. Optimal automated planning] Conference –[M.H.L. van den Briel, R. Sanchez, M.B. Do, and S. Kambhampati. Effective approaches for partial satisfaction (oversubscription) planning. In Proceedings of AAAI, pages 562-569, 2004] –[M.H.L. van den Briel, T. Vossen, and S. Kambhampati. Reviving integer programming approaches for AI planning: A branch-and-cut framework. In Proceedings of ICAPS, pages 161-170, 2005] –[M.B. Do, J. Benton, M.H.L. van den Briel, and S. Kambhampati. Planning with goal utility dependencies. In Proceedings of IJCAI, pages 1872-1878, 2007] –[J. Benton, M.H.L. van den Briel, and S. Kambhampati. A hybrid linear programming and relaxed plan heuristic for partial satisfaction planning problems. In Proceedings of ICAPS, pages 24-41, 2007] –[M.H.L. van den Briel, J. Benton, S. Kambhampati, and T. Vossen. An LP-based heuristic for optimal planning. In Proceedings of CP, pages 651-665, 2007] Cited by 31 Cited by 15 Cited by 3 Cited by 4 Cited by 6 Cited by 3

45. 45 Publications status Workshop and posters –[M.H.L. van den Briel, R. Sanchez, and S. Kambhampati. Over-Subscription in Planning: a Partial Satisfaction Problem. In Proceedings of ICAPS Workshop on Integrating Planning into Scheduling, 2005] –[M.H.L. van den Briel,. Kambhampati, and T. Vossen. Planning with numerical state variables through mixed integer programming. In Proceedings of ICAPS Poster Session, pages 5-8, 2005] –[M.H.L. van den Briel,. Kambhampati, and T. Vossen. Planning with preferences and trajectory constraints by integer programming. In Proceedings of ICAPS Workshop on Preferences and Soft Constraints in Planning, pages 19-22, 2006] –[J. Benton, M.H.L. van den Briel,. Kambhampati. Finding admissible bounds for oversubscription planning problems. In Proceedings of ICAPS Workshop on Heuristics for Domain-Independent Planning: Progress, Ideas, Limitations, Challenges, 2007] –[M.H.L. van den Briel,. Kambhampati, and T. Vossen. Fluent merging: A general technique to improve reachability heuristics and factored planning. In Proceedings of ICAPS Workshop on Heuristics for Domain-Independent Planning: Progress, Ideas, Limitations, Challenges, 2007] Cited by 5 Cited by 1 Citation count by Google Scholar