1. Efficient Automated Planning with New Formulations Ruoyun Huang Washington University in St. Louis

2. Classical AI Planning (STRIPS) loc1loc2 Init State: (AT pkg loc1); (AT truck loc2); loc1loc2 Goal: (AT pkg loc2) Applicable actions: MOVE; LOAD; UNLOAD; ? One solution with time step N=4: Time Step 1: MOVE truck loc2 loc1 Time Step 2: LOAD pkg truck loc1 Time Step 3: MOVE truck loc1 loc2 Time Step 4: UNLOAD pkg truck loc2 2

3.  Mars rover by NASA  High speed printer by PARC  Hubble space telescope  Mobile robots  Anti-air defense system  Airport scheduling  Biological network planning  Natural language processing  …  Challenges  High complexity  Expressiveness Applications and Challenges 3

4.  Formulation/Representation is vital to problem solving  Different formulations of planning  STRIPS representation [Fikes:71]  SAS+ representation [Backstrom:96]  In planning by convert-and-solve methods  Convert planning problem into Satisfiability (SAT) Constraint Satisfaction Programming (CSP), or Integer Programming  How we formulate is the key Motivation 4

5. SAT Outline Automated Planning Classical Planning STRIPS Search SAS+ SAT Search SAT 123 Blue backgrounds indicate my/our work SAT = Satisfiability POR = Partial Order Reduction Temporal Planning Temporally Expressive Applications Web Service Composition Parallel Algorithms POR Theory

6. SAS+ Representation loc1loc2 in truck at loc1at loc2 V(pkg) at loc1at loc2 V(truck) AT pkg loc1 AT pkg loc2 IN pkg truck AT truck loc1 AT truck loc2 StripsSAS+ Transition: Change between values in a multi-valued variable Transitions pkg:loc1  truck pkg:truck  loc1 pkg:truck  loc2 pkg:loc2  truck 6 DTG (Domain Transition Graph)

7. SAS+ is more compact fact… … … 2 F … … … … … … … … … 10 F/10 F: number of STRIPS facts Assume avg. facts/value per DTG is 10 fact… 7 SAS+ also captures more hidden problem structures

8.  Heuristic Function  Inadmissible heuristic function [Helmert:06]  Admissible heuristic function [Helmert:08,Katz:08]  Exception: Mixed integer programming [Briel:04] Related Works on SAS+ 8 call for more comprehensive studies Planning models & algorithms Planning as Search Heuristic function design

9. Outline  SAS+ Based Search Model [Chen, Huang, Zhang: 08] Automated Planning Classical Planning STRIPS SAS+ Search SAT Search SAT 1 Temporal Applications

10.  A popular technique for planning  Limitation of heuristic search  Exponential number of states to visit [Helmert:08]  Our contribution:  Search in abstraction state spaces  Smaller space  Comparing with HTN planning  We don need domain knowledge Heuristic Search 10

11. 11 Causal Dependency and Causal Graph loc1loc2 at loc1at loc2 in truck at loc1at loc2 pkg’s DTG truck’s DTG pkg’s DTG truck’s DTG LOAD pkg truck loc2: Pre: (AT truck loc2) Causal Graph [Helmert08]

12. 12 Motivation of Abstraction  Causal graph induces hierarchies within planning problems To build a house 1. Drive to the city hall 2. Fill out an application 3. Drive to the bank 4. Ask for money 5. Drive somewhere 6. Look for a builder 7. Sign a Contract 8. Build foundation 9. Build frame 10. Build roof 11. Build wall 13. Buy furniture … … 1. Get a permit 2. Get enough money 3. Construction 4. Interior … … 1.a Drive to the city call 1.b Fill out an application 2.a Drive to bank 2.b Fill out an application 3.a Find a builder 3.b Sign a contract 3.c Build foundation 3.d Build frame 3.e Build roof 3.f build wall …

13. 13 Abstraction State Space truck’s DTG package’s DTG full state space Abstraction loc1loc2 1.LOAD pkg truck loc1 2.UNLOAD pkg truck loc2 1.a MOVE truck loc2 loc1 1.b LOAD pkg truck loc1 2.a MOVE truck loc1 loc2 2.b UNLOAD pkg truck loc2 in truck at loc1at loc2 DTGs guide us to determine high-level plans

14. 14 Abstraction State Space Search (Iteration #1) ………… ………… Goal 1 Goal 2 Abstraction State Space DTG Finds a plan quickly

15. 15 Abstraction State Space Search (Iteration #2) ………… ………… G1G1 G2G2 Abstraction State Space DTG Keep running to improve the plan quality

16. Results InstancesAvg. TimeAvg. Plan Length DTG-Plan Fast Downward DTG-Plan Fast Downward DTG-Plan Fast Downward Openstack28 3.3910.91131.6131.5 Pathways30 1.692.90166.3132.8 Rovers40 12.059.69151.9112.2 Storage17180.581.7815.914.6 TPP30 9.1647.16172.2133.1 Trucks161019.1953.0431.5028.4  The average running time and plan length are calculated by the instances solved by both algorithms.  Here DTG-Plan runs for a single iteration, more iterations will improve the solution quality.

17. Results (Rovers Domain) Time FastDownward Quality Quality Iter#1Iter#2Iter#3 Iter#1Iter#2Iter#3 Rover30.090.2120.209151813 Rover40.090.2180.20981388 Rover50.110.3082.2363327 22 Rover60.140.70715.8963846 36 Rover70.130.370.42823362018 Rover80.230.6569.9826363129 Rover90.230.82419.06836544240 Rover100.251.52355.2583741 38 Rover110.280.9734.42156714636 Rover120.240.6520.91221322419 Rover130.416.579120.91281805856 Rover140.30.9124.7632453533 Rover150.411.55927.12744494442 Rover160.483.56428.486517746 Rover170.7917.775366.275269 48 Rover181.374.792 516.72475551 47 Downward has a similar running time to Iter#1

18. Outline  Enhanced mutual exclusion [Chen, Huang, Xing, Zhang: 09] 2 Automated Planning Classical Planning STRIPS SAS+ Search SAT Search SAT Temporal Applications

19. 19 Planning as Satisfiability Given planning problem P : Encode P with time N, resulting in SAT formula E N Use a SAT solver to solve E N E N is solvable? N = 0 Decode and output solution Y N N = N + 1 Encoding is the key!

20. Strips Based Encoding (at pkg loc1) (at truck loc2) move (at pkg loc1) (at truck loc2) (at truck loc1) Time step 1Time step 2 (at pkg loc1) move (at truck loc2) (at truck loc1) load (in pkg truck) ………………………… Time step 3,4,5,… Compile from planning graph 20 SAT Instance : Variables Clauses Planning Graph Nodes (facts, actions) Edges Results in a whole chunk of encoding

21.  Two facts/actions cannot be true at the same time  Fact mutual exclusion  Action mutual exclusion Mutual Exclusion (Mutex) Move-truck-loc1-loc2 Move-truck-loc2-loc1 (in pkg truck) (at truck loc1) (at truck loc2) (at pkg loc1) (at pkg loc2) (in pkg truck) (at truck loc1) (at truck loc2) (at pkg loc1) (at pkg loc2) A key reason for the efficiency of SAT-based planning 21

22. 22 Mutual Exclusion Time #2 Time #3 #4 #5 #6 #7 #8 #9#10 Time #1 Invalid (at truck loc1)(at truck loc2)

23. 23 Long Distance Mutual Exclusion(Londex) Time #2 Time #3 #4 #5 #6 #7 #8 #9#10 Time#1 Invalid (at truck loc1) (at truck loc4) DTG of truck (at truck loc1) (at truck loc2) (at truck loc3)(at truck loc4)

24.  Significant speed-up  Hundreds times faster in some cases (compared to SatPlan04)  Limitation  Too large encoding size (ten times in average) Results of Londex 24

25. Outline  A SAS+ based encoding scheme [Huang,Chen,Zhang:10] AAAI’10 Outstanding Paper Award 3 Automated Planning Classical Planning STRIPS SAS+ Search SAT Search SAT Temporal Applications

26. STRIPS versus SAS+ STRIPSSAS+ Definition a set of preconditions, a set of add effects, a set of delete effects A set of transitions Example (LOAD pkg truck loc1) Pre: (at truck loc1), (at pkg loc1)pkg:(loc1  truck) truck: (loc1  loc1) Del:(at pkg loc1) Add:(in pkg truck) Usually there are fewer transitions than actions Hierarchical relationships between actions and transitions Different Representations of Actions: 26

27. Overview of New Encoding …………………… SAT Instance (Part 1) : transitions SAT Instance (Part 2) : matching actions and transitions (multiple independent ones) …… … Transitions Actions t = 1t =2t =3 SAT Instance : Facts and actions Actions& Facts Planning graph … …… t = 1t =2t =3, 4, … Strips Based Encoding SAS+ Based New Encoding 27

28. 28 Clauses in New Encoding truck:loc2 Time step 1Time step 2 ………………………… Time step 3,4,5,… pkg: loc1 truck:loc1 truck:loc2 truck:loc1 truck:loc2 pkg: loc1 pkg: truck pkg: loc2 pkg: loc1 pkg: truck pkg: loc2 Find matchings set of actions … pkg: truck pkg: loc2 1.Progression of transitions over time steps ( blue one implies green ones ) 2.Initial state and goal (Bold ones) 3.Matching actions and transitions 4.Action mutual exclusions and transition mutual exclusions

29. State Space Size …………………… State Space (Part 1) : O((2 T ) N ) State Space (Part 2) : O(NK) …… … Transitions Actions t = 1t =2t =3 State Space: O((2 A ) N ) Actions Planning graph … …… t = 1t =2t =3, 4, … Strips Based Encoding SAS+ Based New Encoding 29 Total: O((2 T ) N NK) K KK Usually there are fewer transitions than actions A Number of actions N Number of Time Step T Number of transitions

30.  Variable State Independent Decaying Sum (VSIDS) decision heuristic [Moskewicz:01]  The more highly constrained, the more chances a variable will be assigned a value during search Why Transition Variables First? …………………… …… … t = 1t =2t =3 K KK Transition Variables: more constrained Action Variables: less constrained 30

31. Number of Solvable Instances 31 SatPlan06 L = SatPlan06 + Londex

32. Detailed Results SatPlan06New Encoding Instances Time (sec) #Variables#Clauses Size (MB) Time#Variables#ClausesSize Airport402239.4327,51513,206,595807583.3396,212 3,339,914 208 Driverslog 17 2164.861,9152,752,787183544.174,680812,31256 Freecell4364.3175826,114,100392158.426,009371,20725 Openstack4212.13,70966,744533.64,88920,0222 Pipesworld123147.330,07813,562,157854543.743,528634,87344 TPP303589.797,1557,431,0624621844.8136,106997,17770 Trucks71076.021,745396,58127245.735,065255,02018 Zeno14728.426,2016,632,92342158.717,459315,71918 32 Better performances in 10 benchmark domains out of 11 tested (from IPC3,4,5)

33.  To planning  The dynamic world  constraints  To general artificial intelligence  Bounded model checking [Clarke01]  Answer set programming [Lin02]  … Significance of New Encoding

34.  Compactness  Problem structure  Future works  Other (classical) planning algorithms, e.g. Partial Order Causal Link, …  More advanced planning models, e.g. probabilistic, uncertainty, … Conclusions on SAS+

35. Thank You

36.  Transition  ?:?  ? ( e.g. pkg:loc1  truck )  O(b 3 )  Action  DoSth(?, ?, ?, ?, ? …)  O(b p ) Why Transition is fewer  Transition  A smaller unit  Action  A larger concept with more objects involved in

38. Future Works Automated Planning Classical Planning STRIPS Search SAS+ SAT Search SAT Temporal Planning Temporally Expressive Applications Web Service Composition Parallel Algorithms POR Theory POCL … …