1. Efficient Automated Planning with New Formulations Ruoyun Huang Advisors: Dr. Yixin Chen and Dr. Weixiong Zhang Washington University in St. Louis

2.  Dr. Yixin Chen (co-advisor)  Dr. Christopher D. Gill  Dr. Norman I. Katz  Dr. Kilian Weinburger  Dr. Weixiong Zhang (co-advisor) Proposal Committee Members

3. Classical AI Planning (STRIPS) loc1loc2 Init State: (AT pkg loc1); (AT truck loc2); loc1loc2 Goal: (AT pkg loc2) Applicable actions: MOVE; LOAD; UNLOAD; ? One solution with time step N=4: Time Step 1: MOVE truck loc2 loc1 Time Step 2: LOAD pkg truck loc1 Time Step 3: MOVE truck loc1 loc2 Time Step 4: UNLOAD pkg truck loc2 3

4.  Mars rover by NASA  High speed printer by PARC  Storytelling  Mobile robots  Anti-air defense system  Airport scheduling  Biological network planning  Natural language processing  …  Challenges  High complexity  Expressiveness Applications and Challenges 4

5.  Formulation/Representation is vital to problem solving  Different formulations of planning  STRIPS representation [Fikes:71]  SAS+ representation [Backstrom:96]  In planning by convert-and-solve methods  Convert planning problem into Satisfiability (SAT) Constraint Satisfaction Programming (CSP), or Integer Programming  How we formulate is the key Motivation 5

6. Outline Automated Planning Classical Planning Temporally Expressive Planning STRIPS SAS+ STRIPS Search SAT Search SAT Search SAT 123 4 Blue backgrounds indicate my work SAT = Satisfiability

7. SAS+ Representation loc1loc2 in truck at loc1at loc2 V(pkg) at loc1at loc2 V(truck) AT pkg loc1 AT pkg loc2 IN pkg truck AT truck loc1 AT truck loc2 StripsSAS+ Transition: Change between values in a multi-valued variable Transitions pkg:loc1  truck pkg:truck  loc1 pkg:truck  loc2 pkg:loc2  truck 7 DTG (Domain Transition Graph)

8. SAS+ is more compact fact… … … 2 F … … … … … … … … … 10 F/10 F: number of STRIPS facts Assume avg. facts/value per DTG is 10 fact… 8 SAS+ also captures more hidden problem structures

9.  Heuristic Function  Inadmissible heuristic function [Helmert:06]  Admissible heuristic function [Helmert:08,Katz:08]  Mixed integer programming [Briel:04]  Built on commercial solver CPlex Related Works on SAS+ 9

10. Outline Automated Planning Classical Planning Temporally Expressive Planning STRIPS SAS+ STRIPS Search SAT Search SAT Search SAT 1  SAS+ Based Search Model [Chen, Huang, Zhang: 08]

11.  A popular technique for planning  Limitation of heuristic search  Exponential number of states to visit [Helmert:08]  Our contribution:  Search in abstraction state spaces  Smaller space  Different from HTN planning, no domain knowledge needed Heuristic Search 11

12. 12 Causal Dependency and Causal Graph loc1loc2 at loc1at loc2 in truck at loc1at loc2 pkg’s DTG truck’s DTG pkg’s DTG truck’s DTG LOAD pkg truck loc2: Pre: (AT truck loc2) Causal Graph

13. 13 Motivation of Abstraction  Causal graph induces hierarchies within planning problems To build a house 1. Drive to the city hall 2. Fill out an application 3. Drive to the bank 4. Ask for money 5. Drive somewhere 6. Look for a builder 7. Sign a Contract 8. Build foundation 9. Build frame 10. Build roof 11. Build wall 13. Buy furniture … … 1. Get a permit 2. Get enough money 3. Construction 4. Interior … … 1.a Drive to the city call 1.b Fill out an application 2.a Drive to money 2.b Fill out an application 3.a Find a builder 3.b Sign a contract 3.c Build foundation 3.d Build frame 3.e Build roof 3.f build wall …

14. 14 Abstraction State Space truck’s DTG package’s DTG full state space Abstraction loc1loc2 1.LOAD pkg truck loc1 2.UNLOAD pkg truck loc2 1.a MOVE truck loc2 loc1 1.b LOAD pkg truck loc1 2.a MOVE truck loc1 loc2 2.b UNLOAD pkg truck loc2 in truck at loc1at loc2

15. 15 Abstraction State Space Search (Iteration #1) ………… ………… Goal 1 Goal 2 Abstraction State Space DTG Finds a plan quickly

16. 16 Abstraction State Space Search (Iteration #2) ………… ………… G1G1 G2G2 Abstraction State Space DTG Keep running to improve the plan quality

17. Results InstancesAvg. TimeAvg. Plan Length DTG-Plan Fast Downward DTG-Plan Fast Downward DTG-Plan Fast Downward Openstack28 3.3910.91131.6131.5 Pathways30 1.692.90166.3132.8 Rovers40 12.059.69151.9112.2 Storage17180.581.7815.914.6 TPP30 9.1647.16172.2133.1 Trucks161019.1953.0431.5028.4  The average running time and plan length are calculated by the instances solved by both algorithm.  Here DTG-Plan runs for a single iteration, more iterations will improve the solution quality.

18. Results (Rovers Domain) Time FastDownward Quality Quality Iter#1Iter#2Iter#3 Iter#1Iter#2Iter#3 Rover30.090.2120.209151813 Rover40.090.2180.20981388 Rover50.110.3082.2363327 22 Rover60.140.70715.8963846 36 Rover70.130.370.42823362018 Rover80.230.6569.9826363129 Rover90.230.82419.06836544240 Rover100.251.52355.2583741 38 Rover110.280.9734.42156714636 Rover120.240.6520.91221322419 Rover130.416.579120.91281805856 Rover140.30.9124.7632453533 Rover150.411.55927.12744494442 Rover160.483.56428.486517746 Rover170.7917.775366.275269 48 Rover181.374.792 516.72475551 47 Downward has a similar running time to Iter#1

19. Outline Automated Planning Classical Planning Temporally Expressive Planning STRIPS SAS+ STRIPS Search SAT Search SAT Search SAT 2  Enhanced mutual exclusion [Chen, Huang, Xing, Zhang: 09]

20. 20 Planning as Satisfiability Given planning problem P : Encode P with time N, resulting in SAT formula E N Use a SAT solver to solve E N E N is solvable? N = 0 Decode and output solution Y N N = N + 1 Encoding is the key!

21. Strips Based Encoding (at pkg loc1) (at truck loc2) move (at pkg loc1) (at truck loc2) (at truck loc1) Time step 1Time step 2 (at pkg loc1) move (at truck loc2) (at truck loc1) load (in pkg truck) ………………………… Time step 3,4,5,… Compile from planning graph 21

22.  Two facts/actions cannot be true at the same time  Fact mutual exclusion  Action mutual exclusion Mutual Exclusion (Mutex) Move-truck-loc1-loc2 Move-truck-loc2-loc1 (in pkg truck) (at truck loc1) (at truck loc2) (at pkg loc1) (at pkg loc2) (in pkg truck) (at truck loc1) (at truck loc2) (at pkg loc1) (at pkg loc2) A key reason for the efficiency of SAT-based planning 22

23. 23 Mutual Exclusion Time #2 Time #3 #4 #5 #6 #7 #8 #9#10 Time #1 Invalid (at truck loc1)(at truck loc2)

24. 24 Long Distance Mutual Exclusion(Londex) Time #2 Time #3 #4 #5 #6 #7 #8 #9#10 Time#1 Invalid (at truck loc1) (at truck loc4) DTG of truck (at truck loc1) (at truck loc2) (at truck loc3)(at truck loc4)

25. InstanceSat/ Unsat MethodSize#Propagation#DecisionTime (sec.) Zenotravel13 With time=5 unsatSATPlan044.60E+053.59E+075.47E+05369.76 SATPlan04+Londex2.53E+062.10E+053.15E+021.37 Depots14 with Time=10 unsatSATPlan043.39E+063.86E+073.24E+05103.75 SATPlan04+Londex1.51E+071.05E+069.89E+029.19 Driverlog12 with Time=10 unsatSATPlan041.05E+053.79E+073.99E+05225.14 SATPlan04+Londex2.51E+064.01E+056.91E+021.37 Satellite9 with Time=5 unsatSATPlan041.17E+042.33E+074.03E+0569.84 SATPlan04+Londex1.21E+059.00E+063.83E+049.37 Pruning Power of Londex 25

26. Outline Automated Planning Classical Planning Temporally Expressive Planning STRIPS SAS+ STRIPS Search SAT Search SAT Search SAT 3  A SAS+ based encoding scheme [Huang,Chen,Zhang:10]

27. STRIPS versus SAS+ STRIPSSAS+ Definition a set of preconditions, a set of add effects, a set of delete effects A set of transitions Example (LOAD pkg truck loc1) Pre: (at truck loc1), (at pkg loc1)pkg:(loc1  truck) truck: (loc1  loc1) Del:(at pkg loc1) Add:(in pkg truck) Usually there are fewer transitions than actions Hierarchical relationships between actions and transitions Different Representations of Actions: 27

28. Overview of New Encoding …………………… SAT Instance (Part 1) : transitions SAT Instance (Part 2) : matching actions and transitions (multiple independent ones) …… … Transitions Actions t = 1t =2t =3 SAT Instance : Facts and actions Actions Planning graph … …… t = 1t =2t =3, 4, … Strips Based Encoding SAS+ Based New Encoding 28

29. 29 Clauses in New Encoding, Example truck:loc2 Time step 1Time step 2 ………………………… Time step 3,4,5,… pkg: loc1 truck:loc1 truck:loc2 truck:loc1 truck:loc2 pkg: loc1 pkg: truck pkg: loc2 pkg: loc1 pkg: truck pkg: loc2 Find matchings set of actions … pkg: truck pkg: loc2 1.Progression of transitions over time steps ( blue one implies green ones ) 2.Initial state and goal (Bold ones) 3.Matching actions and transitions 4.Action mutual exclusions and transition mutual exclusions

30. State Space Size …………………… State Space (Part 1) : O((2 T ) N ) State Space (Part 2) : O(NK) …… … Transitions Actions t = 1t =2t =3 State Space: O((2 A ) N ) Actions Planning graph … …… t = 1t =2t =3, 4, … Strips Based Encoding SAS+ Based New Encoding 30 Total: O((2 T ) N NK) K KK Usually there are fewer transitions than actions A Number of actions N Number of Time Step T Number of transitions

31. Number of Solvable Instances SATPlan06 has a similar performance to SatPlan04+Londex 31

32. Detailed Results SatPlan06New Encoding Instances Time (sec) #Variables#Clauses Size (MB) Time#Variables#ClausesSize Airport402239.4327,51513,206,595807583.3396,212 3,339,914 208 Driverslog 17 2164.861,9152,752,787183544.174,680812,31256 Freecell4364.3175826,114,100392158.426,009371,20725 Openstack4212.13,70966,744533.64,88920,0222 Pipesworld123147.330,07813,562,157854543.743,528634,87344 TPP303589.797,1557,431,0624621844.8136,106997,17770 Trucks71076.021,745396,58127245.735,065255,02018 Zeno14728.426,2016,632,92342158.717,459315,71918 32 Better performances in 10 domains out of 11 tested (from IPC3,4,5)

33. Outline Automated Planning Classical Planning Temporally Expressive Planning STRIPS SAS+ STRIPS Search SAT Search SAT Search SAT 4  An Optimal Temporally Expressive Planner [Huang,Chen,Zhang:09]

34.  Actions have durations Temporal Planning Light a Match find a manual e.g. got a blackout: Repair a fuse Time

35.  Temporally Expressive (Required Concurrency)  Two actions need to be executed in parallel, in any valid plan. Temporally Expressive [W. Cushing et al. 2007] adds fact (has light) Light a match deletes fact (has light) Time Read Manual condition requires condition (has light) 35  A planning algorithm is temporally expressive if it can handle required concurrency, otherwise it is temporally simple

36.  Required Concurrency is important in the real world  PDDL2.1 supports temporally expressiveness  e.g. Firework, Missile launch  Most state of the art temporal planners are not able to handle temporally expressive  Crikey [Coles:08] is the only capable planner (partially)  Heuristic state space search based  Incomplete, especially when a lot of concurreis Facts about State-of-the-art 36

37.  Discrete time  Contributions:  A novel transformation scheme  A simplified planning problem and  Additional constraints enforcing the concurrencies  Optimal (in terms of overall time)  Enables high concurrency of actions SAT-Based Temporally Expressive Planning 37

38. Experimental results on the Match-lift domain. The numbers in Columns L, M, R, U represent the numbers of floors, matches, rooms and fuses. Results – Matchlift Domain [Coles08] Degree of concurrency is low.

39. Experimental results on Match-Lift Variant, which has more concurrencies. The numbers in Columns E, M, U represent the numbers of electrician, matches, and fuses. Results (Matchlift with more concurrencies) 39

40. Experimental results on the P2P domain. Column ‘P’ is the instance ID. Columns ‘C’ and ‘F’ are the numbers of peers and files, respectively, in the network. Columns ‘Time’ and ‘QAL’ are the running time and time span of solutions, respectively. ‘T’ means the solver ran out of the time limit of 3600s and ‘-’ means no solution found. Results – P2P Domain 40

41.  Search based planner can be very efficiently when the problems are of lower degree of required concurrencies Comparison of Search and SAT  Search based incomplete planners fail on most high degree of concurrency problems action1 a b action2 action1 a d action2 b c e f g e.g. original Match-lift domain e.g. P2P and Match-lift domain with more required concurrencies 41

42. Proposed Task #1 Automated Planning Classical Planning Temporally Expressive Planning STRIPS SAS+ STRIPS Search SAT Search SAT Search SAT General SAT solving

43.  Our new SAS+ based encoding scheme is efficient and compact  Thanks to the problem structure  Can we detect similar structures in general SAT instances?  Need to determine the conditions  Need to devise a greedy algorithms Proposed Task #1 43

44. Proposed Task #2 Automated Planning Classical Planning Temporally Expressive Planning STRIPS SAS+ STRIPS Search SAT Search SAT Search SAT General SAT solving POCL POCL = Partial Order Causal Link POCL

45.  Partial Order Causal Link Planning [Weld:97] Proposed Task #2 Search (State space)POCL (Plan Space) State A set of factsA set of actions and partial orders between them Goal state A state with goal factsA set of actions and constraints, with no open goal or broken link Successors Apply an actionRepair a flaw (i.e. either an “ open goal ” or a “ broken link ” )  Use SAS+ to recast POCL planning  Results in a new transition space  Need good heuristics 45

46. Proposed Task#3 Automated Planning Classical Planning Temporally Expressive Planning STRIPS SAS+ STRIPS Search SAT Search SAT Search SAT General SAT solving POCL POCL = Partial Order Causal Link POCL SAS+ SAT

47.  Our SAT-based temporally expressive planner  The encoding size is usually too large  Our new encoding scheme for classical planning  One major merit is its compactness  Apply the new encoding scheme to the SAT-based temporally expressive planner  Need a SAS+ style temporal planning formulation  Need a parser from PDDL2.1 to a SAS+ style temporal planning formulation  Need to develop a new encoding scheme Proposed Task #3 47

48. Time Line of Proposed Works Task 1Task 2Task 3 Nov 2010 – Dec 2010 Study the conditions lead to the desired properties. In the meanwhile, study the feasibility of transition space planning. n/a Jan 2011 Verify the properties in existing SAT benchmark instances Feb 2011- Apr 2011 Exclusively, Priority: Task 1 > Task 2> Task 3 Devise and implement a greedy algorithm to pre- process general SAT instances by making use of the desired problem structure. Implement a transition space planning algorithm. Implement a new transition based encoding scheme for temporally expressive planning. May 2011- Jul 2011 Dissertation writing and defense. 48 Finish at least one of the proposed tasks.

49.  R. Huang, Y. Chen, and W. Zhang, “ A Novel Transition Based Encoding Scheme for Planning as Satisfiability ”. AAAI’10  R. Huang, Y. Chen, and W. Zhang, “ An Optimal Temporally Expressive Planner: Initial Results and Application to P2P Network Optimization ”. ICAPS’09  Y. Chen, R. Huang, Z. Xing and W. Zhang, “ Long-distance Mutual Exclusion for Planning ”. Artificial Intelligence Journal, 2009  Y. Chen, R. Huang, and W. Zhang, “ Fast Planning by Search in Domain Transition Graphs ”. AAAI’08  C. Hsu, B. Wah, R. Huang, and Y. Chen, “ Constraint Partitioning for Solving Planning Problems with Trajectory Constraints and Goal Preferences ”. IJCAI’07 Publications

50. Questions?

51. 51 Time Fast-Downward Plan Length Level1Level2Iteration#1Iteration#2 TPP10.030.086555 TPP20.030.08610 8 TPP30.040.098131511 TPP40.050.111182014 TPP50.070.184233523 TPP60.141.146355129 TPP70.154.108487238 TPP80.177.762628645 TPP90.2934.329809258 TPP100.3429.76110010664 TPP110.4248.7169111769 TPP120.4947.9888911575 TPP130.8368.622719862 TPP140.92127.71710511095 TPP151.04280.156113127106 TPP162.07699.186146169146 TPP172.791077.613135170118 TPP183.2 n/a122147 n/a Experimental Results (TPP Domain)

52. 52 Abstraction State Space truck’s DTG package’s DTG full state space loc1 loc2 Abstraction Smaller branching factor Actions to expand: Full state spaceAbstraction state space LOAD pkg truck loc2 MOVE truck loc1 loc2 … LOAD pkg truck loc2

53. Strips v.s. SAS+ Based Encodings StripsSAS+ Variables  Actions and Facts  Actions and Transitions Clauses  Logics of actions across time steps, subject to initial state and goal ( O((2 A ) N ))  Logics of transitions across time steps, subject to initial state and goal ( O((2 T ) N ) ) T is much smaller than A  Logics of finding a matching action set for transitions, in each time step t ( K ) N small independent matching problems Exact Cover problem [Karp72] Worst case state space size: O((2 A ) N ) Worst case state space size: O((2 T ) N NK) N, T, A: number of time steps, transitions and actions 53

54. Strips Based Encoding (at pkg loc1) (at truck loc2) move (at pkg loc1) (at truck loc2) (at truck loc1) Time step 1Time step 2 (at pkg loc1) move (at truck loc2) (at truck loc1) load (in pkg truck) ………………………… Time step 3,4,5,… Compile from planning graph 54 Iteration 1 Iteration 2 Iteration 3 Iteration 4,5, …