Planning with Loops Some New Results Yuxiao (Toby) Hu and Hector Levesque University of Toronto

2 Outline Introduction Representing plans with loops –FSA plan: a type of finite state controller Constructing plans with loops –Search in the space of FSA plans Potential for improvements Conclusion

3 The Planning Problem Finitely many functional fluents, f 1,...,f m, whose domain may be finite or infinite; Finitely many actions, a 1,...,a n (world- changing or sensing); Initial state: the possible values of each fluent; Goal: achieve some goal condition in all contingencies. Like contingent planning: solve a class of problems Incomplete initial state with possibly infinite cases

4 Motivating Example (a variant of striped-tower in Srivastava et al. 2008) Fluents: –stackA (a list of block colors) –stackB (a list of block colors) –stackC (a list of block colors) –hand (empty/red/blue) World changing actions: –pickA, pickB, putB, putC; Sensing actions –testA? (empty/nonempty) –testB? (empty/nonempty) –testH? (red/blue)

5 Motivating Example (a variant of striped-tower in Srivastava et al. 2008) Initially: –stackA=[blue,red,red,blue] –stackB=[ ] –stackC=[ ] –hand=empty Goal: –stackA=[ ] –stackB=[ ] –striped(stackC) –hand=empty striped(X) is true iff X=[red,blue,…,red,blue]

6 Example 1 A linear solution 1.pickA; 2.putB; 3.pickA; 4.putC; 5.pickB; 6.putC; 7.pickA; 8.putC; 9.pickA 10.putC.

7 Example 2

8 Example 2 (cont.) pickA; CASE testH? OF - red: putC; pickA; CASE testH? OF -red: putB; blue: putC; blue: putB; pickA; CASE testH? OF -red: putC; blue: putB;......

9 Example 3 Question: Is there a generalized plan solving all problems in this class?

10 Some of the Existing Approaches KPLANNER (Levesque 2005) generates robot programs by winding found conditional plans. Aranda (Srivastava et al. 2008) obtains generalized plans by winding an abstracted example plan. loopDISTILL (Winner and Veloso 2007) learns a “dsPlanner” by merging matching sub- plans of an example partial-order plan.

11 Outline Introduction Representing plans with loops –FSA plan: a type of finite state controller Constructing plans with loops –Search in the space of FSA plans Potential for improvements Conclusion

12 Plan Representation – Robot Programs A robot program is defined inductively by –nil; –seq(A,P); –case(A,[R 1 :P 1,…,R n :P n ]); –loop(P,Q). (Levesque 1996, 2005) KPLANNER (Levesque 2005) uses the robot program representation.

13 Robot Program – Some Examples pickA; CASE testH? OF - red: putC; pickA; CASE testH? OF -red: putB; blue: putC; blue: putB; pickA; CASE testH? OF -red: putC; blue: putB; pickA; putB; pickA; putC; pickB; putC; pickA; putC; pickA putC.

14 Plan Representation An FSA plan is a directed graph Each node represents a program state o One unique "start" state o One unique "final" state o Non-final state associated with an action Each edge is associated with a sensing result o Sensing result of world-changing actions can be omitted

15 Plan Execution (for a single complete initial world) 1.Use the "start state" as current program state; 2.If current state is the "final state", then stop; 3.Execute action associated to the current state; 4.Follow the edge with returned sensing result; 5.Make the node pointed to by this edge the current program state, and repeat from Step 2.

16 Robot Program vs. FSA Plan It can be shown that all robot programs can be represented by equivalent FSA plans. What about the reverse direction?

17 Robot Program vs. FSA Plan feel? thirsty drink hungry eat go2bed sleep CASE feel? OF - thirsty: drink; go2bed; sleep; - hungry: eat; go2bed; sleep

18 Robot Program vs. FSA Plan get? instruction suggestion follow? think? succeed fail workable unworkableok revise? fail

19 Outline Motivation Representing plans with loops –FSA plan: a type of finite state controller Constructing plans with loops –Search in the space of FSA plans Potential for improvements Conclusion

20 Generating Plans with Loops 1.Start with the smallest FSA plan with only one non- final state. 2.If the current program state is final, the goal must be satisfied. 3.Otherwise, execute the action associated to the current program state, non-deterministically pick an applicable one if none is associated. 4.For each possible sensing result of the action, follow the transition and change the current state to the transition target. If no transition is associated to the sensing result, non-deterministically pick one for it. 5.Repeat from step 2.

21 Search in the Space of FSA Plans … (Possible values of stackA)

22 Search in the Space of FSA Plans …

23 Search in the Space of FSA Plans pickA …

24 Search in the Space of FSA Plans testA? … empty

25 Search in the Space of FSA Plans testA? empty nonempty …

26 Search in the Space of FSA Plans testA? empty nonempty …

27 Search in the Space of FSA Plans testA? empty nonempty … pickB

28 Search in the Space of FSA Plans testA? empty nonempty … testA?

29 Search in the Space of FSA Plans testA? empty nonempty … pickA

30 Search in the Space of FSA Plans testA? empty nonempty … pickA

31 Search in the Space of FSA Plans testA? empty nonempty … pickA

32 Search in the Space of FSA Plans testA? empty nonempty … pickA

33 Search in the Space of FSA Plans testA? empty nonempty pickA

And finally…

35 Experimental Results (Using the same pruning rules.)

36 Experimental Results Statistics for Aranda are estimation from figures in (Srivastava et al. 08) without redoing their experiments.

37 Potential for Improvements We have formulated planning with loops as a search problem, and obtained a baseline implementation. It is using blind depth-first (iterative deepening) search, and does not scale well without effective pruning rules. However, the baseline implementation is a good starting point for adding heuristics and trying other improvement.

38 Heuristics for Action Selection We tried a variant of the additive heuristics (Bonet and Geffner 2001): –Assume all fluents are independent; –Count the number of action steps to change each fluent to a value that may satisfy the goal; –Sum of steps across all fluents used as goal distance; –Try successor states with shorter distance first

39 Heuristics for Action Selection Appear relatively effective, but still limited: –The domain of each fluent may be infinite, so exponential BFS was used; –There are two non-deterministic choices in the search algorithm (choice of action and choice of transition). Greedily improving only one does not always lead to a good decision.

40 Discussion Planning with loops is an interesting problem We formally defined FSA plans, representation for loopy plans. [And a logical account for planning problems, FSA plans and correctness.] We formulate planning with loops as a search problem. FSA planner, the baseline implementation, outperforms KPLANNER, and is good starting point for heuristics and other improvements.