1. CPSC 322, Lecture 18Slide 1 Planning: Heuristics and CSP Planning Jim Little UBC CS 322 October 15, 2014 Textbook §8

2. CPSC 322, Lecture 18Slide 2 Lecture Overview Recap: Planning Representation and Forward algorithm Heuristics CSP Planning

3. CPSC 322, Lecture 2Slide 3 Modules we'll cover in this course: R&Rsys Environment Problem Query Planning Deterministic Stochastic Search Arc Consistency Search Value Iteration Var. Elimination Constraint Satisfaction Logics STRIPS Belief Nets Vars + Constraints Decision Nets Markov Processes Var. Elimination Static Sequential Representation Reasoning Technique SLS

4. CPSC 322, Lecture 11Slide 4 Standard Search vs. Specific R&R systems Constraint Satisfaction (Problems): State: assignments of values to a subset of the variables Successor function: assign values to a “free” variable Goal test: set of constraints Solution: possible world that satisfies the constraints Heuristic function: none (all solutions at the same distance from start) Planning : State? Successor function? Goal test? Solution? Heuristic function…. Inference State Successor function Goal test Solution Heuristic function

5. CPSC 322, Lecture 18Slide 5 Lecture Overview Recap: Planning Representation and Forward algorithm Heuristics for forward planning CSP Planning

6. CPSC 322, Lecture 18Slide 6 Heuristics for Forward Planning Heuristic function: estimate of the distance from a state to the goal. In planning the distance is? Two simplifications in the representation: All features are binary: T / F Goals and preconditions can only be assignments to T And a Def. a subgoal is a particular assignment in the goal e.g., if the goal is then…. number of actions remaining from the current state to the goal. a subgoal is some subset of the assignments

7. CPSC 322, Lecture 18Slide 7 Heuristics for Forward Planning: Any ideas? State 1 A=T B=F C=F D=T … State 2 A=T B=F C=T D=F … Goal A=T B=T C=T

8. CPSC 322, Lecture 18Slide 8 Heuristics for Forward Planning (cont’) What kind of simplifications of the actions would justify our proposal for h? a) We have removed all ……………. b) We have removed all ……………. c) We assume no action can achieve………………….. State 1 A=T B=F C=F D=T … h(s1)=2 Goal A=T B=T C=T a1 a2

9. CPSC 322, Lecture 18Slide 9 Heuristics for Forward Planning: empty-delete-list We only relax the problem according to remove delete lists: i.e., we remove all the effects that make a variable F Action a effects (B=F, C=T) But then how do we compute the heuristic? This is often fast enough to be worthwhile empty-delete-list heuristics with forward planning is currently considered a very successful strategy Solve the relaxed problem

10. Heuristics for Forward Planning: One way : remove all the effects that make a variable = F. If we find the path from the initial state to the goal using this relaxed version of the actions: the length of the solution is an underestimate of the actual solution length Why? In the original problem, one action might undo an already achieved goal Action a effects (B=F, C=T) S1 A = T B = F C = F Goal A = T B = T C = T a1 B = T a C = T B = F Finding a good heuristics is what makes forward planning feasible in practice –Factored representation of states and actions allows for definition of domain-independent heuristics Slide 10

11. Informed Forward Planning: Final Comment You should view (informed) Forward Planning as one of the basic planning techniques By itself, it cannot go far, but it can work very well in combination with other techniques, for specific domains See, for instance, descriptions of competing planners in the presentation of results for the 2008 planning competition (posted in the class schedule)

12. CPSC 322, Lecture 18Slide 12 Lecture Overview Recap: Planning Representation and Forward algorithm Heuristics for forward planning CSP Planning

13. CPSC 322, Lecture 18Slide 13 Planning as a CSP An alternative approach to planning is to set up a planning problem as a CSP! We simply reformulate a STRIPS model as a set of variables and constraints Once this is done we can even express additional aspects of our problem (as additional constraints) e.g., see Practice Exercise UBC commuting “careAboutEnvironment” constraint

14. Slide 14 Planning as a CSP: Variables We need to “unroll the plan” for a fixed number of steps: this is called the horizon To do this with a horizon of k: construct a CSP variable for each STRIPS variable at each time step from 0 to k construct a boolean CSP variable for each STRIPS action at each time step from 0 to k - 1.

15. CPSC 322, Lecture 18Slide 15 CSP Planning: Robot Example Variables for actions …. action (non) occurring at that step

16. CPSC 322, Lecture 18Slide 16 CSP Planning: Initial and Goal Constraints initial state constraints constrain the state variables at time 0 goal constraints constrain the state variables at time k

17. CPSC 322, Lecture 18Slide 17 CSP Planning: Prec. Constraints As usual, we have to express the preconditions and effects of actions: precondition constraints hold between state variables at time t and action variables at time t specify when actions may be taken PUC 0 RLoc 0 RHC 0 PUC 0 csTF FT FF mr*F lab*F off*F

18. CSP Planning: Precondition Constraints precondition constraints between state variables at time t and action variables at time t specify when actions may be taken E.g. robot can only pick up coffee when Loc=cs (coffee shop) and RHC = false (don’t have coffee already) RLoc 0 RHC 0 PUC 0 csT F F mr* lab* off* Need to allow for the option of *not* taking an action even when it is valid F F F F F T Truth table for this constraint: list allowed combinations of values Slide 18

19. CPSC 322, Lecture 18Slide 19 CSP Planning: Effect Constraints effect constraints between state variables at time t, action variables at time t and state variables at time t + 1 explain how a state variable at time t + 1 is affected by the action(s) taken at time t and by its own value at time t RHC i DelC i PUC i RHC i+1 TTTT TTFF TFTT ………… …………

20. CSP Planning: Effect Constraints Effect constraints Between action variables at time t and state variables at time t+1 Specify the effects of an action Also depends on state variables at time t (frame rule!) E.g. let’s consider RHC at time t and t+1 Let’s fill in a few rows in this table RHC t DelC i PUC i RHC t+1 TTT T TTF F TFT T TFF T FTT T FTF F FFT T FFF F Does not quite matter what we put here since other constraints enforce that these configurations won’t happen Slide 20

21. CSP Planning: Constraints Contd. Other constraints we may want are action constraints: specify which actions cannot occur simultaneously these are sometimes called mutual exclusion (mutex) constraints DelM i DelC i ??

22. Additional constraints in CSP Planning Other constraints we may want are action constraints: specify which actions cannot occur simultaneously these are often called mutual exclusion (mutex) constraints DelM i DelC i E.g. in the Robot domain DelM and DelC can occur in any sequence (or simultaneously) But we can enforce that they do not happen simultaneously TF TF FF Slide 22

23. CPSC 322, Lecture 18Slide 23 CSP Planning: Constraints Contd. Other constraints we may want are state constraints hold between state variables at the same time step they can capture physical constraints of the system (robot cannot hold coffee and mail) they can encode maintenance goals RHC i RHM i

24. Handling mutex constraints in Forward Planning If we don’t want DelM and DelC to occur simultaneously How would we encode this into STRIPS for forward planning? DelM i DelC i TF TF FF Slide 24

25. Additional constraints in CSP Planning Other constraints we may want are state constraints hold between variables at the same time step they can capture physical constraints of the system (e.g., robot cannot hold coffee and mail) RHC i RHM i TF FT FF Slide 25

26. Handling state constraints in Forward Planning RHC i RHM i TF FT FF How could we handle these constraints in STRIPS for forward planning? Slide 26

27. CSP Planning: Solving the problem 27 Map STRIPS Representation for horizon 1, 2, 3, …, until solution found Run arc consistency and search! k = 0 Is State 0 a goal? If yes, DONE! If no,

28. CSP Planning: Solving the problem 28 Map STRIPS Representation for horizon k =1 Run arc consistency and search! k = 1 Is State 1 a goal If yes, DONE! If no,

29. CSP Planning: Solving the problem 29 Map STRIPS Representation for horizon k = 2 Run arc consistency, search! k = 2: Is State 2 a goal If yes, DONE! If no….continue

30. CPSC 322, Lecture 18Slide 30 CSP Planning: Solving the problem Map STRIPS Representation for horizon: Run arc consistency and search, In order to find a plan, we expand our constraint network one layer at the time, until a solution is found Plan: all actions with assignment T

31. Solve planning as CSP: pseudo code solved = false horizon = 0 While solved = false map STRIPS into CSP with horizon solve CSP -> solution if solution then solved = T else horizon = horizon + 1 Return solution CPSC 322, Lecture 18Slide 31

32. Solving Planning as CSP: pseudo code solved = false for horizon h=0,1,2,… map STRIPS into a CSP csp with horizon h solve that csp if solution exists then return solution else horizon = horizon + 1 end Slide 32

33. CPSC 322, Lecture 18Slide 33 State of the art planner: GRAPHPLAN A similar process is implemented (more efficiently) in the Graphplan planner

34. STRIPS to CSP applet CPSC 322, Lecture 6Slide 34 Allows you: to specify a planning problem in STRIPS to map it into a CSP for a given horizon the CSP translation is automatically loaded into the CSP applet where it can be solved Practice exercise using STRIPS to CSP is available on AIspace

35. You know the key ideas! Ghallab, Nau, and Traverso Automated Planning: Theory and Practice Web site: http://www.laas.fr/planning Also has lecture notes You know You know a little Slide 35

36. West North East South 6 2 8 Q Q  J 6  5  9  7  A K 5 3 A  9   Some applications of planning Emergency Evacuation Robotics Space Exploration Manufacturing Analysis Games (e.g., Bridge)? Generating Natural language Product Recommendations …. Slide 36

37. CPSC 322, Lecture 4Slide 37 Learning Goals for today’s class You can: Construct and justify a heuristic function for forward planning. Translate a planning problem represented in STRIPS into a corresponding CSP problem (and vice versa) Solve a planning problem with CPS by expanding the horizon (new one)

38. CPSC 322, Lecture 2Slide 38 What is coming next ? Environment Problem Inference Planning Deterministic Stochastic Search Arc Consistency Search Value Iteration Var. Elimination Constraint Satisfaction Logics STRIPS Belief Nets Vars + Constraints Decision Nets Markov Processes Var. Elimination Static Sequential Representation Reasoning Technique SLS Textbook Chpt 5.1- 5.1.1 – 5.2