1. Computer Science CPSC 322 Lecture 12 SLS wrap up, Forward Planning, Planning as CSP (Ch 8.2, 8.4) Slide 1

2. Announcements Material for midterm available in Connect 1.List of Learning Goals (posted) 2.Short questions on material (posted) 3.Sample problem-solving questions (will post by early next week) Material covered Until Forward Planning included (covered today) See corresponding learning goals and short questions on Connect Midterm will be close textbook, no calculator or other devices - Part short questions similar or even verbatim from the list posted in connect - Part more problem-solving style questions Possibly a regular individual exam followed by a group exam on the same test (I need to work out if I can fit this format in the time that we have)

3. Lecture Overview Recap of Lecture 11 Forward Planning Heuristics for Forward Planning Intro to Planning as CSP

4. Indiv. Exam Collect Group Exam (same or subset of Indiv. Exam) Form Groups

5. Course Overview Environment Problem Type Query Planning Deterministic Stochastic Constraint Satisfaction Search Arc Consistency Search Logics STRIPS Vars + Constraints Value Iteration Variable Elimination Belief Nets Decision Nets Markov Processes Static Sequential Representation Reasoning Technique Variable Elimination We’ll focus on Planning

6. Goal Description of states of the world Description of available actions => when each action can be applied and what its effects are Planning: build a sequence of actions that, if executed, takes the agent from the current state to a state that achieves the goal Planning Problem

7. Standard Search vs. Specific R&R systems Constraint Satisfaction (Problems): State: assignments of values to a subset of the variables Successor function: assign values to a “free” variable Goal test: all variables assigned a value and all constraints satisfied? Solution: possible world that satisfies the constraints Heuristic function: none (all solutions at the same distance from start) Planning : State Successor function Goal test Solution Heuristic function Inference State Successor function Goal test Solution Heuristic function

8. “Open-up” the representation of states, goals and actions –States and goals as set of features –STRIPS representation for actions: two parts: 1. Preconditions: a set of assignments to features that must be satisfied in order for the action to be applicable in a state 2. Effects: a set of assignments to features that are caused by the action Key Idea of Planning

9. Delivery Robot Example: features RLoc - Rob's location Domain: {coffee shop, Sam's office, mail room, laboratory} short {cs, off, mr, lab} RHC – Rob has coffee Domain: {true, false}. Alternatively notation: rhc indicate that Rob has coffee, and by that Rob doesn’t have coffee SWC – Sam wants coffee {true, false} MW – Mail is waiting {true, false} RHM – Rob has mail {true, false} An example state is 64 possible states in total

10. Delivery Robot Example: Actions The robot’s actions are: Move - Rob's move action move clockwise (mc ), move anti-clockwise (mac ) PUC - Rob picks up coffee must be at the coffee shop and not have coffee DelC - Rob delivers coffee must be at the office, and must have coffee PUM - Rob picks up mail must be in the mail room, and mail must be waiting DelM - Rob delivers mail must be at the office and have mail Preconditions for action application

11. STRIPS actions: Example STRIPS representation of the action pick up coffee, PUC: preconditions Loc = cs and RHC = F effects RHC = T STRIPS representation of the action deliver coffee, Del : preconditions Loc = off and RHC = T effects RHC = F and SWC = F STRIPS representation of the action move clockwise from coffee shop, mc-cs preconditions: Loc = cs effects: Loc = off

12. STRIPS Actions (cont’) The STRIPS assumption: all features not explicitly changed by an action stay unchanged So if the feature V has value v i in state S i, after action a has been performed, what can we conclude about a and/or the state of the world S i-1 immediately preceding the execution of a? S i-1 V = v i SiSi a C. At least one of A and B A. V = v i was TRUE in S i-1 B.One of the effects of a is to set V = v i

13. STRIPS lends itself to solve planning problems either As pure search problems As CSP problems We will look at one technique for each approach Solving planning problems Slide 13

14. Lecture Overview Recap of Lecture 11 Forward Planning Heuristics for Forward Planning Intro to Planning as CSP

15. Forward planning To find a plan, a solution : search in the state-space graph The states are the possible worlds full assignments of values to features The arcs from a state s represent all the actions that are possible in state s A plan is a path from the state representing the initial state to a state that satisfies the goal Which actions a are possible in a state s? 15

16. Forward planning To find a plan, a solution : search in the state-space graph The states are the possible worlds full assignments of values to features The arcs from a state s represent all the actions that are possible in state s A plan is a path from the state representing the initial state to a state that satisfies the goal Which actions a are possible in a state s? C. Those where the state s’ reached via a is on the way to the goal A. Those where a’s effects are satisfied in s B. Those where a’s preconditions are satisfied in s C. Both A and B 16

17. Forward planning To find a plan, a solution : search in the state-space graph The states are the possible worlds full assignments of values to features The arcs from a state s represent all the actions that are possible in state s A plan is a path from the state representing the initial state to a state that satisfies the goal Which actions a are possible in a state s? B. Those where a’s preconditions are satisfied in s 17

18. Example state-space graph: first level Goal: puc mc mcc mcc: move counterclockwise 18

19. Example for state space graph Goal: a sequence of actions that gets us from the start to a goal Solution: What is a solution to this planning problem? 19

20. Example for state space graph What is a solution to this planning problem? Goal: B (puc, mc, mc) C (puc, dc) A (puc, mc) D (puc, mc, dc) a sequence of actions that gets us from the start to a goal Solution: 20

21. Example for state space graph What is a solution to this planning problem? Goal: D (puc, mc, dc) a sequence of actions that gets us from the start to a goal Solution: 21

22. Standard Search vs. Specific R&R systems Constraint Satisfaction (Problems): State: assignments of values to a subset of the variables Successor function: assign values to a “free” variable Goal test: set of constraints Solution: possible world that satisfies the constraints Heuristic function: none (all solutions at the same distance from start) Planning : State: full assignment of values to features Successor function: states reachable by applying actions with preconditions satisfied in the current state Goal test: partial assignment of values to features Solution: a sequence of actions Heuristic function Inference State Successor function Goal test Solution Heuristic function 22

23. Forward Planning Any of the search algorithms we have seen can be used in Forward Planning Problem? Complexity is defined by the branching factor, e.g. 23 C. Average number of preconditions in the actions applicable in a state A. Number of actions defined in the planning problem B. Number of actions applicable in a state D. Average number of effects in the actions applicable in a state

24. Forward Planning Any of the search algorithms we have seen can be used in Forward Planning Problem? Complexity is defined by the branching factor, e.g.… Can be very large Solution? 24

25. Forward Planning Any of the search algorithms we have seen can be used in Forward Planning Problem? Complexity is defined by the branching factor, e.g.… Number of applicable actions to a state Can be very large Solution? 25

26. Standard Search vs. Specific R&R systems Constraint Satisfaction (Problems): State: assignments of values to a subset of the variables Successor function: assign values to a “free” variable Goal test: set of constraints Solution: possible world that satisfies the constraints Heuristic function: none (all solutions at the same distance from start) Planning : State: full assignment of values to features Successor function: states reachable by applying actions with preconditions satisfied in the current state Goal test: partial assignment of values to features Solution: a sequence of actions Heuristic function Inference State Successor function Goal test Solution Heuristic function 26

27. Lecture Overview Recap of Lecture 11 Forward Planning Heuristics for Forward Planning Intro to Planning as CSP

28. Heuristics for Forward Planning Not in textbook, but you can see details in Russel&Norvig, 10.3.2 Heuristic function: estimate of the distance from a state to the goal In planning this is the………………. 28

29. Heuristics for Forward Planning Not in textbook, but you can see details in Russel&Norvig, 10.3.2 Heuristic function: estimate of the distance from a state to the goal In planning this is the………………. B. # of actions needed to get from s to the goal C. # of legal actions in s A. # of goal features not true in s 29

30. Heuristics for Forward Planning Not in textbook, but you can see details in Russel&Norvig, 10.3.2 Heuristic function: estimate of the distance from a state to the goal In planning this distance is the………………. Finding a good heuristics is what makes forward planning feasible in practice Factored representation of states and actions allows for definition of domain-independent heuristics We will look at one example of such domain-independent heuristic that has proven to be quite successful in practice B.# of actions needed to get from s to the goal 30

31. Heuristics for Forward Planning: We make two simplifications in the STRIPS representation All features are binary: T / F Goals and preconditions can only be assignments to T e.g. positive assertions Definition: a subgoal is the specific assignment for one of the features in the goal e.g., if the goal is then…. S1 A = T B = F C = F Goal A = T B = T C = T Slide 31

32. Heuristics for Forward Planning: ignore delete-list One strategy to find a non-trivial admissible heuristics is to relax the original problem Slide 32

33. 33

34. Heuristics for Forward Planning: ignore delete-list One strategy to find an admissible heuristics is to relax the original problem One way : remove all the effects that make a variable = F. Name of this heuristic derives from complete STRIPS representation Action effects are divided into those that add elements to the new state (add list) and those that remove elements (delete list) If we find the path from the initial state to the goal using this relaxed version of the actions: the length of the solution is an underestimate of the actual solution length Why? Action a effects (B=F, C=T) Slide 34

35. Heuristics for Forward Planning: ignore delete-list One strategy to find an admissible heuristics is to relax the original problem One way : remove all the effects that make a variable = F. If we find the path from the initial state to the goal using this relaxed version of the actions: the length of the solution is an underestimate of the actual solution length Why? In the original problem, one action (e.g. a above) might undo an already achieved goal (e.g. by a1 below) Action a effects (B=F, C=T) S0S0 A = T B = F C = F Goal A = T B = T C = T Slide 35

36. Heuristics for Forward Planning: ignore delete-list One strategy to find an admissible heuristics is to relax the original problem One way : remove all the effects that make a variable = F. If we find the path from the initial state to the goal using this relaxed version of the actions: the length of the solution is an underestimate of the actual solution length Why? In the original problem, one action (e.g. a above) might undo an already achieved goal (e.g. by a1 below). It would have to be achieved again Action a effects (B=F, C=T) A = T B = F C = F A = T B = T C = T a1 B = T a C = T B = F Slide 36 S0S0 Goal

37. Example for ignore-delete-list Let’s stay in the robot domain But say our robot has to bring coffee to Bob, Sue, and Steve: G = {bob_has_coffee, sue_has_coffee, steve_has_coffee} They all sit in different offices Original actions “pick-up coffee” achieves rhc = T “deliver coffee” achieves rhc = F “Ignore delete lists”  remove rhc = F from “deliver coffee” once you have coffee you keep it Problem gets easier: only need to pick up coffee once, navigate to the right locations, and deliver Slide 37

38. Can you think of another way of deriving a domain-independent admissible heuristic for planning? Let’s stay in the robot domain But say our robot has to bring coffee to Bob, Sue, and Steve: G = {bob_has_coffee, sue_has_coffee, steve_has_coffee} They all sit in different offices B. Count Number of unsatisfied sub-goals C. Plan by ignoring action preconditions A. Count Number of satisfied sub-goals D. None of the above Slide 38

39. Can you think of another way of deriving a domain- independent admissible heuristic for planning? Let’s stay in the robot domain But say our robot has to bring coffee to Bob, Sue, and Steve: G = {bob_has_coffee, sue_has_coffee, steve_has_coffee} They all sit in different offices BOTH in this domain, and for this goal, since one cannot achieve more than one subgoal with a single action In general, only C; B might be an overestimate of the actual cost when actions can achieve multiple subgoals B. Count Number of unsatisfied sub-goals C. Plan by ignoring action preconditions Slide 39

40. Example Let’s stay in the robot domain But say our robot has to bring coffee to Bob, Sue, and Steve: G = {bob_has_coffee, sue_has_coffee, steve_has_coffee} They all sit in different offices Admissible heuristic: ignore preconditions: Can simply apply “DeliverCoffee(person)” action for each person Slide 40

41. 41

42. Heuristics for Forward Planning: ignore delete-list But how do we compute the actual heuristics values for ignore delete-list? Slide 42

43. Slide 43

44. Heuristics for Forward Planning: ignore delete-list But how do we compute the actual heuristics values for ignore delete-list? To compute h(s i ), run forward planner with s i as start state Same goal as original problem Actions without “delete list” Often fast enough to be worthwhile Planning is PSPACE-hard (that’s really hard, includes NP-hard) Without delete lists: often very fast Slide 44

45. Example Planner FF or Fast Forward Jörg Hoffmann: Where 'Ignoring Delete Lists' Works: Local Search Topology in Planning Benchmarks. J. Artif. Intell. Res. (JAIR) 24: 685- 758 (2005)J. Artif. Intell. Res. (JAIR) 24 Winner of the 2001 AIPS Planning CompetitionAIPS Planning Competition Estimates the heuristics by solving the relaxed planning problem with a planning graph method (next class) Uses Best First search with this heuristic to find a solution Slide 45

46. Final Comment You should view Forward Planning as one of the basic planning techniques (we’ll see another one next week) By itself, it cannot go far, but it can work very well in combination with other techniques, for specific domains See, for instance, descriptions of competing planners in the presentation of results for the 2002 and 2008 planning competition (posted in the class schedule) Slide 46

47. Learning Goals for Planning so Far Included in midterm Represent a planning problem with the STRIPS representation Explain the STRIPS assumption Solve a planning problem by search (forward planning). Specify states, successor function, goal test and solution. Construct and justify a heuristic function for forward planning

48. STRIPS lends itself to solve planning problems either As pure search problems As CSP problems We will look at one technique for each approach Solving planning problems Slide 48

49. Course Overview Environment Problem Type Query Planning Deterministic Stochastic Constraint Satisfaction Search Arc Consistency Search Logics STRIPS Vars + Constraints Value Iteration Variable Elimination Belief Nets Decision Nets Markov Processes Static Sequential Representation Reasoning Technique Variable Elimination

50. Lecture Overview Recap of Lecture 11 Forward Planning Heuristics for Forward Planning Intro to Planning as CSP

51. Planning as a CSP We simply reformulate a STRIPS model as a set of variables and constraints Give it a try: please work in groups of two or three for a few minutes and try to define what would you chose as Variables Constraints Use the Rob Delivery World as a leading example Slide 51

52. What will be the CSP variables and constraints? Features change over time Might need more than one CSP variable per feature Initial state constraints Goal state constraints STRIPS example actions STRIPS representation of the action pick up coffee, PUC: preconditions Loc = cs and RHC = effects RHC = rhc STRIPS representation of the action deliver coffee, DelC: preconditions Loc = off and RHC = rhc effects RHC = and SWC = Have to capture these conditions as constraints Slide 52

53. Planning as a CSP: General Idea Action preconditions and effects are virtually constraints between the action, the states in which it can be applied the states that it can generate Thus, we can make both states and actions into the variables of our CSP formulations However, constraints based on action preconditions and effects relate to states at a given time t, the corresponding valid actions and the resulting states at t +1 need to have as many state and action variables as there are planning steps Slide 53

54. Planning as a CSP: General Idea Both features and actions are CSP variables Action preconditions and effects are constraints among the action, the states in which it can be applied the states that it can generate Slide 54

55. Planning as a CSP: General Idea These action constraints relate to states at a given time t, the corresponding valid actions and the resulting states at t +1 we need to have as many state and action variables as we have planning steps Slide 55

56. Planning as a CSP: Variables We need to ‘unroll the plan’ for a fixed number of steps: this is called the horizon k To do this with a horizon of k: construct a CSP variable for each STRIPS state variable at each time step from 0 to k construct a Boolean CSP variable for each STRIPS action at each time step from 0 to k - 1. Slide 56