1. Computer Science CPSC 322 Lecture 18 Heuristics for Forward Planning

2. Announcements Midterm will be 50’, close textbook, no calculator or other devices - Part short questions similar or even verbatim from the list posted in connect - Part more problem-solving style questions Office hours this week (red is new) – Also posted on Connect Tuesday 12:30 pm Kamyar Wednesday 11 am Mike (new room ICCS X141) Wednesday 4 pm Cristina Thursday 12 pm Samad Thursday 2pm Cristina Friday 9:30 am Lucas Material for midterm available in Connect - Short conceptual questions (already posted) - List of Learning Goals (already posted) - Sample problem-solving questions (will be posted by Monday) Topics Covered - from beginning to Forward Planning included (no heuristics for forward planning)

3. Recap of Lecture 17 Heuristics for forward planning Intro to planning as CSP Outline

4. Course Overview Environment Problem Type Query Planning Deterministic Stochastic Constraint Satisfaction Search Arc Consistency Search Logics STRIPS Vars + Constraints Value Iteration Variable Elimination Belief Nets Decision Nets Markov Processes Static Sequential Representation Reasoning Technique Variable Elimination We’ll focus on Planning

5. Goal Description of states of the world Description of available actions => when each action can be applied and what its effects are Planning: build a sequence of actions that, if executed, takes the agent from the current state to a state that achieves the goal Planning Problem

6. Standard Search vs. Specific R&R systems Constraint Satisfaction (Problems): State: assignments of values to a subset of the variables Successor function: assign values to a “free” variable Goal test: all variables assigned a value and all constraints satisfied? Solution: possible world that satisfies the constraints Heuristic function: none (all solutions at the same distance from start) Planning : State Successor function Goal test Solution Heuristic function Inference State Successor function Goal test Solution Heuristic function

7. “Open-up” the representation of states, goals and actions –States and goals as set of features –STRIPS representation for actions: two parts: 1. Preconditions: a set of assignments to features that must be satisfied in order for the action to be applicable in a state 2. Effects: a set of assignments to features that are caused by the action Key Idea of Planning

8. Delivery Robot Example: features RLoc - Rob's location Domain: {coffee shop, Sam's office, mail room, laboratory} short {cs, off, mr, lab} RHC – Rob has coffee Domain: {true, false}. Alternatively notation: rhc indicate that Rob has coffee, and by that Rob doesn’t have coffee SWC – Sam wants coffee {true, false} MW – Mail is waiting {true, false} RHM – Rob has mail {true, false} An example state is 64 possible states in total

9. Delivery Robot Example: Actions The robot’s actions are: Move - Rob's move action move clockwise (mc ), move anti-clockwise (mac ) PUC - Rob picks up coffee must be at the coffee shop and not have coffee DelC - Rob delivers coffee must be at the office, and must have coffee PUM - Rob picks up mail must be in the mail room, and mail must be waiting DelM - Rob delivers mail must be at the office and have mail Preconditions for action application

10. STRIPS actions: Example STRIPS representation of the action pick up coffee, PUC: preconditions Loc = cs and RHC = F effects RHC = T STRIPS representation of the action deliver coffee, Del : preconditions Loc = off and RHC = T effects RHC = F and SWC = F STRIPS representation of the action move clockwise from coffee shop, mc-cs preconditions: Loc = cs effects: Loc = off

11. STRIPS Actions (cont’) The STRIPS assumption: all features not explicitly changed by an action stay unchanged So if the feature V has value v i in state S i, after action a has been performed, what can we conclude about a and/or the state of the world S i-1 immediately preceding the execution of a? S i-1 V = v i SiSi a C. At least one of A and B A. V = v i was TRUE in S i-1 B.One of the effects of a is to set V = v i

12. STRIPS lends itself to solve planning problems either As pure search problems As CSP problems We will look at one technique for each approach Solving planning problems

13. Forward planning To find a plan, a solution : search in the state-space graph The states are the possible worlds full assignments of values to features The arcs from a state s represent all the actions that are possible in state s A plan is a path from the state representing the initial state to a state that satisfies the goal Which actions a are possible in a state s? B. Those where a’s preconditions are satisfied in s

14. Example for state space graph Goal: D (puc, mc, dc) a sequence of actions that gets us from the start to a goal Solution:

15. Standard Search vs. Specific R&R systems Constraint Satisfaction (Problems): State: assignments of values to a subset of the variables Successor function: assign values to a “free” variable Goal test: set of constraints Solution: possible world that satisfies the constraints Heuristic function: none (all solutions at the same distance from start) Planning : State: full assignment of values to features Successor function: states reachable by applying valid actions Goal test: partial assignment of values to features Solution: a sequence of actions Heuristic function Inference State Successor function Goal test Solution Heuristic function

16. Forward Planning Any of the search algorithms we have seen can be used in Forward Planning Problem? Complexity is defined by the branching factor, e.g.…

17. Forward Planning Any of the search algorithms we have seen can be used in Forward Planning Problem? Complexity is defined by the branching factor, e.g.… Number of applicable actions to a state Can be very large Solution?

18. Standard Search vs. Specific R&R systems Constraint Satisfaction (Problems): State: assignments of values to a subset of the variables Successor function: assign values to a “free” variable Goal test: set of constraints Solution: possible world that satisfies the constraints Heuristic function: none (all solutions at the same distance from start) Planning : State: full assignment of values to features Successor function: states reachable by applying valid actions Goal test: partial assignment of values to features Solution: a sequence of actions Heuristic function Inference State Successor function Goal test Solution Heuristic function

19. Recap of Lecture 17 Heuristics for forward planning Intro to planning as CSP Outline

20. Heuristics for Forward Planning Not in textbook, but you can see details in Russel&Norvig, 10.3.2 Heuristic function: estimate of the distance from a state to the goal In planning this distance is the………………. B. # of actions needed to get from s to the goal C. # of legal actions in s A. # of goal features not true in s

21. Heuristics for Forward Planning Not in textbook, but you can see details in Russel&Norvig, 10.3.2 Heuristic function: estimate of the distance from a state to the goal In planning this distance is the………………. Finding a good heuristics is what makes forward planning feasible in practice Factored representation of states and actions allows for definition of domain-independent heuristics We will look at one example of such domain-independent heuristic that has proven to be quite successful in practice B.# of actions needed to get from s to the goal

22. Heuristics for Forward Planning: We make two simplifications in the STRIPS representation All features are binary: T / F Goals and preconditions can only be assignments to T e.g. positive assertions Definition: a subgoal is the specific assignment for one of the features in the goal e.g., if the goal is then…. S1 A = T B = F C = F Goal A = T B = T C = T

23. Heuristics for Forward Planning: ignore delete-list One strategy to find an admissible heuristics is to relax the original problem One way : remove all the effects that make a variable = F. Name of this heuristic derives from complete STRIPS representation Action effects are divided into those that add elements to the new state (add list) and those that remove elements (delete list) If we find the path from the initial state to the goal using this relaxed version of the actions: the length of the solution is an underestimate of the actual solution length Why? Action a effects (B=F, C=T)

24. Heuristics for Forward Planning: ignore delete-list One strategy to find an admissible heuristics is to relax the original problem One way : remove all the effects that make a variable = F. If we find the path from the initial state to the goal using this relaxed version of the actions: the length of the solution is an underestimate of the actual solution length Why? In the original problem, one action (e.g. a above) might undo an already achieved goal (e.g. by a1 below) Action a effects (B=F, C=T) S1 A = T B = F C = F Goal A = T B = T C = T

25. Heuristics for Forward Planning: ignore delete-list One strategy to find an admissible heuristics is to relax the original problem One way : remove all the effects that make a variable = F. If we find the path from the initial state to the goal using this relaxed version of the actions: the length of the solution is an underestimate of the actual solution length Why? In the original problem, one action (e.g. a above) might undo an already achieved goal (e.g. by a1 below) Action a effects (B=F, C=T) S1 A = T B = F C = F Goal A = T B = T C = T a1 B = T a C = T B = F

26. Example for ignore-delete-list Let’s stay in the robot domain But say our robot has to bring coffee to Bob, Sue, and Steve: G = {bob_has_coffee, sue_has_coffee, steve_has_coffee} They all sit in different offices Original actions “pick-up coffee” achieves rhc = T “deliver coffee” achieves rhc = F “Ignore delete lists”  remove rhc = F from “deliver coffee” once you have coffee you keep it Problem gets easier: only need to pick up coffee once, navigate to the right locations, and deliver

27. Can you think of another way of deriving a domain-independent admissible heuristic for planning? Let’s stay in the robot domain But say our robot has to bring coffee to Bob, Sue, and Steve: G = {bob_has_coffee, sue_has_coffee, steve_has_coffee} They all sit in different offices B. Count Number of unsatisfied sub-goals C. Plan by ignoring action preconditions A. Count Number of satisfied sub-goals D. None of the above

28. Can you think of another way of deriving a domain- independent admissible heuristic for planning? Let’s stay in the robot domain But say our robot has to bring coffee to Bob, Sue, and Steve: G = {bob_has_coffee, sue_has_coffee, steve_has_coffee} They all sit in different offices BOTH (in this domain, and for this goal, since one cannot achieve more than one subgoal with a single action) In general, only C. B might be an overestimate of the actual cost when actions can achieve multiple subgoals B. Count Number of unsatisfied sub-goals C. Plan by ignoring action preconditions

29. Example Let’s stay in the robot domain But say our robot has to bring coffee to Bob, Sue, and Steve: G = {bob_has_coffee, sue_has_coffee, steve_has_coffee} They all sit in different offices Admissible heuristic: ignore preconditions: Can simply apply “DeliverCoffee(person)” action for each person Is ignore-preconditions better or worse than ignore- delete-list?

30. Ignore-delete-list provides a tighter estimate of the actual solution length (number of actions needed to achieve the goal), because it is more constrained Keep some of the action preconditions as opposed to removing them all

31. Heuristics for Forward Planning: ignore delete-list But how do we compute the actual heuristics values for ignore delete-list?

32. Heuristics for Forward Planning: ignore delete-list But how do we compute the actual heuristics values for ignore delete-list? To compute h(s i ), run forward planner with s i as start state Same goal as original problem Actions without “delete list” Often fast enough to be worthwhile Planning is PSPACE-hard (that’s really hard, includes NP-hard) Without delete lists: often very fast

33. Example Planner FF or Fast Forward Jörg Hoffmann: Where 'Ignoring Delete Lists' Works: Local Search Topology in Planning Benchmarks. J. Artif. Intell. Res. (JAIR) 24: 685- 758 (2005)J. Artif. Intell. Res. (JAIR) 24 Winner of the 2001 AIPS Planning CompetitionAIPS Planning Competition Estimates the heuristics by solving the relaxed planning problem with a planning graph method (next class) Uses Best First search with this heuristic to find a solution When it hits a plateau or local minimum Uses IDS to get away (i..e, to reach a state that is better)

34. Final Comment You should view Forward Planning as one of the basic planning techniques (we’ll see another one next week) By itself, it cannot go far, but it can work very well in combination with other techniques, for specific domains See, for instance, descriptions of competing planners in the presentation of results for the 2002 and 2008 planning competition (posted in the class schedule)

35. Learning Goals Included in midterm Represent a planning problem with the STRIPS representation Explain the STRIPS assumption Solve a planning problem by search (forward planning). Specify states, successor function, goal test and solution. Not Included in midterm Construct and justify a heuristic function for forward planning