1. Planning CSE 473

2. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning Supervised Learning LogicProbability

3. © Daniel S. Weld 3 Ways to make “plans” Generative Planning Reason from first principles (knowledge of actions) Requires formal model of actions Case-Based Planning Retrieve old plan which worked on similar problem Revise retrieved plan for this problem Reinforcement Learning Act ”randomly” - noticing effects Learn reward, action models, policy

4. © Daniel S. Weld 4 Generative Planning Input Description of (initial state of) world (in some KR) Description of goal (in some KR) Description of available actions (in some KR) Output Controller E.g. Sequence of actions E.g. Plan with loops and conditionals E.g. Policy = f: states -> actions

5. © Daniel S. Weld 5 Input Representation Description of initial state of world E.g., Set of propositions: ((block a) (block b) (block c) (on-table a) (on-table b) (clear a) (clear b) (clear c) (arm-empty)) Description of goal: i.e. set of worlds or ?? E.g., Logical conjunction Any world satisfying conjunction is a goal (and (on a b) (on b c))) Description of available actions

6. © Daniel S. Weld 6 Simplifying Assumptions Environment Percepts Actions What action next? Static vs. Dynamic Fully Observable vs. Partially Observable Deterministic vs. Stochastic Instantaneous vs. Durative Full vs. Partial satisfaction Perfect vs. Noisy

7. © Daniel S. Weld 7 Classical Planning Environment Static Fully Observable Deterministic Instantaneous Full Perfect I = initial state G = goal state OiOi (prec)(effects) [ I ] OiOi OjOj OkOk OmOm [ G ]

8. © Daniel S. Weld 8 StaticDeterministicObservable Instantaneous Propositional “Classical Planning” Dynamic Replanning/ Situated Plans Durative Temporal Reasoning Continuous Numeric Constraint reasoning (LP/ILP) Stochastic Contingent/Conformant Plans, Interleaved execution MDP Policies POMDP Policies Partially Observable Contingent/Conformant Plans, Interleaved execution Semi-MDP Policies

9. © Daniel S. Weld 9 Today’s Hot Research Areas Durative Actions Simultaneous actions, events, deadline goals Planning Under Uncertainty Modeling sensors; searching belief states [ I ] OiOi OjOj OkOk ? ObOb OaOa OcOc

10. © Daniel S. Weld 10 Representing Actions Situation Calculus STRIPS PDDL UWL Dynamic Bayesian Networks

11. © Daniel S. Weld 11 How Represent Actions? Simplifying assumptions Atomic time Agent is omniscient (no sensing necessary). Agent is sole cause of change Actions have deterministic effects STRIPS representation World = set of true propositions Actions: Precondition: (conjunction of literals) Effects (conjunction of literals) north11north12 W0W0 W2W2 W1W1

12. © Daniel S. Weld 12 STRIPS Actions Action = function: worldState  worldState Precondition says where function defined Effects say how to change set of propositions a a north11 W0W0 W1W1 precond: (and (agent-at 1 1) (agent-facing north)) effect: (and (agent-at 1 2) (not (agent-at 1 1))) Note: strips doesn’t allow derived effects; you must be complete!

13. © Daniel S. Weld 13 Action Schemata (:operator pick-up :parameters ((block ?ob1)) :precondition (and (clear ?ob1) (on-table ?ob1) (arm-empty)) :effect (and (not (clear ?ob1)) (not (on-table ?ob1)) (not (arm-empty)) (holding ?ob1))) Instead of defining: pickup-A and pickup-B and … Define a schema: Note: strips doesn’t allow derived effects; you must be complete! }

14. © Daniel S. Weld 14 Immediate Outline Constraint satisfaction The planning problem Searching world states Regression Heuristics Graphplan SATplan Reachability analysis & heuristics Planning under uncertainty

15. © Daniel S. Weld 15 Planning as Search Nodes Arcs Initial State Goal State World states Actions The state satisfying the complete description of the initial conds Any state satisfying the goal propositions

16. © Daniel S. Weld 16 Forward-Chaining World-Space Search A C B C B A Initial State Goal State

17. © Daniel S. Weld 17 Backward-Chaining Search Thru Space of Partial World-States D C B A E D C B A E D C B A E * * * Problem: Many possible goal states are equally acceptable. From which one does one search? A C B Initial State is completely defined D E