1. 1 Approaches to Agent Control zAgent senses world, acts to affect world zReactive Control ySet of situation-action rules yE.g. x1) if dog-is-behind-me then run-forward x2)if food-is-near then eat zPlanning yReason about effect of combinations of actions y“Planning ahead” yAvoiding “painting oneself into a corner”

2. 2 Different Planning Approaches zGenerative Planning yReason from first principles (knowledge of actions) to generate plan yRequires formal model of actions zCase-Based Planning yRetrieve old plan which worked for similar problem yRevise retrieved plan for this problem zSee also yPolicy Iteration / Markov-Decision Processes yReinforcement Learning

3. 3 Generative Planning zInput yDescription of initial state of world (in some KR) yDescription of goal (in some KR) yDescription of available actions (in some KR) zOutput ySequence of actions

4. 4 Planning as the Arena of AI zRepresentation yLogic ySearch yNLP zSolving yLogic ySearch yLearning

5. 5 Applications zDeep Space One zMarsokhod (Mars Rover) zGround control (Atlanta & Sydney) zMetacrawler zRobot control zRodney and the Internet Softbots

6. 6 Input Representation zDescription of initial state of world ySet of propositions:  ((block a) (block b) (block c) (on-table a) (on-table b) (clear a) (clear b) (clear c) (arm-empty)) zDescription of goal (i.e. set of desired worlds) yLogical conjunction yAny world that satisfies the conjunction is a goal  (and (on a b) (on b c))) zDescription of available actions

7. 7 Representing Actions STRIPS UWL ADL SADL TractableExpressive Situation Calculus

8. 8 How Represent Actions? zSimplifying assumptions yAtomic time yAgent is omniscient (no sensing necessary). yAgent is sole cause of change yActions have deterministic effects zSTRIPS representation yWorld = set of true propositions yActions: xPrecondition: (conjunction of literals) xEffects (conjunction of literals) a a a north11 north12 W0W0 W2W2 W1W1

9. 9 STRIPS Actions zAction =function from world-state  world-state zPrecondition says where function defined zEffects say how to change set of propositions a a north11 W0W0 W1W1 precond: (and (agent-at 1 1) (agent-facing north)) effect: (and (agent-at 1 2) (not (agent-at 1 1))) Note: strips doesn’t allow derived effects; you must be complete!

10. 10 Action Schemata (:operator pick-up :parameters ((block ?ob1)) :precondition (and (clear ?ob1) (on-table ?ob1) (arm-empty)) :effect (and (not (clear ?ob1)) (not (on-table ?ob1)) (not (arm-empty)) (holding ?ob1))) zInstead of defining: pickup-A and pickup-B and … zDefine a schema: Note: strips doesn’t allow derived effects; you must be complete! }

11. 11 Planning as Search zNodes zNext state generator zInitial State zGoal checker

12. 12 Planning as Search zNodes zNext state generator zInitial State zGoal Checker World states Instantiate all actions The state satisfying the complete description of the initial conds Any state satisfying the goal propositions

13. 13 “Causal Link” Planning zNodes zNext state generator zInitial State zGoal Checker Partially specified plans Adding + deleting actions or constraints (e.g. <) to plan The empty plan (Actually two dummy actions…) Does plan achieve the goal (in simulation)? Need efficient way to evaluate quality (percentage of preconditions satisfied) of partial plan … Hence causal link datastructures

14. 14 Forward-Chaining World-Space Search A C B C B A Initial State Goal State

15. 15 Backward-Chaining Search Thru Space of Partial World-States D C B A E D C B A E D C B A E * * * zProblem: Many possible goal states are equally acceptable. zFrom which one does one search? A C B Initial State is completely defined D E

16. 16 Plan-Space Search pick-from-table(C) pick-from-table(B) pick-from-table(C) put-on(C,B) zHow represent plans? zHow test if plan is a solution?

17. 17 Graphplan zPhase 1 - Graph Expansion y“Grow” potential facts in the world yNecessary (insufficient) conditions for plan existence zPhase 2 - Solution Extraction ySearch for solution sequence xNon-exclusive xStems from initial state

18. 18 Constructing the planning graph… zInitial proposition layer yJust the initial conditions zAction layer i yIf all of an action’s preconds are in i-1 yThen add action to layer I zProposition layer i+1 yFor each action at layer i yAdd all its effects at layer i+1

19. 19 Mutual Exclusion zActions A,B exclusive (at a level) if yA deletes B’s precond, or yB deletes A’s precond, or yA & B have inconsistent preconds zPropositions P,Q inconsistent (at a level) if yall ways to achieve P exclude all ways to achieve Q

20. 20 Graphplan zCreate level 0 in planning graph zLoop yIf goal  contents of highest level (nonmutex) yThen search graph for solution xIf find a solution then return and terminate yElse Extend graph one more level A kind of double search: forward direction checks necessary (but insufficient) conditions for a solution,... Backward search verifies...

21. 21 Dinner Date Initial Conditions: (:and (cleanHands) (quiet)) Goal: (:and (noGarbage) (dinner) (present)) Actions: (:operator carry :precondition :effect (:and (noGarbage) (:not (cleanHands))) (:operator dolly :precondition :effect (:and (noGarbage) (:not (quiet))) (:operator cook :precondition (cleanHands) :effect (dinner)) (:operator wrap :precondition (quiet) :effect (present))

22. 22 Planning Graph noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 0 Prop 1 Action 2 Prop 3 Action 4 Prop

23. 23 Are there any exclusions? noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 0 Prop 1 Action 2 Prop 3 Action 4 Prop

24. 24 Do we have a solution? noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 0 Prop 1 Action 2 Prop 3 Action 4 Prop

25. 25 Extend the Planning Graph noGarb cleanH quiet dinner present carry dolly cook wrap carry dolly cook wrap cleanH quiet noGarb cleanH quiet dinner present 0 Prop 1 Action 2 Prop 3 Action 4 Prop

26. 26 One (of 4) possibilities noGarb cleanH quiet dinner present carry dolly cook wrap carry dolly cook wrap cleanH quiet noGarb cleanH quiet dinner present 0 Prop 1 Action 2 Prop 3 Action 4 Prop