1. CLASSICAL PLANNING

2. Outline  The challenges in planning with standard search algorithm  Representing Plans – the PDDL language  Planning as state - space search  Planning Graphs and the GRAPHPLAN Algorithm  Other Approaches

3. So Far...  AI is defined as the study of rational ( reasonable, logical ) action  Means that planning - devising a plan of action to achieve one ' s goals - is a critical part of AI.  Two examples of planning agents so far :  the search - based problem - solving agent of Chapter 3  logical agent of Chapter 7  Now, we introduce a representation for planning problems that scales up to problems that could not be handled by those earlier approaches.

4. Challenges in Planning with Standard Search Method  The problem - solving agent of Chapter 3 can find sequences of actions that result in a goal state.  But it deals with atomic representations of states  and thus needs good domain - specific heuristics to perform well.

5. Challenges in Planning with Standard Search Method ( A * algorithm etc )  The hybrid propositional logical agent of Chapter 7 can find plans  it uses domain - independent heuristics based on the logical structure of the problem.  But it relies on ground ( variable - free ) propositional inference  which means that it may be swamped when there are many actions and states.  Ex : In wumpus world, the simple action of moving a step forward had to be repeated for all four agent orientations, T time steps, and n 2 (=16) current locations.

6. Challenges in Planning – too many choices !

7. Planning problem  Classical planning environment : fully observable, deterministic, finite, static and discrete.  Find a sequence of actions that achieves a given goal when executed from a given initial world state. That is, given  a set of action descriptions ( defining the possible primitive actions by the agent ),  an initial state description, and  a goal state description or predicate,  Compute a plan, which is  a sequence of action instances, such that executing them in the initial state will change the world to a state satisfying the goal - state description.  Goals are usually specified as a conjunction of subgoals to be achieved

8. Planning vs. problem solving Planning and problem solving methods can often solve the same sorts of problems Planning is more powerful because of the representations and methods used States, goals, and actions are decomposed into sets of sentences ( usually in first - order logic ) Subgoals can be planned independently, reducing the complexity of the planning problem

9. Goal of Planning  Choose actions to achieve a certain goal  But isn ’ t it exactly the same goal as for problem solving ?  Some difficulties with problem solving :  The successor function is a black box : it must be “ applied ” to a state to know which actions are possible in that state and what are the effects of each one

10. Have ( AI _ TEXTBOOK ) Suppose that the goal is HAVE ( AI _ TEXTBOOK ). – From some initial state where HAVE ( AI _ TEXTBOOK ) is not satisfied, the successor function must be repeatedly applied to eventually generate a state where HAVE ( AI _ TEXTBOOK ) is satisfied. – An explicit representation of the possible actions and their effects would help the problem solver select the relevant actions – Otherwise, in the real world an agent would be overwhelmed by irrelevant actions

11. Planning vs Problem Solving  Another difficulty with problem solving :  The goal test is another black - box function, states are domain - specific data structures, and heuristics must be supplied for each new problem  Suppose that the goal is HAVE ( AI _ TEXTBOOK ) ∧ HAVE ( AI _ SLIDES )  Without an explicit representation of the goal, the problem solver cannot know that a state where HAVE ( AI _ TEXTBOOK ) is already achieved is more promising than a state where neither HAVE ( AI _ TEXTBOOK ) nor HAVE ( AI _ SLIDES ) is achieved

12. Planning vs Problem Solving  A third difficulty with problem solving :  The goal may consist of several nearly independent subgoals, but there is no way for the problem solver to know it  HAVE ( AI _ TEXTBOOK ) and HAVE ( AI _ SLIDES ) may be achieved by two nearly independent sequences of actions

13. Representations in Planning  Planning opens up the black - boxes by using logic to represent :  Actions  States  Goals Problem solvingLogic representation Planning

14. Challenges in Planning with Standard Search Method ( A * algorithm etc ) - Summarized  Why real - world planning problem with state - space search is hard :  Too many irrelevant actions – goal oriented search would help  Difficult to find a good heuristic function  So, introduce a representation for planning problems that scales up to problems that could not be handled by those earlier approaches.  So the language for planning should make it easy for algorithms to exploit the logical structure of the problem

15. Classical Planning  What is planning :  The task of coming up with a sequence of actions to achieve a given goal is called planning  Classical planning : When the environment is  Fully observable  Deterministic  Static  Discrete  Single agent

16. Planning Languages  Languages must represent..  States  Goals  Actions  Languages must be  Expressive for ease of representation and for describing wide variety of problems  Flexible for manipulation by algorithms Ex : We will talk about Planning Domain Definition Language ( PDDL )

17. Languages for Planning Problems  STRIPS  Stanford Research Institute Problem Solver  Historically important  ADL  Action Description Languages  Relaxed some of the restrictions that made STRIPS inadequate for real - world problems  PDDL  Planning Domain Definition Language  Revised & enhanced for the needs of the International Planning Competition  Currently version 3.1 version 3.1  Includes STRIPS and ADL 17

18. 18 Planning Languages  PDDL or STRIPS is simplest  Important limit : function - free literals Allows for propositional representation Function symbols lead to infinitely many states and actions But poor expressivity  Extension : Action Description language ( ADL )  Allows negative literals  Allows quantified variables, conjunctions, disjunctions in goals  Open World assumption Action ( Fly ( p : Plane, from : Airport, to : Airport ), PRECOND : At ( p, from ) ∧ ( from ≠ to ) EFFECT : ¬At ( p, from ) ∧ At ( p, to ))

19. Planning Domain Definition Language  Components :  Objects : things we are interested in  Predicates : properties of objects, true or false  Initial state : state of the world we start in  Goal specification : state we want to end up in  Actions : ways we can change state

20. Classical Planning  In response to this, planning researchers have settled on a factored representation  One in which a state of the world is represented by a collection of variables.  A language called PDDL, the Planning Domain Definition Language is used  that allows to express all 4 Tn 2 actions with one action schema.

21. Classical Planning  A simple version of PDDL will be followed  Show how PDDL describes the four things we need to define a search problem :  the initial state,  The actions that are available in a state,  the result of applying an action, and  the goal test.

22. PDDL Language – Representing state  State : represented by a conjunction of ground ( positive literals ) atoms without function symbols  Ex : Definition (state): Each state is represented as a conjunction of fluents that are ground, functionless atoms. Ex1: the state of a hapless agent Poor ^ Unknown Ex2: A state in a package delivery problem might be At(Truck1, Manipal) ^ At(Truck2, Mangalore).

23. Fluent - Background  Fluent – means flowing  Refers to an aspect of the world that changes  Refers to a state variable

24. PDDL Language – Representing state  Assumptions :  Database semantics :  the closed - world assumption means that any fluents that are not mentioned are false  unique names assumption :  means that Truck 1 and Truck 2 are distinct. Ex : The following fluents are not allowed in a state :  At ( x, y ) ( because it is non - ground )   Poor ( because it is a negation ),  At ( Father ( Fred ), Sydney ) ( because it uses a function symbol )

25. Fluent - Background  Assumptions :  The representation of states is carefully designed  so that a state can be treated either as a conjunction of fluents which can be manipulated by logical inference,  or as a set of fluents which can be manipulated with set operations.  Note : The set semantics is easier to deal with.

26. PDDL Language – Representing Actions  Actions  are described by a set of action schemas  that implicitly define the ACTIONS ( s ) and RESULT ( s, a ) functions needed to do a problem - solving search.  Any system for action description ( Chapter 7)  needs to solve the frame problem -  “ Consider what changes and what stays the same as the result of the action ”

27. So …  resolution theorem proving only finds a proof ( plan ), not necessarily a good plan  Need to address frame problem  So, the language is restricted and a special - purpose algorithm ( a planner ) rather than general theorem prover is used

28. PDDL Language – Representing Actions  Classical planning concentrates on problems where most actions leave most things unchanged.  Think of a world consisting of a bunch of objects on a flat surface. The action of nudging an object causes that object to change its location by a vector delta A concise description of the action should mention only delta it shouldn ' t have to mention all the objects that stay in place. PDDL does that by specifying the result of an action in terms of what changes ; everything that stays the same is left unmentioned.

29. PDDL Language – Representing Actions  A set of ground ( variable - free ) actions can be represented by a single action schema.  The schema is a lifted representation -  it lifts the level of reasoning from propositional logic  to a restricted subset of first - order logic.  Ex : An action schema for flying a plane from one location to another

30. Representing Actions  The action schema consists of the action name, a list of all the variables used in the schema a precondition and an effect

31. Representing Actions  Actions are described in terms of preconditions and effects. Preconditions are predicates that must be true before the action can be applied. Precondition ( conj. of function - free literals – positive or negative ) Effects are predicates that are made true ( or false ) after the action has executed. Effect ( conj of function - free literals and P is True and not P is false - – positive or negative )  Sets of similar actions can be expressed as a schema

32. Conversion of Action schema  How the action schema converts into logical sentences ?  think of the variables as being universally quantified.  choose values to instantiate the variables.  Ex : Consider one ground action that results from substituting values for all the variables :  Action schema A ground action Action ( Fly ( P 1, SFO, JFK ), PRECOND : At ( P 1, SFO ) ^ Airplane ( P 1) ^ Airport ( SFO ) ^ Airport ( JFK ) EFFECT :  At ( P 1, SFO ) 1 ^ At ( P 1, JFK ))

33. Precondition vs. effect  The precondition defines the states in which the action can be executed  and the effect defines the result of executing the action.  An action a can be executed in state s if s entails the precondition of a.  Entailment can also be expressed with the set semantics : s ╞ a iff every positive literal in a is in s and every negated literal in a is not.

34. Representing Actions  In formal notation  ( a € ACTIONS ( s ))  s ╞ PRECOND ( a ),  where any variables in a are universally quantified.  Ex :   p, from, to ( Fly ( p, from, to ) € ACTIONS ( s ))  s ╞ ( At ( p, from ) ^ Airplane ( p ) ^ Airport ( from ) ^ Airport ( to )) Action schema

35. Representing Actions  We say that action a is applicable in state s if the preconditions are satisfied by s.  When an action schema a contains variables, it may have multiple applicable instantiations.  Ex : with the initial state defined for FLY, the Fly action can be instantiated as  Fly ( P 1, SFO, JFK ) or as  Fly ( P 2, JFK, SFO ), both of which are applicable in the initial state.  If an action a has v variables, then, in a domain with k unique names of objects, it takes O ( v k ) time in the worst case to find the applicable ground actions.

36. 36 Language semantics – Formal  How do actions affect states ?  An action is applicable in any state that satisfies the precondition.  For FLY action schema, applicability involves a substitution θ for the variables in the PRECOND. At ( P 1, JFK ) ∧ At ( P 2, SFO ) ∧ Plane ( P 1) ∧ Plane ( P 2) ∧ Airport ( JFK ) ∧ Airport ( SFO ) Satisfies : At ( p, from ) ∧ Plane ( p ) ∧ Airport ( from ) ∧ Airport ( to ) With θ ={ p / P 1, from / JFK, to / SFO } Thus the action is applicable. Action schema

37. Language semantics  The result of executing action a in state s is the state s ’  s ’ is same as s except Any positive literal P in the effect of a is added to s ’ Any negative literal ¬P is removed from s ’ EFFECT : ¬AT ( p, from ) ∧ At ( p, to ): At ( P 1, SFO ) ∧ At ( P 2, SFO ) ∧ Plane ( P 1) ∧ Plane ( P 2) ∧ Airport ( JFK ) ∧ Airport ( SFO )  PDDL assumption : ( avoids representational frame problem ) every literal NOT in the effect remains unchanged Action schema

38. Applying an Action  Find a substitution list  for the variables using the current state description  Apply the substitution to the propositions in the effect list  Add the result to the current state description to generate the new state  Example :  Current state : At ( P 1, JFK )  At ( P 2, SFO )  Plane ( P 1)  Plane ( P 2)  Airport ( JFK )  Airport ( SFO )  It satisfies the precondition with  ={ p / P 1, from / JFK, to / SFO )  Thus the action Fly ( P 1, JFK, SFO ) is applicable  The new current state is : At ( P 1, SFO )  At ( P 2, SFO )  Plane ( P 1)  Plane ( P 2)  Airport ( JFK )  Airport ( SFO )

39. Effects  May split Add - list and delete - list in Effect  RESULT ( s, a ) = ( s - DEL ( a )) U ADD ( a )  Ex : with the action Fly ( P 1, SFO, JFK ), we would remove At ( P 1, SFO ) and add At ( P 1, JFK ).  It is a requirement of action schemas that any variable in the effect must also appear in the precondition.  That way, when the precondition is matched against the state s, all the variables will be bound, and RESULT ( s, a ) will therefore have only ground atoms.  In other words, ground states are closed under the RESULT operation.

40. Representing Actions  In PDDL the times and states are implicit in the action schemas :  the precondition always refers to time t and  the effect to time t + 1

41. A specific planning problem  Clearly a planning problem can be seen as a search problem  Initial state – a conjunction of ground atoms Goal state – a conjunction of literals ( positive + negative ) that may contain ( existentially quantified ) variables  A set of action schemas

42. A specific planning problem  domain : A set of action schemas serves as a definition of a planning domain.  Problem : A specific problem within the domain is defined with the addition of an initial state and a goal.

43. A specific planning problem  The initial state is a conjunction of ground atoms.  With all states, the closed - world assumption is used, which means that any atoms that are not mentioned are false.  The goal is like a precondition : a conjunction of literals ( positive or negative ) that may contain variables  Ex : At ( p, SFO ) ^ Plane ( p ).  Any variables are treated as existentially quantified, so the goal is to have any plane at SFO.  The problem is solved when we can find a sequence of actions that end in a state s that entails the goal

44. Goal Representation  Goals are usually specified as a conjunction of subgoals to be achieved  A proposition satisfies a goal if it contains all the atoms of the goal and possibly others..  Ex1: Rich  Famous  Miserable satisfies the goal Rich  Famous  Ex2: Plane(Plane1) ^ At(Plane1, SFO) entails the goal At(p, SFO ) ^ Plane(p)

45. Example – the blocks - world

46. What is planning ? - Revise  Planning is an AI approach to control  It is deliberation about actions  Key ideas  We have a model of the world  Model describes states and actions  Give the planner a goal and it outputs a plan  Aim for domain independence  Planning is search

47. Classical planning restrictions 1. S is finite 2. Environment is fully observable 3. Environment is deterministic 4. Environment is static ( no external events ) 5. S has a factored representation 6. Goals are restricted to reachability 7. Plans are ordered sequences of actions 8. Actions have no duration 9. Planning is done offline

48. Major approaches  Situation calculus  State space planning  Partial order planning  Planning graphs  Planning with Propositional Logic  Hierarchical decomposition ( HTN planning )  Reactive planning  Note : Planning is an area in which AI is changing quickly

49. Example : Blocks World One of the most famous planning domains is known as the blocks world. This domain consists of a set of cube-shaped blocks sitting on a table. The blocks can be stacked, but only one block can fit directly on top of another. A robot arm can pick up a block and move it to another position, either on the table or on top of another block. The arm can pick up only one block at a lime, so it cannot pick up a block that has another one on it. The goal will always be to build one or more stacks of blocks, specified in terms of what blocks are on top of what other blocks. For example, a goal might be to get block A on B and block B on C

50. 50 Blocks world  Some domain constraints :  Only one block can be on another block  Any number of blocks can be on the table  The hand can only hold one block A C B TABLE

51. 51 Blocks world  Typical representation :  On(b, x) – to indicate that block b is on x, where x is either another block or the table.  Move(b, x, y) - The action for moving block b from the top of x to the top of y  Preconditions on moving b – is that no other block be on it.  In first-order logic, we write  x  On(x,b) or   x On(x,b)  Basic PDDL does not allow quantifiers  so instead we introduce a predicate Clear(x) that is true when nothing is on x.

52. 52 Blocks world - Move schema  The action Move moves a block b from x to y if both b and y are clear.  After the move is made, b is still clear but y is not.  Move schema

53. 53 Blocks world - Move schema  What happens if x or y is the table ?  When x = Table  this action has the effect Clear ( Table ), but the table should not become clear ;  when y = Table  it has the precondition Clear ( Table ), but the table does not have to be clear for us to move a block onto it.  So, the action schema does not maintain Clear properly when x or y is the table.

54. Solution  1. Introduce another action to move a block b from x to the table :

55. Solution  2. Redefine interpretation of Clear ( x ) - " there is a clear space on x to hold a block."  Under this interpretation, Clear ( Table ) will always be true.

56. Solution  Use MoveToTable ( b, x ) instead of Move ( b, x, Table )  Introduce the predicate Block and add Block ( b ) /\ Block ( y ) to the precondition of Move

57. Example 1: Blocks World

58. Example 1: Blocks World -1

59. Example 1: Blocks World 2

60. Exercise  Write action for the following version of blocks world problem  solution

61. Exercise  Write action for the following version of blocks world problem  solution  MoveToTable ( C, A )  Move ( B, Table, C )  Move ( A, Table, B )

62. Example 2: Air Cargo Air cargo transport problem loading and unloading cargo and flying it from place to place. The problem can be defined with three actions: Load, Unload, and Fly. The actions affect two predicates: In(c,p) means that cargo c is inside plane p At(x, a) means that object x (either plane or cargo) is at airport a.

63. Example 2: Air Cargo How to specify At predicate? When a plane flies from one airport to another, all the cargo inside the plane goes with it. In first-order logic it would be easy to quantify over all objects that are inside the plane. But basic PDDL does not have a universal quantifier, so we need a different solution. So redefine interpretation for At We say that a piece of cargo ceases to be At when it is In a plane; the cargo only becomes At the new airport when it is unloaded. So At really means "available for use at a given location."

64. Example 2: Air Cargo

65. Solution For the above initial condition and goal expressed in PDDL, write an action for moving cargo C1 to JFK and to move C2 to SFO Solution: Load(C1, P1, SFO), Fly (P1, SFO, JFK, Unload (C1, P1, JFK) Load(C2, P2, JFK), Fly(P2, JFK, SFO), Unload(C2, P2, SFO)

66. Example 3: Spare Tire Problem

67. Planning through Search  Search the space of states connected by actions  Each action takes a single timestep  Use familiar algorithms  BFS  DFS  A *  …

68. Forward Search  Forward ( progression ) state - space search, starting in the initial state and using the problem ’ s actions to search forward for the goal state.

69. Backward Search  Backward ( regression ) state - space search : search starting at the goal state ( s ) and using the inverse of the actions to search backward for the initial state.

70. Relevant Actions  An action is relevant  In Progression planning, when its preconditions match a subset of the current state  In Regression planning, when its effects match a subset of the current goal state

71. Planning with state - space search  Search the space of states  Progression planners  forward state - space search  Consider the effect of all possible actions in a given state  Regression planners  backward state - space search  To achieve a goal, what must have been true in the previous state.

72. Complexity of classical planning  PlanSAT : Is there a plan for a planning problem  Bounded PlanSAT : is there a plan of length <= k ? Both are decidable if the planning language is function - free – finitely many states  Both in the class of PSPACE – more difficult than NP !  But for many domains :  Bounded PlanSAT : NP - complete  PlanSAT : P – thus finding A ( possibly suboptimal ) plan may be not so hard !