1. Introduction to Planning Han Yu University of Central Florida

2. Outline What is planning? Formal definitions of planning problems Basic planning algorithms Recent development

3. What is Planning? Planning is a search problem that requires to find an efficient sequence of actions that transform a system from a given starting state to the goal state

4. What’s Given? Initial state of the problem Goal state of the problem A finite set of actions: pre-conditions: a finite set of conditions for the action to be performed post-conditions: a finite set of conditions that will be changed after the action is performed cost

5. What’s Output? A sequence of actions that meet the following criteria every action matches the current system state can transform system from initial state to goal state the total cost of the actions is below a specified value

6. Planning as a Real-World Problem Planning problem has a wide range of applications in the real world planning in daily life game world workflow management

7. Planning a Trip Begin PreparationAirline Reservation Hotel ReservationRental Car Reservation Confirm ReservationsEnd

8. Towers of Hanoi

9. Problem Analysis Optimal solution contains 2 n -1 actions, where n is the number of disks Up to 6 possible actions in each system state The upper bound on search space to find an optimal solution will be 6 (2 n -1)

10. Sliding-Tile Puzzle

11. Planning in Workflow Management Start Assembly Examine Order Gather Components Install Motherboard Test Install Internal Disk Install Network Card Assembly Box and Motherboard Plug in CD Insert Modem Plug in Battery Install Video Card End Assembly

12. Partial-Order Plan versus Total-Order Plan Partial-order plan consists partially ordered set of actions sequence constraints exist on these actions plan generation algorithm can be applied to transform partial-order plan to total-order plan Total-order plan consists totally ordered set of actions

13. Partial-Order Plan Start Get brush Get ladder Paint ceilingFinish

14. Total-Order Plan StartGet brushGet ladderPaint ceilingFinish StartGet brushGet ladderPaint ceilingFinish

15. Features of Planning Problems Large search space Action is associated with system states Restrictions on the action sequence Valid solution may not exist Optimization requirement

16. STRIPS Planning System A tuple T = (P, O, I, G), where P is a finite set of ground literals, the conditions O is a finite set of operators I is the initial state, a subset of P G is the goal state, a subset of P

17. STRIPS Operators Each operator O has the following attributes PC, a set of ground literals, defines the precondition of the operator D, a set of ground literals, defines the conditions that will be removed after the operation is executed A, a set of ground literals, defines the conditions that will be added after the operation is executed C, the cost of the operation

18. A Simple Example - Blocks World BA C C B A Initial State Goal State On(C, A) Clear(Fl) On(A, Fl) Clear(B) On(B, Fl) Clear(C) On(A, B) On(B, C) On(C, Fl) Clear(A) Clear(Fl)

19. Graphical Representation of Initial State Start T Clear(Fl)On(A, Fl)On(B, Fl)On(C, A)Clear(B)Clear(C)

20. Graphical Representation of Goal State Nil finish On(A, B)On(B, C)On(C, Fl)Clear(A)Clear(Fl)

21. Block World - Operator Move(x, y, z) Move block x that is above y to above z PC: On(x,y), Clear(x), Clear(z) D: Clear(z), On(x, y) A: On(x,z), Clear(y), Clear(Fl)

22. Graphical Representation of Operator A Clear(y)On(x, z) Move(x, y, z) On(x, y)Clear(x)Clear(z) Operator PC Clear(Fl)

23. Forward Chaining Search from the initial state Expand the search tree by finding the set of all applicable operators from the current state applicable means the precondition of the operator is a subset of current state Update current state For every state that is reached, record the shortest path (or path with lowest cost) from the initial state to this state If the goal state is reached, stop the algorithm

24. Forward Chaining On(C, A) Clear(Fl) On(A, Fl) Clear(B) On(B, Fl) Clear(C) On(C, Fl) Clear(Fl) On(A, Fl) Clear(B) On(B, Fl) Clear(C) Clear(A) Move(C, A, Fl) Move(B, Fl, C) On(B, C) Clear(Fl) On(C, A) Clear(B) On(A, Fl)

25. Backward Chaining Search backward from the goal state Expand the search tree by finding the set of all applicable operators that can reach the current state applicable means set A of the operator is a subset of current state Update the state If the initial state is reached, stop the algorithm The solution is a partial-ordered plan Constraints in action ordering may be violated

26. Backward Chaining On(A, B) On(B, C) On(C, Fl) Clear(A) Clear(Fl) Move(A, Fl, B) On(B, C) On(C, Fl) Clear(A) Clear(Fl) On(A, Fl) Clear(B)

27. Recent Developments Plan Reuse Graphplan Problem-specific planning Evolutionary computation approach

28. Plan Reuse Reuse old plans for new planning problems Consists of two steps plan matching plan modification Research findings generally, plan reuse is even harder than plan from scratch do better only when two problems are close enough plan matching could be the bottleneck

29. Graphplan Partial-order general planner Constructing a planning graph before search plan graph contains all possible actions that can be taken in each time step actions that interfere with one another can coexist in the graph More efficient than other general planners in some problems

30. Problem-specific Planning Heuristics combined during search heuristics is problem dependent cannot apply to other problems Usually outperforms general planners in specific problems Example in sliding-tile puzzle, accurate estimation of the distance between current state and goal state can speed up the search for a plan

31. Genetic Algorithms Parallel search and optimization algorithm Inspired by the basic rule of natural selection, survival-of-the-fittest Non-deterministic algorithm randomness is incorporated during implementation

32. Solution Formation Candidate solution is encoded as a list of genes, called chromosome Start with a population of randomly generated chromosomes Population is evolved in every generation evaluate each chromosome select chromosomes to the next generation apply genetic operations 110101

33. Genetic Operation crossover 110101011100 111101 010100 Crossover point Before crossover After crossover

34. Genetic Operation Mutation 110101 110100 Before mutation After mutation

35. GA Procedure Initialize the population Evaluate each chromosome in the population While the stopping condition is not met select chromosomes to the next generation crossover, mutation evaluate newly generated chromosomes End while

36. Evolutionary Computation Approach Non-deterministic algorithm Starts from a set of randomized plan Plans are evolved during generations In each generation evaluate the performance of the plan select the plans to next generation, based on performance crossover, mutation to create new plans

37. Evolutionary Computation Approach Solution encoding a list of floating-point number, each number maps to an action in the problem number of actions can vary during search 0.80.50.30.20.5 A4A3A2 A1

38. Evolutionary Computation Approach State-aware crossover crossover points are selected based on the matching of states Parent 1 Parent 2 Child 1 Child 2 s1 s2 Match(s1, s2) = true

39. Evolutionary Computation Approach Mutation randomly select an action and replace it with a random floating-point number 0.80.50.30.20.5 0.80.50.60.50.2 Before mutation After mutation

40. Evolutionary Computation Approach Fitness evaluation consists of two parts: goal fitness f g and cost fitness f c goal fitness: how the solutions reach the goal state cost fitness: the total cost of the solution overall fitness = a * f g + b * f c

41. Experimental Results Towers of Hanoi can solve the 5 and 6-disk case with high probability state-aware crossover does not perform well Sliding-tile puzzle can solve 3*3 problem case with high probability state-aware crossover perform much better since a looser definition of matching state is used Performance scalability is not good Heuristics may be helpful to improve the performance

42. References Artificial Intelligence: A Modern Approach. Stuart Russell and Peter Norvig. Prentice Hall. Artificial Intelligence: A New Symthesis. Nils Nilsson. Kaufmann. Intelligent Planning: A Decomposition and Abstraction Based Approach. Qiang Yang. Springer. Internet-Based Workflow Management: Towards a Semantic Web. Dan C. Marinescu. Wiley, 2002.

43. References Plan reuse versus plan generation: a theoretical and empirical analysis. B. Nebel, J. Koehler. Journal of Artificial Intelligence 76 (1995), 427-454. Fast planning through planning graph analysis. A. L. Blum, M. L. Furst Journal of Artificial Intelligence 90 (1997), 281- 300. Planning as heuristic search. B. Bonet, H. Geffner. Journal of Artificial Intelligence 129 (2001), 5-33. What it is, What it could be, An introduction to the Special Issue on Planning and Scheduling. D. McDermott, J. Handler. Journal of Artificial Intelligence 76 (1995), 1-16.

44. Concluding Words Failing to plan is planning to fail --Effie Jones