1. Integrating Planning & Scheduling Subbarao Kambhampati Scheduling: The State of the Art

2. Integrating Planning & Scheduling Subbarao Kambhampati Planning –Initial state & a set of Goals, –A library of actions »Preconditions/effects l Discrete/Continuous »Resource requirements Synthesize a sequence of actions capable of satisfying goals I = initial state G = goal state O i (prec)(effects) [ I ] O i O j O k O m [ G ] Planning vs. Scheduling Scheduling –Set of jobs (may have of tasks in some (partial) order) –Temporal constraints on jobs »EST, LFT, Duration –Contention constraints »Each task can be done on a subset of machines Find start times for jobs that are optimal (wrt make-spans, resource consumption etc) A Continuum --Research into planning and scheduling methods has largely been de-coupled Resource Reasoning Causal Reasoning

3. Integrating Planning & Scheduling Subbarao Kambhampati Why separate scheduling from planning? G Clearly, Scheduling can be seen as a sub-problem of temporal planning (unlike scheduling, planning also contains action selection). So, why separate it? –Reasons from automated planning point of view »If multiple agents make their plans and execute them on central resources, separating resource scheduling from individual agent planning makes sense »Even if a single agent is doing planning, it may be worth separating causal reasoning and resource reasoning l Such “de-coupling” improves efficiency, but at the expense of global optimality guarantees –Reasons from “real world practice” point of view »Historically, in many domains, action selection and automated causal reasoning was out of question (either because it couldn’t be modeled and solved or because the humans didn’t want to delegate it to automated methods »So, the only computer support for planning activity was given for resource scheduling (humans make plans, and schedulers do resource allocation)

4. Integrating Planning & Scheduling Subbarao Kambhampati Simple job-shop Scheduling: Brief Overview Jobshop scheduling –Set of jobs »Each job consists of tasks in some (partial) order –Temporal constraints on jobs »Sequencing constraints »Release times, deadlines, durations l EST, LFT, Duration –Contention/capacity constraints »Each task can be done on a subset of machines CSP Models –Time point model »Tasks as variables, Time points as values »EST, LFT, Machine contention as constraints –Inter-task precedences as variables (PCP model) CSP Techniques –Customized consistency enforcement techniques »ARC-B consistency »Edge-finding –Customized variable/value ordering heuristics »Contention-based »Slack-based –MaxCSP; B&B searches T2 M EST LFT P1,P2 st2 st1

5. Integrating Planning & Scheduling Subbarao Kambhampati Connections… G Scheduling seems to be “bounded length planning” where actions are already given to you! –The CSP encoding for scheduling will be a special case of the CSP encoding for bounded length planning! »The main difference is that in planning “steps” can correspond to one of many different actions, while in scheduling each step corresponds to a single action l “disjunctive scheduling” allows disjunction of jobs –Schedule optimality criteria are similar to “temporal plan” optimality criteria (makespan, tardiness, slack etc based) »The action cost doesn’t enter the picture since we don’t have any action choice.. G Scheduling (at least in the unary capacity resource case) is equivalent to solving disjunctive temporal networks!

6. Integrating Planning & Scheduling Subbarao Kambhampati 11/3: Scheduling continued Incompleteness and uncertainty (start)

7. Integrating Planning & Scheduling Subbarao Kambhampati Minimizing Schedule “Makespan” G Approach: –Establish lower and upper bounds on overall schedule end. –Repeatedly apply PCP to find the best solution within these bounds. G Details: –Generate schedule ignoring resource constraints to provide determine lower bound. –Apply one or more dispatch scheduling procedures to provide upper bound. –Apply PCP k times with “common deadlines” evenly distributed between these bounds.

8. Integrating Planning & Scheduling Subbarao Kambhampati Job Shop Scheduling as a CSP Precedence Constraints Capacity Constraints Variables: Start time st i l Domain: [est i l ….lst i l ] Precedence constraints: st i l + p i l  st j l Release times/deadlines: R  st i 1 ; st i 1 + p i 1  D Capacity Constraints : st i l + p i l  st j l  st j l + p j l  st i l Start Point Representation PCP Representation Variables: Ordering(i,j) for each task i and j Domain: {i-before-j, j-before-I,*} * means don’t care Dependent Var: Sti Constraints: Sequencing constraints: O(i,j)=i-bef-j Capacity constraints: O(i,j)=i-bef-j OR O(I,j,R)=j-bef-I {So, * is not a possibility} Release times, deadlines: R  sti1 ; sti1 +dui1  D Inter-variable constraints: O(i,j)=i-bef-j => sti +pi <= stj Disjunction typically comes through capacity constraints Making the problem harder: --Multi-capacity resources >> a machine that can handle 4 jobs at a time --Disjunctive activities >>schedule at least one of the following tasks --Setup constraints >>if you schedule task1, you need to schedule 3 and 4

9. Integrating Planning & Scheduling Subbarao Kambhampati The Choice Spectrum planning job-shop scheduling

10. Integrating Planning & Scheduling Subbarao Kambhampati The Choice Spectrum planning job-shop scheduling Job1 task1 < task2 < task3 < … Job2 Job3 … R3R7R1 Ordering choices only

11. Integrating Planning & Scheduling Subbarao Kambhampati The Choice Spectrum cascading levels of choice planning job-shop scheduling … ………… ……

12. Integrating Planning & Scheduling Subbarao Kambhampati The Choice Spectrum job-shop scheduling planning resource choices (RCSP) umfagoggin clavitracle fernambulator 5 11 Task1 Task2 Task3 Task4 Task5 Task6 Task8 Task7 [8,17] Ordering choices Resource choices

13. Integrating Planning & Scheduling Subbarao Kambhampati The Choice Spectrum job-shop scheduling planning resource choices (RCSP) alternative processes process3process7 process8 Task1 Task2 Task3 Task4 Task5 Task6 Task8 Task7 Ordering choices Resource choices Process choices

14. Integrating Planning & Scheduling Subbarao Kambhampati The Choice Spectrum planning ambitious spacecraft job-shop scheduling resource choices (RCSP) alternative processes Observation choices Instrument choices Calibration target choices Ordering choices Communication choices Instrument status choices

15. Integrating Planning & Scheduling Subbarao Kambhampati The Choice Spectrum planning Subset Selection ambitious spacecraft observation scheduling process planning job-shop scheduling resource choices (RCSP) alternative processes

16. Integrating Planning & Scheduling Subbarao Kambhampati Start point vs. PCP (not all that unlike State-space vs. PO) G Solution to the Start point encoding is a single feasible schedule G Handling of multi-capacity resources easy.. G Solution to the PCP encoding is a simple temporal network, all of whose dispatches are feasible schedules –Sort of like PO planning— which gives a PO plan, all of whose linearizations are valid plans G Conventional wisdom is that PCP does not scale well to multi-capacity resource scenarios

17. Integrating Planning & Scheduling Subbarao Kambhampati Contention-based Ordering Heuristics time demand Individual Demand of O1 for Rj time demand Aggregate Demand (of all O) for Rj time Individual Demand of O2 for Rj demand 369 0 03 6 63 0 9 Most critical region Operation O1 Start Time Distribution probability Start time 03 6 Contention: Aggregated curves found for each resource Critical Region: Where a resource is contended the most Most Critical Unassigned Operation: Contributes the largest area in critical region Variable Ordering Heuristic: Choose the most critical unassigned operation [Sadeh, 1991]

18. Integrating Planning & Scheduling Subbarao Kambhampati Other Constraint Propagation Ideas Through Resource constraints Arc-Bounds Edge-Finding Many other ideas: --Energy-based propagation --Time-table propagation etc. See Laborie, AIJ 2002 Arc-bounds is related to 2-consistency While edge-finding is related to k-consistency

19. Integrating Planning & Scheduling Subbarao Kambhampati Constraint Propagation 1.Pick ordering decision with the overall minimum slack min{slack[(u->v ); slack(v->u)] {Most constrained variable} 2. Assign that ordering decision the value for which the slack is higher. {Least constraining value}  B-slack = slack/sqrt(S) S is the ratio of min and max slacks of a given ordering. to normalize for the variation [20, 3] vs. [4, 3] PCP SCHEDULING

20. Integrating Planning & Scheduling Subbarao Kambhampati Current State of Scheduling as CSP G Constraint-based scheduling techniques are an integral part of the scheduling suites by ILOG/I2 –Optimization results comparable to best approximation algorithms currently known on standard benchmark problems. –Best known solutions to more idiosyncratic, “multi- product hoist scheduling” application (PCB electroplating). G Better in most large-scale problems than IP formulations

21. Integrating Planning & Scheduling Subbarao Kambhampati Slides beyond this were not covered…

22. Integrating Planning & Scheduling Subbarao Kambhampati Need for Integration G Most existing planners concentrate on action selection, ignoring resource allocation –Plan-based interfaces –Interactive decision support G Most existing schedulers concentrate only on resource allocation, ignoring action selection –E.g. HSTS operation scheduling G Many real-world problems require both capabilities –Supply Chain Management problems »I2, ILOG, Manugistics –Planning in domains with durative actions, continuous change »NASA RAX experiment

23. Integrating Planning & Scheduling Subbarao Kambhampati Why now? G Significant scale-up in plan synthesis in last 4-5 years –5/6 action plans in minutes to 100 action plans in minutes –Breakthroughs in search space representation, heuristic and domain-specific G Significant strides in our understanding of connections between planning and scheduling –Rich connections between planning and CSP/SAT/ILP »Vanishing separation between planning techniques and scheduling techniques

24. Integrating Planning & Scheduling Subbarao Kambhampati Approaches for Integration G Extend schedulers to handle action and resource choices G Extend planners to deal with resources, durative actions and continuous quantities G Coupled Architectures –De-coupled –Loosely Coupled (RealPlan System)

25. Integrating Planning & Scheduling Subbarao Kambhampati Approaches G Decoupled –Existing approaches G Monolithic –Extend Planners to handle time and resources –Extend Schedulers to handle choice G Loosely Coupled –Making planners and schedulers interact

26. Integrating Planning & Scheduling Subbarao Kambhampati Decoupled approaches (which is how Project Mgmt Done now) Management Technology Development Mid-lower manager Implementers MS Project (task planning) (scheduling)

27. Integrating Planning & Scheduling Subbarao Kambhampati Extending Planners G ZENO [Penberthy & Weld], IxTET [Ghallab & Laborie], HSTS/RAX [Muscettola] extend a conjunctive plan-space planner with temporal and numeric constraint reasoners G LPSAT [Wolfman & Weld] integrates a disjunctive state- space planner with an LP solver to support numeric quantities G IPPlan [Kautz & Walser; 99] constructs ILP encodings with numeric constraints G TGP [Smith & Weld; 99] supports actions with durations in Graphplan

28. Integrating Planning & Scheduling Subbarao Kambhampati Actions with Resources and Duration Load(P:package, R:rocket, L:location) Resources: ?h : robot hand Preconditions: Position(?h,L) [?s, ?e] Free(?h) ?s Charge(?h) > 5 ?s Effects: holding(?h, P) [?s, ?t1] depositing(?h,P,R) [?t2, ?e] Busy(?h) [?s, ?e] Free(?h) ?e Charge - =.03*(?e - ?s) ?e Constraints: ?t1 < ?t2 ?e - ?s in [1.0, 2.0] Capacity(robot) = 3 ?s?e Pos(?h,L) Hold(?h,P) dep(?h,P) [1,2] Free(?h) Busy(?h)

29. Integrating Planning & Scheduling Subbarao Kambhampati What planners are good for handling resources and time? G State-space approaches have an edge in terms of ease of monitoring resource usage –Time-point based representations are known to be better for multi- capacity resource constraints in scheduling G Plan-space approaches have an edge in terms of durative actions and continuous change –Notion of state not well defined in such cases (Too many states) –PCP representations are known to be better for scheduling with single-capacity resources

30. Integrating Planning & Scheduling Subbarao Kambhampati Extending Scheduling job-shop scheduling planning resource choices (RCSP) alternative processes process3process7 process8 Task1 Task2 Task3 Task4 Task5 Task6 Task8 Task7 Ordering choices Resource choices Process choices

31. Integrating Planning & Scheduling Subbarao Kambhampati Monolithic Architectures Scale Poorly G Extended planning systems are hard to control –RAX uses a very error-prone hand-coded search control strategy G Extended scheduling systems tend to lose effectiveness due to increased disjunction G Monolithic systems can sometimes show counter- intuitive behavior (by multiplying search failures)

32. Integrating Planning & Scheduling Subbarao Kambhampati Loosely Coupled Architectures Schedulers already routinely handle resources and metric/temporal constraints. –Let the “planner”concentrate on causal reasoning –Let the “scheduler” concentrate on resource allocation, sequencing and numeric constraints for the generated causal plan PLANNER SCHEDULER Causal Plan Schdule Need better coupling to avoid inter-module thrashing….

33. Integrating Planning & Scheduling Subbarao Kambhampati Making Loose Coupling Work G How can the Planner keep track of consistency? –Low level constraint propagation »Loose path consistency on TCSPs »Bounds on resource consumption, » LP relaxations of metric constraints –Pre-emptive conflict resolution The more aggressive you do this, the less need for a scheduler.. G How do the modules interact? –Failure explanations; Partial results

34. Integrating Planning & Scheduling Subbarao Kambhampati RealPlan--Master/Slave Planner does causal reasoning. Scheduler attempts resource allocation If scheduler fails, planner has to restart [Srivastava & Kambhampati ECP,99; AAAI, 2000]

35. Integrating Planning & Scheduling Subbarao Kambhampati

36. Integrating Planning & Scheduling Subbarao Kambhampati Performance of Master-Slave Coupling When scheduler fails, no specific guidance is given to the planner

37. Integrating Planning & Scheduling Subbarao Kambhampati RealPlan: Peer-to-Peer Explanation-directed backtracking between Planner and Scheduler Planner’s CSP: Variables: “goals” Values: “actions” Scheduler’s CSP: Variables: “Actions” Values: “Resources” Set of actions that cannot be scheduled Resource constraints activated by the selected actons

38. Integrating Planning & Scheduling Subbarao Kambhampati Inter-module Dependency Directed Backtracking Explanation Generation Explanation Translation Generation of Alternative Plan Scheduler’s Task Interface’s Task Planner’s Task Generate compact explanation of the Scheduler’s failure in allocating resources Translate the explanation into the form that make sense to the planner. Use the translated explanation to generate plan that avoid this failure. Planner’s CSP: Variables: “goals” Values: “actions” Scheduler’s CSP: Variables: “Actions” Values: “Resources” Set of actions that cannot be scheduled Resource constraints activated by the selected actons

39. Integrating Planning & Scheduling Subbarao Kambhampati Resource Domains: A 1, A 2, A 3 : {R 1, R 2 } A 4, A 5 : {S 1, S 2, S 3 } Resource Constraints: A 1  A 2 ; A 2  A 3 ; A 1  A 3 ; A 4  A 5 ; N 1 : {A 1 = R 1 } N 1 : {A 1 = R 1, A 2 = R 2 } N 1 : {A 1 = R 1, A 2 = R 2, A 4 = S 1 } N 1 : {A 1 = R 1, A 2 = R 2, A 4 = S 1, A 3 = R 1 }N 1 : {A 1 = R 1, A 2 = R 2, A 4 = S 1, A 3 = R 2 } A 1 = R 1 A 2 = R 2 A 4 = S 1 A 3 = R 1 A 3 = R 2 Subset of variables that can not be assigned values (reason of failure): (A 1, A 2, A 3 )

40. Integrating Planning & Scheduling Subbarao Kambhampati

41. Integrating Planning & Scheduling Subbarao Kambhampati A temporal planner supporting causal plan synthesis CSP-based Finite capacity resource scheduler Mixed Integer/Linear programming module for metric constraints Mission Profile Executor interface Plans with annotated waypoints Execution status Replanning requests Mission modifications Plan criticism Evolving specs Next-Generation Realplan

42. Integrating Planning & Scheduling Subbarao Kambhampati Summary & Conclusion G Motivated the need for integrating Planning and Scheduling G Discussed the state of the art in Planning and Scheduling G Discussed approaches for Integrating them –Loosely coupled architectures are a promising approach