1. 1 Planning and Scheduling to Minimize Tardiness John Hooker Carnegie Mellon University September 2005

2. 2 The Problem Given a set of tasks, each with a deadline…

3. 3 The Problem Given a set of tasks, each with a deadline… …and processing facilities that run at different speeds.

4. 4 The Problem Allocate tasks to facilities.

5. 5 The Problem Allocate tasks to facilities. Schedule tasks on each facility (cumulative scheduling)

6. 6 The Problem Schedule tasks on each facility (cumulative scheduling) Each task has a given processing time on each facility. Allocate tasks to facilities.

7. 7 The Problem Schedule tasks on each facility (cumulative scheduling) Each task has a given processing time on each facility. Each task consumes resources at a given rate on each facility. Allocate tasks to facilities.

8. 8 The Problem Schedule tasks on each facility (cumulative scheduling) Each task has a given processing time on each facility. Each task consumes resources at a given rate on each facility. Each facility has a resource limit. Allocate tasks to facilities.

9. 9 The Problem Schedule tasks on each facility (cumulative scheduling) Objectives:  Minimize number of late tasks.  Minimize total tardiness Allocate tasks to facilities.

10. 10  In practice, problem is often solved by give-and- take. Approach Planner Schedulers Planner allocates tasks to facilities

11. 11  In practice, problem is often solved by give-and- take. Approach Planner Schedulers If there are problems, schedulers telephone planners and ask for a different allocation.

12. 12  In practice, problem is often solved by give-and- take. Approach Planner Schedulers Planner allocates tasks differently

13. 13  In practice, problem is often solved by give-and- take. Approach Planner Schedulers And so forth until agreement is reached.

14. 14  In practice, problem is often solved by give-and- take. Approach Planner Schedulers And so forth until agreement is reached.  Benders decomposition is a mathematical formulation of this process.  Planning is the master problem.  Scheduling is the subproblem.  Telephone calls are Benders cuts.

15. 15  Decomposition permits hybrid solution:  Apply MILP to planning master problem.  MILP is generally better at resource allocation.  Apply CP to scheduling subproblem.  CP is generally better at scheduling. Approach  Use logic-based Benders.  Since classical Benders requires that the subproblem be a linear or nonlinear programming problem.

16. 16 Related Work 1995 (JH & Yan) – Logic-based Benders.  Applied to logic circuit verification. 2000 (JH) – Theory of logic-based Benders.  Branch-and-check proposed. 2000 (JH) – Application to planning & scheduling proposed.  Combine CP & MILP.  Application to integer programming, SAT. 2001 (Jain & Grossmann) – Application to machine assignment and disjunctive scheduling.  Simple Benders cuts, since subproblem is a feasibility problem.

17. 17 Related Work 2001 (Thorsteinsson) – Branch and check applied to Jain & Grossmann problems.  Update solution of master problem. 2002 (Harjunkowski & Grossmann) – Generalization of Jain & Grossmann to multistage. 2002 (Timpe) – Polypropylene batch scheduling at BASF.  Similar approach applied to automobile assembly (Peugeot/Citroën), paint mixing (Barbot). 2003 (JH, Ottosson) – Integer programming.  Benders cuts less obvious, since subproblem is an optimization problem.  Suitable for stochastic integer programming.

18. 18 Related Work 2004 (JH) – Min-cost and min-makespan planning & cumulative scheduling.  Benders cuts less obvious in min makespan, since subproblem is optimization problem. 2004 (Cambazard et al.) – Real-time scheduling of computer processes.  CP master problem. 2004 (Chiu & Xia) – Integer programming.  Used min-conflict + classical Benders cuts. 2004 (Maravelias and Grossmann) – Batch scheduling in chemical plants. 2004 (Correa, Langevin & Rousseau) – Automated guided vehicles.

19. 19 Today’s Presentation  Planning and cumulative scheduling to minimize tardiness.  Number of late jobs.  Total tardiness.  Main contribution:  Analysis of cumulative constraint to provide linear relaxation of the scheduling subproblem.  This is most difficult when minimizing total tardiness.  Add relaxation to master problem.  The relaxation is essential to computational performance.

20. 20 Logic-Based Benders Decomposition Master Problem Subproblem Benders cuts for all iterations h Solution of master problem Basic idea: Search over values of x in master problem. For each examined, solve subproblem for y.

21. 21 Logic-Based Benders Decomposition Subproblem After solving the subproblem, generate a Benders cut where is the optimal value of the subproblem. The Benders cut is based on a logical analysis of subproblem solution. Re-solve the master problem and continue until it has the same optimal value as the subproblem.

22. 22  Decompose problem into assignment + resource-constrained assign tasks scheduling to facilities schedule tasks on each facility Applying Benders to Planning & Scheduling  Use logic-based Benders to link these.  Solve: master problem with MILP -- good at resource allocation subproblem with Constraint Programming -- good at scheduling  We will use Benders cuts that require no internal information from the CP solver.

23. 23 Notation p ij = processing time of task j on facility i c ij = resource consumption of task j on facility i C i = resources available on facility i C1C1 task 1 task 2 task 3 task 4 task 5 p 11 c 11 Total resource consumption  C i at all times. C2C2 p 22 c 22 Facility 1 Facility 2

24. 24 Objective functions Minimize # late tasks = facility assigned to task j start time of task j Minimize tardiness = deadline for task j

25. 25 Minimize # Late Tasks: MILP Model = 1 if task j is late

26. 26 Minimize # Late Tasks: MILP Model = 1 if task j is late Number of discrete times t = 1 if task j starts at time t on facility i

27. 27 Minimize # Late Tasks: MILP Model Task j starts at one time on one facility = 1 if task j is late Number of discrete times t = 1 if task j starts at time t on facility i

28. 28 Minimize # Late Tasks: MILP Model Task j starts at one time on one facility Tasks underway at time t consume  C i in resources = 1 if task j is late Number of discrete times t = 1 if task j starts at time t on facility i

29. 29 Minimize # Late Tasks: MILP Model Task j starts at one time on one facility Tasks underway at time t consume  C i in resources Tasks observe time windows = 1 if task j is late Number of discrete times t = 1 if task j starts at time t on facility i

30. 30 Minimize # Late Tasks: CP Model y j = facility assigned to task j start time of task j

31. 31 Minimize # Late Tasks: CP Model y j = facility assigned to task j start times of tasks assigned to facility i start time of task j

32. 32 Minimize # Late Tasks: CP Model Observe resource limit on each facility y j = facility assigned to task j start times of tasks assigned to facility i start time of task j

33. 33 Minimize # Late Tasks: CP Model Observe time windows Observe resource limit on each facility y j = facility assigned to task j start times of tasks assigned to facility i start time of task j

34. 34 Minimize # Late Tasks: Logic-Based Benders Master Problem: Assign tasks to facilities Iteration h = 1 if task j is assigned to facility i

35. 35 Subproblem: Schedule tasks assigned to each facility Separates into an independent scheduling problem on each facility i. Solve by constraint programming. Set of tasks assigned to facility i by solution of master problem.

36. 36 Benders Cuts Min # late tasks on facility i (solution of subproblem) Lower bound on # late tasks on facility i

37. 37 Benders Cuts Min # late tasks on facility i (solution of subproblem) Lower bound on # late tasks on facility i subset of J hi for which min # late tasks is still L hi * (found by heuristic that repeatedly solves subproblem on facility i )

38. 38 Benders Cuts Min # late tasks on facility i (solution of subproblem) To reduce # late tasks, must remove one of the tasks in from facility i.

39. 39 Benders Cuts Min # late tasks on facility i (solution of subproblem) subset of J hi for which min # late tasks is still L hi * (found by heuristic that repeatedly solves subproblem on facility i ) Smaller subset of J hi for which min # late tasks is L hi *  1 (found while running same heuristic)

40. 40 Benders Cuts Min # late tasks on facility i (solution of subproblem) To reduce # late tasks by more than 1, must remove one of the tasks in from facility i.

41. 41 Benders Cuts These Benders cuts are added to the master problem in each iteration h.

42. 42 Relaxation of Subproblem Lower bound on # late tasks on facility i

43. 43 Relaxation of Subproblem Lower bound on # late tasks on facility i CiCi task 1 task 2 task 3 Set of tasks assigned to facility i with deadline at or before d j

44. 44 Relaxation of Subproblem Lower bound on # late tasks on facility i CiCi task 1 task 2 task 3 Set of tasks assigned to facility i with deadline at or before d j Area = p i1 c i1

45. 45 Relaxation of Subproblem Lower bound on # late tasks on facility i CiCi task 1 task 2 task 3 Area = p i1 c i1 Area of tasks assigned to facility i with deadline at or before d j

46. 46 Relaxation of Subproblem CiCi task 1 task 2 task 3 Lower bound on (makespan  latest deadline)

47. 47 Relaxation of Subproblem CiCi task 1 task 2 task 3 Lower bound on (makespan  latest deadline) Max processing time

48. 48 Relaxation of Subproblem Min # of late jobs on facility i

49. 49 Relaxation of Subproblem Min # of late jobs on facility i This relaxation is added to the master problem at the outset.

50. 50 Minimize Total Tardiness: MILP Model tardiness of task j

51. 51 Minimize Total Tardiness: CP Model

52. 52 Minimize Total Tardiness: Logic-Based Benders Master Problem: Assign tasks to facilities Iteration h = 1 if task j is assigned to facility i

53. 53 Subproblem: Schedule tasks assigned to each facility Separates into an independent scheduling problem on each facility i. Solve by constraint programming. Set of tasks assigned to facility i by solution of master problem.

54. 54 Benders Cuts Min tardiness on facility i (solution of subproblem) Lower bound on tardiness for facility i

55. 55 Benders Cuts Min tardiness on facility i (solution of subproblem) Lower bound on tardiness for facility i To reduce tardiness on facility i, must remove one of the tasks assigned to it.

56. 56 Benders Cuts Min tardiness on facility i (solution of subproblem) Set of tasks that can be removed, one at a time from facility i without reducing min tardiness.

57. 57 Benders Cuts Min tardiness on facility i when all tasks in Z hi are removed simultaneously. Set of tasks that can be removed, one at a time from facility i without reducing min tardiness.

58. 58 Benders Cuts Min tardiness on facility i when all tasks in Z hi are removed simultaneously. Set of tasks that can be removed, one at a time from facility i without reducing min tardiness. To reduce tardiness below on facility i, must remove one of the tasks in J hi \ Z hi

59. 59 Benders Cuts These Benders cuts are added to the master problem in each iteration h.

60. 60 Subproblem Relaxation I Lower bound on total tardiness for facility i

61. 61 Subproblem Relaxation I task 1 task 2 task 3 Set of tasks assigned to facility i with deadline at or before d k Lower bound on total tardiness for facility i

62. 62 Subproblem Relaxation I task 1 task 2 task 3 Area of tasks assigned to facility i with deadline at or before d k Lower bound on total tardiness for facility i

63. 63 Subproblem Relaxation I task 1 task 2 task 3 Lower bound on total tardiness Lower bound on total tardiness for facility i

64. 64 Lemma. Consider a min tardiness problem that schedules tasks 1, …, n on facility i, where d 1    d n. The min tardiness T* is bounded below by where and  is a permutation of 1, …, n such that Subproblem Relaxation II

65. 65 Example of Lemma jdjdj p ij c ij p ij c ij 13236 24428 35515 CiCi task 1 task 2 task 3 d1d1 d2d2 d3d3 Min tardiness = 4

66. 66 Idea of proof Let  0 (1), …,  0 (n) be order of jobs in any optimal solution, so that and min tardiness is T* Consider bubble sort on  0 (1), …,  0 (n) to obtain 1,…,n. Let  0,…,  S be resulting sequence of permutations, so that  s,  s+1 differ by a swap and  s (j) = j. For a permutation  of 1,…,n let where

67. 67 Now we have since swap k and k+1 So since A  a, B  b areas def. of 

68. 68 From the lemma, we can write the relaxation where To linearize this, we write and where Writing relaxation II

69. 69 Computational Results  Random problems on 3 facilities.  Facilities run at different speeds.  All release times = 0, due dates differ.  Random due date parameters set so that a few tasks tend to be late.  No precedence or other side constraints.  They would make the problem easier.  Implement with OPL Studio  CPLEX for MILP.  ILOG Scheduler for CP. Use AssignAlternatives & SetTimes.

70. 70 Min # late tasks Smaller problems Tasks Time (sec) CP MILP Benders Min # late tasks 14 1092 5.8 0.51 382 8.0 0.71 265 3.2 0.72 85 2.6 1.32 5228 1315 6653 16 304 2.7 0.50 ? 31 0.21 310 22 0.41 4925 29 2.72 19 5.7 244 18 >7200 2.0 0.10 ? 8.0 0.21 >7200 867 8.51 >7200 6.3 1.42 >7200 577 3.42

71. 71 Tasks Time (sec) MILP Benders Best solution MILP Benders 20 97 0.400 >7200 2.3(1)1 219 5.011 >7200 11(2)2 843 16633 22 16 1.300 >7200 3.7(1)1 >7200 49(3)2 >7200 3453(5)2 >7200 (6) 24 25 0.800 >7200 18(1)0 >7200 62(2)0 >7200 124(3)1 >7200 234(2)1 Min # late tasks Larger problems ( ) = optimality not proved

72. 72 Tasks Time (sec) with relax without relax 16 0.5 2.6 0.4 1.5 0.2 1.3 2.7 4.2 24 18 0.1 1.1 0.2 0.7 3.4 3.3 1.4 15 8.5 11 20 0.4 88 2.3 9.7 5.0 63 11 19 166 226 Effect of subproblem relaxation Min # late tasks

73. 73 Min total tardiness Smaller problems Tasks Time (sec) CP MILP Benders Min tardiness 14 838 7.0 6.11 7159 34 3.72 1783 45 1915 >7200 73 4019 >7200 326926 16 >7200 2 0.10 >7200 8 0.23 >7200 867 8.55 >7200 6 1.411 >7200 577 3.431 18 187 2.80 15 5.33 46 495 256 4711 >7200 120314

74. 74 Tasks Time (sec) MILP Benders Best solution MILP Benders 20 105 1800 4141 2311 39 2944 1442 33288 >7200 (75)(37) 22 6 1900 584 3722 >7200 (120)(40) >7200 (162)(46) >7200 (375)(141) 24 10 32400 >7200 94(20)0 >7200 110(57)0 >7200 (20)(5) >7200 (25)(7) Min total tardiness Larger problems ( ) = optimality not proved

75. 75 Tasks Time (sec) with relax without relax 16 1.4 4.4 2.1 6.5 4.2 30 156 199 765 763 18 2.8 10 5.3 17 47 120 49 354 1203 5102 20 18 151 23 1898 29 55 332 764 >7200 Effect of subproblem relaxation Min total tardiness

76. 76 Conclusions  Benders is significantly faster than MILP (which is faster than CP).  Speedup for min # late tasks is often orders of magnitude.  Smaller speedup for min total tardiness.  Problems with more than a few late tasks are hard for all methods.  Benders finds better suboptimal solutions.  Even when Benders fails to prove optimality, it obtains much better solutions than MILP in the same time period.  Subproblem relaxation is important.  Linear relaxation of cumulative constraint can be critical to performance, especially when minimizing total tardiness.

77. 77 Future Research  Implement branch-and-check for Benders problem.  Exploit dual information from the subproblem solution process (e.g. edge finding).  Explore other problem classes.  Integrated long- and short-term scheduling  Vehicle routing  SAT (subproblem is renamable Horn)  Stochastic IP