1. ICUAS 2014 Towards Real Time Scheduling for Persistent UAV Service: A Rolling Horizon MILP Approach, RHTA and the STAH Heuristic Byung Duk Song, Jonghoe Kim, and James R. Morrison* Department of Industrial and Systems Engineering KAIST, South Korea Thursday, May 29, 2014

2. ICUAS 2014 – 2 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

3. ICUAS 2014 – 3 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

4. ICUAS 2014 – 4 Motivation Increasing demand for commercial small size UAVs Applications: Patrol, tracking, communication relay, environmental / fire / national boundary monitoring, cartography, disaster relief Limitations: Flight time & mission duration Long duration, multiple customers and persistent service require - Collection of UAVs, refueling stations, automatic operation system - Methods to orchestrate their operations

5. ICUAS 2014 – 5 Motivation: Application Example (1) Example 1: Tracking a moving ground target Station 1 Station 2 Station 3

6. ICUAS 2014 – 6 Motivation: Application Example (2) Example 2: UAV border patrol Station 1 Station 2

7. ICUAS 2014 – 7 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

8. ICUAS 2014 – 8 Goal and Ideas Provide uninterrupted UAV service with continuous appearance of mission requests (or unanticipated system state changes) Persistent UAV operation via service stations Real time service perspective: - Incorporate current UAV information (location, fuel level) - Efficient solving algorithms Uninterrupted customer service ■ Goal ■ Ideas

9. ICUAS 2014 – 9 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

10. ICUAS 2014 – 10 Mathematical Formulation: Assumptions ■ Assumptions 1. Moving target’s path and location at specific times are known. 2. Current location and fuel levels are known. 3. Recharge time for a UAV depends on the remaining fuel(battery) amount 4. UAV travel speed is constant

11. ICUAS 2014 – 11 Mathematical Formulation: Split Job Start End Service station 1 Service station 2 1 2 3 10 4 5 6 7 8 9 Job 1 UAV 1 UAV 2 ▪ Objective moves - From point (50,250) to (950,350) - From 13:10 to 13:20 Split job Start point End point Start time End time 150,250150,25013:1013:11 2150,250250,25013:1113:12 3250,250350,25013:1213:13 4350,250450,25013:1313:14 5450,250550,25013:1413:15 6550,250650,25013:1513:16 7650,250750,25013:1613:17 8750,250850,25013:1713:18 9850,250950,25013:1813:19 10950,250950,35013:1913:20 ■ Split job: Moving objective path is divided into set of split jobs

12. ICUAS 2014 – 12 Mathematical Formulation: Notation ■ Notation i, j:Indices for jobs s:Index for stations k:Index for UAVs r:Index of a UAV’s r th flight NJNJ :Number of split jobs N UAV :Number of UAVs in the system N STA :Number of recharge stations NRNR :Maximum number of flight per UAV during the time horizon M:Large positive number (xjs, yjs)(xjs, yjs):Start point of split job j (xje, yje)(xje, yje):End point of split job j D ij :Distance from split job i th finish point to split job j th start point, D ij ≠ D ji Ei:Start time of split job i Pi:Processing time or split job i qkqk :Maximum traveling time of UAV k S ok :Initial location(station) of UAV k TS k :Travel speed of UAV k H:Required time for fully recharge(refuel) the empty fuel tank (battery) q k,ini :Initial remaining battery (fuel) of UAV k

13. ICUAS 2014 – 13 Mathematical Formulation: Notation ■ Notation ΩJΩJ := {1, …, N J }, Set of split jobs Ω INI := Set of initial UAV location Ω SS := {N J +1, N J +3, …, N J +2∙ N STA -1}, set of start station Ω SE := {N J +2, N J +4, …, N J +2∙ N STA }, set of end station ΩAΩA : = (Ω D U Ω SS U Ω SE ) = {1,…, N J +2∙N STA }, set of all jobs and recharge stations ■ Decision Variables ▪ X ijkr = 1 if UAV k processes split job j or recharges at station j after processing split job i or recharging at station i during the r th flight; 0, otherwise ▪ C ikr is job i’s start time by UAV k during its r th flight or UAV k’s recharge start time at station i; otherwise its value is 0. ▪ q kr is total fuel (battery) consumption amount for UAV k during its r th flight

14. ICUAS 2014 – 14 Mathematical Formulation ■ Mathematical formulation Network flow & job assignment

15. ICUAS 2014 – 15 Mathematical Formulation Start time constraints Fuel constraints Decision variables

16. ICUAS 2014 – 16 Formulation Demonstration (CPLEX) ■ Problem description : Problem map Customer 1 Customer 2 Station ▲ UAV U1 S1 S2 S3 U2 U3 U4 U5 U6

17. ICUAS 2014 – 17 Formulation Demonstration (CPLEX) ■ Problem description : Split job & station & UAV information Stationxy S1394126 S217079 S372229 UAVxy Speed (meter / min) q k,ini (min) 15601302403 22801802408 3170792408 43001002406 53941262406 6902002408 Customer Split jo b Start pointEnd point Start time xyxy 1 1596167532161 5 2532161483129 6 343812943294 7 44329437291 8 53729131587 9 63158726274 10 72627421899 11 821899171134 12 9171134142186 13 10142186102225 14 1110222554251 15 12542516266 16 2 1345864479105 8 1447910551463 9 155146353615 10

18. ICUAS 2014 – 18 Formulation Demonstration (CPLEX) ■ Result of example 1 UAVSchedulingq ini q k1 q k2 1Initial → 1,2,3,4,5 → Station 165.580 2Initial → 13, 14, 15 → Station 184.630 3Initial (Station 2)800 4Initial → 6,7,8,9,10,11,12 → Station 3107.40 5Initial (Station 1)600 6Initial → Station 380.140 - Objective value : 662.570 - CPU time: 5.42 seconds

19. ICUAS 2014 – 19 Formulation Demonstration (CPLEX) ■ Result of example 2 : Change q k,ini UAVSchedulingq ini q k1 q k2 1Initial location → 1,2 → Station 132.590 2 Initial location → Station 1 → 3,4,5,6,7,8,9,10,11,12→ Station 3 80.5210.7 3Initial location (Station 2) → Station 2800 4Initial location → 13,14,15 → Station 164.40 5Initial location (Station 1) → Station 1600 6Initial location → Station 380.140 - Objective value : 809.067 - CPU time: 9.17 seconds

20. ICUAS 2014 – 20 Formulation Demonstration (CPLEX) ■ These examples shows us - Real time perspectives: Current location and fuel level of UAVs - Access to the persistent service - Sharing of multiple stations

21. ICUAS 2014 – 21 Formulation Demonstration (CPLEX) ■ Computational limit of CPLEX System parametersCPLEX NJNJ N STA N UAV CPU time Obj. value 15365.42662.570 3036N/A 4536N/A 45510N/A 6036N/A 60510N/A Needs for alternative solution approaches to achieve real time perspective

22. ICUAS 2014 – 22 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

23. ICUAS 2014 – 23 Solution Approach: Solution Properties ■ Analysis on solutions 1) Strict condition: Feasibility issue - Fuel - Split job start time 2) Flexible condition: Solution quality issue - Total moving distance - The number of split jobs which UAV can serve in that flight - Trade off between direct & indirect flight

24. ICUAS 2014 – 24 Solution Approach: Solution Properties ■ Analysis on solutions 3) Sequential assignment of split jobs UAVSchedulingq ini q k1 q k2 1Initial → 1,2,3,4,5 → Station 165.580 2Initial → 13, 14, 15 → Station 184.630 3Initial (Station 2)800 4Initial → 6,7,8,9,10,11,12 → Station 3107.40 5Initial (Station 1)600 6Initial → Station 380.140

25. ICUAS 2014 – 25 Solution Approach: Solution Properties ■ Trade off between direct and indirect flight Service station 1 1 2 3 10 4 5 6 7 8 9 UAV 1 - Without replenishment - Short moving distance - Serve less number of split jobs

26. ICUAS 2014 – 26 Solution Approach: Solution Properties ■ Trade off between direct and indirect flight - Replenishment on the station - Long moving distance - Serve more number of split jobs Service station 1 1 2 3 10 4 5 6 7 8 9 UAV 1

27. ICUAS 2014 – 27 Solution Approach: STAH ■ Based on the analysis of solutions 1) Strict condition 2) Flexible condition Assign UAVs which satisfy strict conditions (A jk ) Contribution value of direct and indirect flight V D (k) = { αw( # of split jobs) –(1- α)(total moving distance) } ∙ A jk V I (k) = { αw( # of split jobs) –(1- α)(total moving distance) } ∙ A jk Select the flight which provide highest value From customer 1 (earliest customer), split jobs are assigned sequentially

28. ICUAS 2014 – 28 Solution Approach: RHTA ■ Receding horizon task assignment algorithm (RHTA) – Classical heuristic (petal algorithm) for vehicle routing problem – Iterative IP model – Generate good quality feasible solution in short time – Classical RHTA was modified: Eliminate the resource selection components Modify the equation of replenishment time for a UAV

29. ICUAS 2014 – 29 Solution Approach: RHTA ■ Petal : Sequence of split jobs to be served by a UAV Step 1: Enumerate all feasible petals for each UAV k – Generate petal – Check feasibility of petal – Evaluate of petal value Fuel limitation Time window

30. ICUAS 2014 – 30 Solution Approach: RHTA STEP 2: Solve a single IP to select petal STEP 3: Assign split job of selected petal to UAVs STEP 4: Send exhausted UAVs to a service station - if there are remaining jobs on the list, go to STEP 1. STEP 5: Send any UAVs not located at a station to a station A job can be processed by at most one UAV At least P job should be selected Minimize total travel distance

31. ICUAS 2014 – 31 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

32. ICUAS 2014 – 32 Results for Various Problem Sizes ■ Results System parametersCPLEXSTAHRHTA NJNJ N STA N UAV CPU time Obj. value CPU time Obj. value Gap CPU time Obj. value Gap 15365.42662.5700.004665.2610.406 %0.203664.8570.345 % 3036N/A 0.0081010.24-0.3271274.991- 4536N/A 0.0151314.34-0.4581901.816- 45510N/A 0.0151346.96-0.6081656.667- 6036N/A 0.0161642.34-0.5782677.189- 60510N/A 0.0311565.90-0.7651997.992 - - STAH and RHTA obtain very near optimal values. - CPLEX issues an out of memory error. - STAH and RHTA derive feasible solution in a reasonably short time.

33. ICUAS 2014 – 33 Real Time Operation with New Service Request Arrival Customer Split job Start pointEnd point Start ti me xyxy 1 1’2627421899 11 2’21899171134 12 3’171134142186 13 4’142186102225 14 5’10222554251 15 6’542516266 16 3 7’413210430184 13 8’430184454162 14 9’454162482137 15 10’48213745110 16 11’4511044282 17 12’4428242865 18 UAVxyqkqk q k,ini TS k 139412612 240 226274127.629240 31707912 240 453615122.325240 539412612 240 67222912 240 ■ Remaining jobs and UAV information at T=11. ■ During the UAV service of example 1, new service request arrived at T=10.

34. ICUAS 2014 – 34 Real Time Operation with New Service Request Arrival Initial plan UAVSchedulingq k,ini Obj. value CPU Time (sec) 1Initial location → 1, 2 → Station 13 809.0679.17 2Initial location → Station 1 → 3,4,5,6,7,8,9,10,11,12 → Station 38 3Initial location (Station 2)8 4Initial location → 13, 14, 15 → Station 16 5Initial location (Station 1)6 6Initial location → Station 38 New plan from time 11 UAVSchedulingq k,ini Obj. value CPU Time (sec) 1(394,126) (Station 1)12 411.3064.67 2(262,74) → 1’,2’,3’,4’,5’,6’ → Station 37.629 3(170,79) (Station 2)12 4(536,15) → Station 12.325 5(394,126) (Station 1) → 7’,8’,9’,10’,11’,12’ → Station 112 6(72,229) (Station 3)12

35. ICUAS 2014 – 35 Real Time Operation with New Service Request Arrival Station 1 Station 2 Station 3 T = 4 Customer 1 Customer 2 5678910 Customer 3 111213141516171819

36. ICUAS 2014 – 36 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

37. ICUAS 2014 – 37 Concluding Remarks Fleets of UAVs conducting missions over a field of operations can achieve persistence - Supported by shared service stations distributed over the field - Real time operation to handle new arrival of service request MILP scheduling model - Allows a mobile robot to perform persistent service by sharing base for energy resupply - Maximize service quality by serving 100% service requests by orchestration of components - Demonstration of real time perspectives Develop a efficient heuristics to overcome computational limit of MILP formulation - STAH - RHTA Application & numerical examples

38. ICUAS 2014 Back-UP Materials

39. ICUAS 2014 – 39 Solution approach: STAH ■ STAH procedure STEP 1: Arrange customers in the non-decreasing order of (first) split job start time – Let P= {1,2,…,p} be the set of customers and P t be the set of split jobs in customer index t – Element of P t also arranged according to the non-decreasing order of split job start time. Set P=1. STEP 2: From customer P, split jobs are assigned to the UAVs. – Calculate V D (k) and V I (k) – Among V D (k) and V I (k) for every UAV, the biggest function value and corresponding UAV schedule is selected STEP 3: Update parameters – Current location, fuel level, available time of UAVs – Remaining split job information for customer P is updated

40. ICUAS 2014 – 40 Solution approach: STAH ■ STAH procedure STEP 4: Repeat step 2 to 3 until every split job is assigned to UAV in customer P. STEP 5: Set P=P+1, and repeat step 2,3 and 4. This step is repeated until P=|P|+1. STEP 6: Send any UAVs not located at a station to a station

41. ICUAS 2014 – 41 Solution approach: STAH ■ Pseudo code (STAH)

42. ICUAS 2014 – 42 Feasibility Y Y Y Y N Solution approach: RHTA ■ Petal : sequence of split jobs to be served by a UAV Step 1: Enumerate all feasible petals for each UAV k – Generate petal (Limitation of maximum size of petal) – Check feasibility of petal – Evaluate value of petal Petal size Instance of petal 1{2} 1{3} 2{2,3} 3{6,7,8} 4{1,2,3,4} Fuel limitation Time window Travel distance 6 10 (6+3) (10+4+5) infeasible

43. ICUAS 2014 – 43 Solution approach: RHTA STEP 2: Solve a single IP for all UAVs to minimize the travel distance – Select petals to be used subject to A job can be processed by at most one UAV At least P job should be selected Minimize total travel distance

44. ICUAS 2014 – 44 Solution approach: RHTA STEP 3: Assign split job to UAV – Assign UAV to a first split job of selected petal – Update system state (UAV location, fuel, available time and job information) – Remove assigned job from remaining job list STEP 4: Find exhausted UAVs – Send UAVs that do not have any feasible petals to a service station – Update system state – Return to STEP 1 if remaining job list is not empty STEP 5: Send any UAVs not located at a station to a station

45. ICUAS 2014 – 45 Solution approach: RHTA ■ Pseudo code (RHTA)