ICUAS 2014 Towards Real Time Scheduling for Persistent UAV Service: A Rolling Horizon MILP Approach, RHTA and the STAH Heuristic Byung Duk Song, Jonghoe Kim, and James R. Morrison* Department of Industrial and Systems Engineering KAIST, South Korea Thursday, May 29, 2014

ICUAS 2014 – 2 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

ICUAS 2014 – 3 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

ICUAS 2014 – 4 Motivation Increasing demand for commercial small size UAVs Applications: Patrol, tracking, communication relay, environmental / fire / national boundary monitoring, cartography, disaster relief Limitations: Flight time & mission duration Long duration, multiple customers and persistent service require - Collection of UAVs, refueling stations, automatic operation system - Methods to orchestrate their operations

ICUAS 2014 – 5 Motivation: Application Example (1) Example 1: Tracking a moving ground target Station 1 Station 2 Station 3

ICUAS 2014 – 6 Motivation: Application Example (2) Example 2: UAV border patrol Station 1 Station 2

ICUAS 2014 – 7 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

ICUAS 2014 – 8 Goal and Ideas Provide uninterrupted UAV service with continuous appearance of mission requests (or unanticipated system state changes) Persistent UAV operation via service stations Real time service perspective: - Incorporate current UAV information (location, fuel level) - Efficient solving algorithms Uninterrupted customer service ■ Goal ■ Ideas

ICUAS 2014 – 9 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

ICUAS 2014 – 10 Mathematical Formulation: Assumptions ■ Assumptions 1. Moving target’s path and location at specific times are known. 2. Current location and fuel levels are known. 3. Recharge time for a UAV depends on the remaining fuel(battery) amount 4. UAV travel speed is constant

ICUAS 2014 – 11 Mathematical Formulation: Split Job Start End Service station 1 Service station Job 1 UAV 1 UAV 2 ▪ Objective moves - From point (50,250) to (950,350) - From 13:10 to 13:20 Split job Start point End point Start time End time 150,250150,25013:1013: ,250250,25013:1113: ,250350,25013:1213: ,250450,25013:1313: ,250550,25013:1413: ,250650,25013:1513: ,250750,25013:1613: ,250850,25013:1713: ,250950,25013:1813: ,250950,35013:1913:20 ■ Split job: Moving objective path is divided into set of split jobs

ICUAS 2014 – 12 Mathematical Formulation: Notation ■ Notation i, j:Indices for jobs s:Index for stations k:Index for UAVs r:Index of a UAV’s r th flight NJNJ :Number of split jobs N UAV :Number of UAVs in the system N STA :Number of recharge stations NRNR :Maximum number of flight per UAV during the time horizon M:Large positive number (xjs, yjs)(xjs, yjs):Start point of split job j (xje, yje)(xje, yje):End point of split job j D ij :Distance from split job i th finish point to split job j th start point, D ij ≠ D ji Ei:Start time of split job i Pi:Processing time or split job i qkqk :Maximum traveling time of UAV k S ok :Initial location(station) of UAV k TS k :Travel speed of UAV k H:Required time for fully recharge(refuel) the empty fuel tank (battery) q k,ini :Initial remaining battery (fuel) of UAV k

ICUAS 2014 – 13 Mathematical Formulation: Notation ■ Notation ΩJΩJ := {1, …, N J }, Set of split jobs Ω INI := Set of initial UAV location Ω SS := {N J +1, N J +3, …, N J +2∙ N STA -1}, set of start station Ω SE := {N J +2, N J +4, …, N J +2∙ N STA }, set of end station ΩAΩA : = (Ω D U Ω SS U Ω SE ) = {1,…, N J +2∙N STA }, set of all jobs and recharge stations ■ Decision Variables ▪ X ijkr = 1 if UAV k processes split job j or recharges at station j after processing split job i or recharging at station i during the r th flight; 0, otherwise ▪ C ikr is job i’s start time by UAV k during its r th flight or UAV k’s recharge start time at station i; otherwise its value is 0. ▪ q kr is total fuel (battery) consumption amount for UAV k during its r th flight

ICUAS 2014 – 14 Mathematical Formulation ■ Mathematical formulation Network flow & job assignment

ICUAS 2014 – 15 Mathematical Formulation Start time constraints Fuel constraints Decision variables

ICUAS 2014 – 16 Formulation Demonstration (CPLEX) ■ Problem description : Problem map Customer 1 Customer 2 Station ▲ UAV U1 S1 S2 S3 U2 U3 U4 U5 U6

ICUAS 2014 – 17 Formulation Demonstration (CPLEX) ■ Problem description : Split job & station & UAV information Stationxy S S S UAVxy Speed (meter / min) q k,ini (min) Customer Split jo b Start pointEnd point Start time xyxy

ICUAS 2014 – 18 Formulation Demonstration (CPLEX) ■ Result of example 1 UAVSchedulingq ini q k1 q k2 1Initial → 1,2,3,4,5 → Station Initial → 13, 14, 15 → Station Initial (Station 2)800 4Initial → 6,7,8,9,10,11,12 → Station Initial (Station 1)600 6Initial → Station Objective value : CPU time: 5.42 seconds

ICUAS 2014 – 19 Formulation Demonstration (CPLEX) ■ Result of example 2 : Change q k,ini UAVSchedulingq ini q k1 q k2 1Initial location → 1,2 → Station Initial location → Station 1 → 3,4,5,6,7,8,9,10,11,12→ Station Initial location (Station 2) → Station Initial location → 13,14,15 → Station Initial location (Station 1) → Station Initial location → Station Objective value : CPU time: 9.17 seconds

ICUAS 2014 – 20 Formulation Demonstration (CPLEX) ■ These examples shows us - Real time perspectives: Current location and fuel level of UAVs - Access to the persistent service - Sharing of multiple stations

ICUAS 2014 – 21 Formulation Demonstration (CPLEX) ■ Computational limit of CPLEX System parametersCPLEX NJNJ N STA N UAV CPU time Obj. value N/A 4536N/A 45510N/A 6036N/A 60510N/A Needs for alternative solution approaches to achieve real time perspective

ICUAS 2014 – 22 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

ICUAS 2014 – 23 Solution Approach: Solution Properties ■ Analysis on solutions 1) Strict condition: Feasibility issue - Fuel - Split job start time 2) Flexible condition: Solution quality issue - Total moving distance - The number of split jobs which UAV can serve in that flight - Trade off between direct & indirect flight

ICUAS 2014 – 24 Solution Approach: Solution Properties ■ Analysis on solutions 3) Sequential assignment of split jobs UAVSchedulingq ini q k1 q k2 1Initial → 1,2,3,4,5 → Station Initial → 13, 14, 15 → Station Initial (Station 2)800 4Initial → 6,7,8,9,10,11,12 → Station Initial (Station 1)600 6Initial → Station

ICUAS 2014 – 25 Solution Approach: Solution Properties ■ Trade off between direct and indirect flight Service station UAV 1 - Without replenishment - Short moving distance - Serve less number of split jobs

ICUAS 2014 – 26 Solution Approach: Solution Properties ■ Trade off between direct and indirect flight - Replenishment on the station - Long moving distance - Serve more number of split jobs Service station UAV 1

ICUAS 2014 – 27 Solution Approach: STAH ■ Based on the analysis of solutions 1) Strict condition 2) Flexible condition Assign UAVs which satisfy strict conditions (A jk ) Contribution value of direct and indirect flight V D (k) = { αw( # of split jobs) –(1- α)(total moving distance) } ∙ A jk V I (k) = { αw( # of split jobs) –(1- α)(total moving distance) } ∙ A jk Select the flight which provide highest value From customer 1 (earliest customer), split jobs are assigned sequentially

ICUAS 2014 – 28 Solution Approach: RHTA ■ Receding horizon task assignment algorithm (RHTA) – Classical heuristic (petal algorithm) for vehicle routing problem – Iterative IP model – Generate good quality feasible solution in short time – Classical RHTA was modified: Eliminate the resource selection components Modify the equation of replenishment time for a UAV

ICUAS 2014 – 29 Solution Approach: RHTA ■ Petal : Sequence of split jobs to be served by a UAV Step 1: Enumerate all feasible petals for each UAV k – Generate petal – Check feasibility of petal – Evaluate of petal value Fuel limitation Time window

ICUAS 2014 – 30 Solution Approach: RHTA STEP 2: Solve a single IP to select petal STEP 3: Assign split job of selected petal to UAVs STEP 4: Send exhausted UAVs to a service station - if there are remaining jobs on the list, go to STEP 1. STEP 5: Send any UAVs not located at a station to a station A job can be processed by at most one UAV At least P job should be selected Minimize total travel distance

ICUAS 2014 – 31 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

ICUAS 2014 – 32 Results for Various Problem Sizes ■ Results System parametersCPLEXSTAHRHTA NJNJ N STA N UAV CPU time Obj. value CPU time Obj. value Gap CPU time Obj. value Gap % % 3036N/A N/A N/A N/A N/A STAH and RHTA obtain very near optimal values. - CPLEX issues an out of memory error. - STAH and RHTA derive feasible solution in a reasonably short time.

ICUAS 2014 – 33 Real Time Operation with New Service Request Arrival Customer Split job Start pointEnd point Start ti me xyxy 1 1’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ UAVxyqkqk q k,ini TS k ■ Remaining jobs and UAV information at T=11. ■ During the UAV service of example 1, new service request arrived at T=10.

ICUAS 2014 – 34 Real Time Operation with New Service Request Arrival Initial plan UAVSchedulingq k,ini Obj. value CPU Time (sec) 1Initial location → 1, 2 → Station Initial location → Station 1 → 3,4,5,6,7,8,9,10,11,12 → Station 38 3Initial location (Station 2)8 4Initial location → 13, 14, 15 → Station 16 5Initial location (Station 1)6 6Initial location → Station 38 New plan from time 11 UAVSchedulingq k,ini Obj. value CPU Time (sec) 1(394,126) (Station 1) (262,74) → 1’,2’,3’,4’,5’,6’ → Station (170,79) (Station 2)12 4(536,15) → Station (394,126) (Station 1) → 7’,8’,9’,10’,11’,12’ → Station 112 6(72,229) (Station 3)12

ICUAS 2014 – 35 Real Time Operation with New Service Request Arrival Station 1 Station 2 Station 3 T = 4 Customer 1 Customer Customer

ICUAS 2014 – 36 Presentation Overview Motivation Goal and ideas Mathematical formulation Solution approaches: STAH & RHTA Numerical examples Concluding remarks

ICUAS 2014 – 37 Concluding Remarks Fleets of UAVs conducting missions over a field of operations can achieve persistence - Supported by shared service stations distributed over the field - Real time operation to handle new arrival of service request MILP scheduling model - Allows a mobile robot to perform persistent service by sharing base for energy resupply - Maximize service quality by serving 100% service requests by orchestration of components - Demonstration of real time perspectives Develop a efficient heuristics to overcome computational limit of MILP formulation - STAH - RHTA Application & numerical examples

ICUAS 2014 Back-UP Materials

ICUAS 2014 – 39 Solution approach: STAH ■ STAH procedure STEP 1: Arrange customers in the non-decreasing order of (first) split job start time – Let P= {1,2,…,p} be the set of customers and P t be the set of split jobs in customer index t – Element of P t also arranged according to the non-decreasing order of split job start time. Set P=1. STEP 2: From customer P, split jobs are assigned to the UAVs. – Calculate V D (k) and V I (k) – Among V D (k) and V I (k) for every UAV, the biggest function value and corresponding UAV schedule is selected STEP 3: Update parameters – Current location, fuel level, available time of UAVs – Remaining split job information for customer P is updated

ICUAS 2014 – 40 Solution approach: STAH ■ STAH procedure STEP 4: Repeat step 2 to 3 until every split job is assigned to UAV in customer P. STEP 5: Set P=P+1, and repeat step 2,3 and 4. This step is repeated until P=|P|+1. STEP 6: Send any UAVs not located at a station to a station

ICUAS 2014 – 41 Solution approach: STAH ■ Pseudo code (STAH)

ICUAS 2014 – 42 Feasibility Y Y Y Y N Solution approach: RHTA ■ Petal : sequence of split jobs to be served by a UAV Step 1: Enumerate all feasible petals for each UAV k – Generate petal (Limitation of maximum size of petal) – Check feasibility of petal – Evaluate value of petal Petal size Instance of petal 1{2} 1{3} 2{2,3} 3{6,7,8} 4{1,2,3,4} Fuel limitation Time window Travel distance 6 10 (6+3) (10+4+5) infeasible

ICUAS 2014 – 43 Solution approach: RHTA STEP 2: Solve a single IP for all UAVs to minimize the travel distance – Select petals to be used subject to A job can be processed by at most one UAV At least P job should be selected Minimize total travel distance

ICUAS 2014 – 44 Solution approach: RHTA STEP 3: Assign split job to UAV – Assign UAV to a first split job of selected petal – Update system state (UAV location, fuel, available time and job information) – Remove assigned job from remaining job list STEP 4: Find exhausted UAVs – Send UAVs that do not have any feasible petals to a service station – Update system state – Return to STEP 1 if remaining job list is not empty STEP 5: Send any UAVs not located at a station to a station

ICUAS 2014 – 45 Solution approach: RHTA ■ Pseudo code (RHTA)