Presentation is loading. Please wait.

Presentation is loading. Please wait.

©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 Persistent UAV Service: An Improved Scheduling Formulation and Prototypes.

Similar presentations


Presentation on theme: "©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 Persistent UAV Service: An Improved Scheduling Formulation and Prototypes."— Presentation transcript:

1 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 Persistent UAV Service: An Improved Scheduling Formulation and Prototypes of System Components Byung Duk Song, Jonghoe Kim, Jeongwoon Kim, Hyorin Park, James R. Morrison* and David Hyunchul Shim Department of Industrial and Systems Engineering Department of Aerospace Engineering KAIST, South Korea Friday, May 31, 2013

2 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 2 Presentation Overview Motivation UAV service system concept Comparison with existing research UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station System demonstration Concluding remarks

3 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 3 Presentation Overview Motivation UAV service system concept Comparison with existing research UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station System demonstration Concluding remarks

4 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 4 Motivation Large expensive UAVs – Usually military purpose – Operate for many hours – Travel long distances Small inexpensive UAVs – A lot of application area such as tracking, communication relay, environmental / fire / national boundary monitoring, cartography, disaster relief and so on. – Limited duration of mission – Limited distance Methods to ensure persistent operation can increase effectiveness of small UAVs – Collection of UAVs, refueling stations, automatic guidance – Algorithms to orchestrate the system operations

5 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 5 Presentation Overview Motivation UAV service system concept Comparison with existing research UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station System demonstration Concluding remarks

6 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 6 UAV Service System Concept UAV service system Persistent UAV service Random arrival of customer information Random path and duration Vision technology Heterogeneous UAVs UAV operation system Automatic replenishment station Central planning

7 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 7 UAV Service System Concept UAV service system Persistent UAV service Random path and duration Vision technology Heterogeneous UAVs UAV operation system Automatic replenishment station Central planning

8 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 8 UAV Service System Concept UAV service system Persistent UAV service Vision technology Heterogeneous UAVs UAV operation system Automatic replenishment station Central planning

9 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 9 Presentation Overview Motivation UAV service system concept Comparison with existing research UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station System demonstration Concluding remarks

10 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 10 Comparison with Existing Research <Automated 1.5 Hour persistent surveillance mission with three autonomous vehicles> 1.Persistent path following with multiple shared service stations distributed across the field of operations 2.Prototype components for a system seeking to provide a persistent UAV security escort service [1] M. Alighanbari and J. P. How, “Decentralized task assignment for unmanned aerial vehicle”, Proceedings of the 44 th IEEE Conference on Decision and Control, and the European Control Conference, December 2005 [2] M. Valenti, D. Dale, J. P. How and D. P. de Farias, “Mission health management for 24/7 persistent surveillance operations”, AIAA Guidance, Navigation and Control Conference and Exhibit, August 2007

11 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 11 Presentation Overview Motivation UAV service system concept Comparison with existing research UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station System demonstration Concluding remarks

12 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 12 UAV Service System: Components and Prototype UAV UAV guidance system Central planning by MILP UAV schedule Automatic control Tracking Customer information Replenishment station Customer feedback Automatic replenishment Web or smart phone

13 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 13 Central Planning: Deterministic Customer Paths UAV UAV guidance system Central planning by MILP UAV schedule Automatic control Tracking Customer information Replenishment station Customer feedback Automatic replenishment Web or smart phone

14 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 14 Persistent UAV Service ■ Persistent UAV service system with heterogeneous UAVs and multiple service stations - A system of UAVs that is supported by automated replacement systems can support long term or even indefinite duration missions in a near autonomous mode with multiple service stations - The UAVs can return to any service station, replenish their resources and resume their duties

15 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 15 Customer Paths ■ To follow a time-space trajectory, the trajectory is divided into pieces (split jobs) Start End Service station 1 Service station 2 1 2 3 10 4 5 6 7 8 9 Split Job 1 UAV 1 UAV 2 ▪ Objective moves - From point (50,250) to (950,350) - From 13:10 to 13:20 Split job Start point End point Start time End time 150,250150,25013:1013:11 2150,250250,25013:1113:12 3250,250350,25013:1213:13 4350,250450,25013:1313:14 5450,250550,25013:1413:15 6550,250650,25013:1513:16 7650,250750,25013:1613:17 8750,250850,25013:1713:18 9850,250950,25013:1813:19 10950,250950,35013:1913:20 UAV 1 UAV 2

16 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 16 Assumptions ■ Assumptions 1. Moving target’s path and location at specific times are known. 2. UAVs start its travel from a recharge station 3. Recharge time for a UAV is constant 4. Initially all UAV batteries or fuel tanks are empty 5. UAV travel speed is constant

17 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 17 Initial Mathematical Formulation ■ Notation i, j:Indices for jobs s:Index for stations k:Index for UAVs r:Index of a UAV’s r th flight NJNJ :Number of split jobs N UAV :Number of UAVs in the system N STA :Number of recharge stations NRNR :Maximum number of flight per UAV during the time horizon M:Large positive number (xjs, yjs)(xjs, yjs):Start point of split job j (xje, yje)(xje, yje):End point of split job j D ij :Distance from split job i th finish point to split job j th start point, D ij ≠ D ji Ei:Start time of split job i Pi:Processing time or split job i qkqk :Maximum traveling time of UAV k S ok :Initial location(station) of UAV k TS k :Travel speed of UAV k

18 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 18 Initial Mathematical Formulation ■ Notation ΩJΩJ := {1, …, N J }, Set of split jobs Ω JD := {1, …, N J +1}, Set of split jobs and dummy jobs Ω SS := {N J +2, N J +4, …, N J +2∙ N STA }, set of UAV flight start station Ω SE := {N J +3, N J +5, …, N J +2∙ N STA +1}, set of UAV flight end station ΩAΩA : = (Ω JD U Ω SS U Ω SE ) = {1,…, N J +2∙N STA +1}, set of all jobs and recharge stations ■ Decision Variables ▪ X ijkr = 1 if UAV k processes split job j or recharges at station j after processing split job i or recharging at station i during the r th flight; 0, otherwise ▪ C ikr is job i’s start time by UAV k during its r th flight or UAV k’s recharge start time at station i; otherwise its value is 0. ▪ Y ikr = 1 if UAV k processes split job i during its r th flight; 0, otherwise.

19 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 19 Initial Mathematical Formulation ■ Mathematical formulation Recharge station constraints Initial recharge station constraints Split job assignment constraints Start time constraints Fuel constraints Dummy job constraints Decision variables

20 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 20 Reduce Variables and Constraints ■ Mathematical formulation Recharge station constraints Initial recharge station constraints Split job assignment constraints Start time constraints Fuel constraints Dummy job constraints Decision variables

21 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 21 Improved Formulation ■ Mathematical formulation Recharge station constraints Initial recharge station constraints Split job assignment constraints Start time constraints Fuel constraints Decision variables

22 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 22 Improved Formulation ■ Complexity : Number of decision variables and constraints Kim et al. (2012)Improved formulationDifference Total # of binary decision variable N UAV ∙N R ∙{(N J +2∙N STA +1) 2 +(N J +2∙N STA +1)} N UAV ∙N R ∙(N J +2∙N STA ) 2 (N J +2∙N STA +1) 2 +(N J +2∙N STA +1) -(N J +2∙N STA ) 2 Total # of continu ous decision variable N UAV ∙N R ∙ (N J +2∙N STA +1)N UAV ∙N R ∙(N J +2∙N STA )N UAV ∙N R Total # of decision variable N UAV ∙N R ∙{(N J +2∙N STA +1) 2 +2∙(N J +2∙N STA +1)} N UAV ∙N R ∙{(N J +2∙N STA ) 2 +N J +2∙N STA } (N J +2∙N STA +1) 2 +N J +2∙N STA +2 -(N J +2∙N STA ) 2 Total # of constraints N UAV {2(N R -1)∙N STA +1} + N J (3N UAV ∙N R +2) + N UAV ∙N R {( N J +N STA +1) 2 +2N J +4N STA +5} N UAV {2(N R -1)∙N STA + N R ∙ N STA +1} + N J (N UAV ∙N R +2) + N UAV ∙N R {( N J +N STA ) 2 +2N J +3N STA +3} N UAV ∙N R {( N J +N STA +1) 2 -( N J +N STA ) 2 +N STA +2} +2∙N J ∙ (N UAV ∙N R ) -N UAV ∙ (N R ∙ N STA )

23 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 23 Computational Results Kim et al. (2012)Improved formulation CPU Time Redu ction NJNJ N STA N UAV # of D.V # of const CPU Time Obj. Value # of D.V # of const CPU Time Obj. Value 822 7807223.0020486245661.8420481.6x 1436 5796500215.841846504042224.3618463.6x 1536 63365508220.37245544468035.787246.2x 2048 1438412048N/A 1299210592348.972894- ■ Comparison of computational result

24 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 24 UAV Guidance System UAV UAV guidance system Central planning by MILP UAV schedule Automatic control Tracking Customer information Replenishment station Customer feedback Automatic replenishment Web or smart phone

25 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 25 UAV Guidance System ■ Roles of UAV guidance system 1. Receive and implement the schedule from the MILP. 2. Convert the video from the UAV cameras into usable information for directing the motion of the UAVs 3. Enable a human overseer to monitor the UAV progress via video and adjust feedback control gain values for various situations 4. Allows for a human overseer to initiate emergency actions such as immediate landing.

26 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 26 UAV Guidance System 1280 720 pixel front camera 320 240 pixel belly camera ■ System components

27 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 27 UAV Guidance System 1. The color video from the camera is acquired via TCP port and processed using OpenCV framework. 2. The image is separated into three RGB channels. These three images are used to determine the color of the targeted image. 3. Control inputs including the longitudinal-lateral tilt angles, height and yaw angular velocity are calculated from the number and mean coordinate of target pixels in the processed image.

28 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 28 UAV Guidance System ■ P-D gain controller block diagram

29 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 29 Automatic Replenishment Station UAV UAV guidance system Central planning by MILP UAV schedule Automatic control Tracking Customer information Replenishment station Customer feedback Automatic replenishment Web or smart phone

30 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 30 Automatic Replenishment Station ▪ Each AR Drone 2.0 uses a three cell lithium polymer battery ▪ four copper leads (three for each terminal and one for the ground terminal) were threaded from the battery inside the UAV to the four feet of the drone ▪ The service station consists of four pads, one for each foot of the drone. ▪ Each such pad connects to the UAV battery via the leads on the drone feet

31 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 31 Presentation Overview Motivation UAV service system concept Comparison with existing research UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station System demonstration Concluding remarks

32 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 32 System Demonstration: Layout UAV Start station Assigned job End station Service start time Service end time 111,2,3,42210 225,6,7,831018 ■ Demonstration description Split job 1 Split job 2 Split job 8 Split job 7 ∙ ∙ ∙ 5m

33 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 33 System Demonstration: Video ■ Demonstration video

34 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 34 Presentation Overview Motivation UAV service system concept Comparison with existing research UAV service system: Components and prototype – Central planning: Deterministic customer paths – UAV guidance system – Automatic replenishment station System demonstration Concluding remarks

35 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 35 Concluding Remarks Towards a persistent UAV service Components of such a system – System orchestration MILP (deterministic customer paths) Improved formulation Reduced computational time – UAV guidance system Vision for UAV localization relative to customer, location flags and platforms Feedback control for UAV via iPad controller – Automatic replenishment stations (battery recharge) Demonstration of proposed UAV service system Future directions – Real time customer requests – Random customer behavior during service – Implementation outdoors – Improved UAV localization algorithms

36 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 Back up materials

37 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 37 Improvement : Efficient formulation To enhance the cplex computational power, efficient mathematical formulation was developed Delete unnecessary decision variable and dummy job concepts - Delete Y jkr decision variable because C jkr decision variable can replace it. ▪ C ikr is job i’s start time by UAV k during its r th flight or UAV k’s recharge start time at station i; otherwise its value is 0. ▪ Y ikr = 1 if UAV k processes split job i during its r th flight; 0, otherwise. C jkr =0 >0 Y jkr =0 =1

38 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 38 Improvement : Efficient formulation Delete the concept of dummy job which is used for idle UAVs by allowing direct flight from start(end)station to end(start) station stations Dummy job

39 ©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 – 39 Literature Review Scheduling methods without a distance or time restriction – T. Shima and C. Schumacher, “Assignment of cooperating UAVs to simultaneous tasks using genetic algorithm,” In Proc. AIAA Guidance, Navigation, and Control Conference and Exhibit, San Francisco, 2005 – J. Zeng, X. Yang L. Yang and G. Shen, “Modeling for UAV resource scheduling under mission synchronization,” Journal of Systems Engineering and Electronics, Vol. 21, No. 5, 2010, pp. 821-826 Scheduling methods for limited flight duration – A. L. Weinstein and C. Schumacher, “UAV scheduling via the vehicle routing problem with time windows,” In Proc. AIAA Infotech@Aerospace 2007 Conference and Exhibit, Rohnert Park, California, 2007 – T. Shima, S. Rasmussen and D. Gross, “Assigning micro UAVs to task tours in an urban terrain,” IEEE Transactions on Control Systems Technology, Vol. 15, No. 4, 2007, pp. 601 – 612 – Y.S. Kim, D.W. Gu and I. Postlethwaite, “Real-time optimal mission scheduling and flight path selection, IEEE Transactions on Automatic Control, Vol. 52, No. 6, 2007, pp. 1119-1123. – B. Alidaee, H. Wang, and F. Landram, “A note on integer programming formulations of the real-time optimal scheduling and flight selection of UAVS,” IEEE Transactions of Control Systems Technology, Vol. 17, No. 4, 2009, pp.839-843 Scheduling method for persistent UAV operation – M. Alighanbari and J. P. How, “Decentralized task assignment for unmanned aerial vehicle”, Proceedings of the 44 th IEEE Conference on Decision and Control, and the European Control Conference 2005 seville, spain, december 12-15, 2005 Battery recharge/exchange methods – J. How, thesis papers at MIT, 2005, 2007 – A.S. Kurt, B.H. Clarence, R.R. Johnhenri, D.W. Richardson, Z.H. White, Q. Elizabeth and G. Anouck, “Autonomous Battery Swapping System for Small-scale Helicopters”, 2010 IEEE International Conference on Robotics and Automation – R. Godzdanker, M. J. Rutherford and K. P. Valavanis, “ISLANDS: A self-leveling platform for autonomous miniature UAVs”, 2011 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp 170-175 – A.O.S. Koji, K.F. Paulo and James R. Morrison, “Automatic battery replacement system for UAVs: Analysis and design” Journal of Intelligent and Robotic Systems, Special Issue on Unmanned Aerial Vehicles (Springer), a Special Volume on Selected Papers from ICUAS’11, Vol. 65, No. 1, pp. 563-586, January 2012. First published online September 9, 2011 – M. Valenti, D. Dale, J. P. How and D. P. de Farias, “Mission health management for 24/7 persistent surveillance operations”, AIAA Guidance, Navigation and Control Conference and Exhibit, 20-23 August 2007, Hilton Head, South Carolina


Download ppt "©2013 – James R. Morrison – ICUAS’13 – Atlanta, Georgia, USA – May 28-31, 2013 Persistent UAV Service: An Improved Scheduling Formulation and Prototypes."

Similar presentations


Ads by Google