1. EUROCONTROL RESEARCH CENTRE1 21/11/2016 An algorithmic approach to air path computation Devan SOHIER LDCI-EPHE (Paris) 23/11/04

2. EUROCONTROL RESEARCH CENTRE2 21/11/2016 Outline  Introduction  Situation  Modeling  Markov decision processes  Conclusion and perspectives

3. EUROCONTROL RESEARCH CENTRE3 21/11/2016 Outline  Introduction  Situation  Modeling  Markov decision processes  Conclusion and perspectives

4. EUROCONTROL RESEARCH CENTRE4 21/11/2016 Problem  Find safe trajectories for all aircrafts in a given portion of the airspace  Taking into account stochastic events:  Temporary flyover interdiction (due to meterological conditions or some other reason)  Deviation  …

5. EUROCONTROL RESEARCH CENTRE5 21/11/2016 Levels of ATC  ATC can be divided in several levels:  Strategic level for mid-term planning of flights:  many aircrafts  meteorological uncertainties  Tactical level for short-term management  few aircrafts (2 or 3)  uncertainties about the location (deviation)

6. EUROCONTROL RESEARCH CENTRE6 21/11/2016 Outline  Introduction  Situation  Modeling  Solution  Conclusion and perspectives

7. EUROCONTROL RESEARCH CENTRE7 21/11/2016 Aircrafts crossing  Two aircrafts x and y  go from x d and y d to x f and y f  risks of conflict  Find a « good » trajectory for each of them (safe, and cheap)

8. EUROCONTROL RESEARCH CENTRE8 21/11/2016 Aircrafts crossing  Minimize: ∫x’ 2 +y’ 2 dt  Under the safety constraint: d(x,y)>d s and some constraints on speed!  We work on (x, y)  R 6  The trajectory of (x, y) is composed of segments of straight lines and arcs of ellipses in R 6

9. EUROCONTROL RESEARCH CENTRE9 21/11/2016 Stochastic aircrafts crossing  A stochastic deviation (d 1, d 2 ) is added to the model:  minimize: E[∫(x+d 1 )’ 2 +(y+d 2 )’ 2 dt]  under the constraint: d(x+d 1,y+d 2 )>d s

10. EUROCONTROL RESEARCH CENTRE10 21/11/2016 Problems  Continuous modeling of the deviation:  difficult to determine  difficult to exploit (a continuous time markovian modeling cannot be adequate)  Moreover will it provide useful information?  discretization

11. EUROCONTROL RESEARCH CENTRE11 21/11/2016 Outline  Introduction  Situation  Modeling  Markov decision processes  Conclusion and perspectives

12. EUROCONTROL RESEARCH CENTRE12 21/11/2016 Our Modeling  Existing modelings use:  Continuous space  Continuous time  We propose a discrete modeling more adequate to programming

13. EUROCONTROL RESEARCH CENTRE13 21/11/2016 Bricks  Discretization of the airspace :  Bricks (parallelepipeds)  Size = safety distances

14. EUROCONTROL RESEARCH CENTRE14 21/11/2016 Modeling of the airspace  To improve the modeling:  Use of a honeycomb paving  Discrete time

15. EUROCONTROL RESEARCH CENTRE15 21/11/2016 Voronoi paving  Introduction of dynamic safety distances by the use of a Voronoi paving

16. EUROCONTROL RESEARCH CENTRE16 21/11/2016 The graph  Allowed movements are modeled by a graph

17. EUROCONTROL RESEARCH CENTRE17 21/11/2016 Statistics  Markov (resp. semi-markovian) processes are a simple, general and well-known modeling  All the information is contained in the most recent observation(s)  The deviation evolves in a memoryless way: the deviation at time t+1 only depends on the deviation at time t (resp. t, t-1, …, t-k)

18. EUROCONTROL RESEARCH CENTRE18 21/11/2016 Statistics  Preliminary Markov tests on the deviation  highlights a different behaviour of transversal and longitudinal deviations  semi-Markovian with a dependence to history of about 5

19. EUROCONTROL RESEARCH CENTRE19 21/11/2016 Outline  Introduction  Situation  Modeling  Markov decision processes  Conclusion and perspectives

20. EUROCONTROL RESEARCH CENTRE20 21/11/2016 Static vs. dynamic  Static solutions  Worst-case analysis  Loss of airspace  Dynamicity  Adapt the solution to the current situation  Use all the available information  But dynamicity requires more computing power

21. EUROCONTROL RESEARCH CENTRE21 21/11/2016 Dynamic programming  An optimal path (x t ) t>0 is such that for all t0, (x t ) t>t0 is also optimal starting from the situation x t0  Continuous time  difficult to apply  Through discretization we obtain an adequate framework

22. EUROCONTROL RESEARCH CENTRE22 21/11/2016 Markov Decision Process  Dynamic programming with a Markov « opponent »  Find rules giving the decision to make in each situation, taking into account the probabilities of evolution under constraints  Safe: in each safe situation, a safe reaction is proposed

23. EUROCONTROL RESEARCH CENTRE23 21/11/2016 Markov Decision Process  We define for each deviation d and situation s: d k,d (s, g)=min{d(s,s’)+  d2 p d,d2 d k-1,d2 (s’, g)/s  s’} Next k,d (s)=argmin{ d(s,s’)+  d2 p d,d2 d k-1,d2 (s’,g)/ s  s’ } with g=(x f, y f ) the final situation, for all k: d k, d (s 1, s 2 )=  if s 1 +d is forbidden and d k, d (s, s)=0  When these quantities do not evolve any longer, we obtain the optimization rules.

24. EUROCONTROL RESEARCH CENTRE24 21/11/2016 Complexity  Complexity of this MDP grows with the size of the history (5 in this case) of the Markov chain  Much more efficient than the computation of exact optimal solutions

25. EUROCONTROL RESEARCH CENTRE25 21/11/2016 Outline  Introduction  Situation  Modeling  Markov decision processes  Conclusion and perspectives

26. EUROCONTROL RESEARCH CENTRE26 21/11/2016 Conclusions  Dynamic computation of air trajectories may save much airspace without decreasing the safety  Markovian (memoryless) discrete modelings provide an efficient and adequate framework allowing computer programming of the solution

27. EUROCONTROL RESEARCH CENTRE27 21/11/2016 Works in Progress  Works in collaboration with L. El Ghaoui (Berkeley), A. d’Aspremont (Princeton) on the strategic level

28. EUROCONTROL RESEARCH CENTRE28 21/11/2016 Perspectives  Statistical validation of the modeling  Use of continuous modeling and decision rules, and discretization of the solution  Use of pretopological tools to refine the notion of conflict  Decentralization of the decision by the use of negociations protocols  Introduction of some equity constraints