1. 1 J.Y Greff, L. Idoumghar and R. Schott TDF && IECN / LORIA - INRIA July 2002 Using Markov Decision Processes to the Frequency Assignment Problem

2. 2 Summary Radio network planning Markov Decision Processes Experimental results Conclusion

3. 3 Radio network planning: Network dimensionning 10 9 8 6 5 4 1 3 2 0

4. 4 Radio network planning: Interference phenomena

5. 5 3 Radio network planning: Frequency Assignment 1 10 2 9 5 4 3 8 60 3 2 3 2 1 31 2 2 1 3 3 2 2 2 1 1 3 1 Constraints graph

6. 6 = 0 = 1 = 2 8 10 7 6 9 12 1 0 2 4 5 11 13 2 0 0 2 1 0 0 1 1 0 2 2 1 1 0 3 4 5 1 2 6 9 10 11 7 8 12 13 3 Radio network planning: Frequency Assignment

7. 7 One tool of AI that can be used to get Optimal solution under a stochastic domain. A formal description of MDP is a Tuple (S, A, T, R) : * S: is a finite set of states of the world, * A: is a finite set of actions, * T: S x A  P Transition function of states (T(s,a,s’) represents the probability p s,s’ of reaching state s’ starting in state s’ and given the action a) * R: S x A   reward function. R(s,a) represents the reward of executing action a. Markov Decision Processes

8. 8

9. 9 Our Approach * S: is given by a set of transmitters, * A: each action correspond to the search of a best frequency to assign to a transmitter s. If there does not exist a best frequency, the action consists in keeping the frequency assigned to this transmitter, Markov Decision Processes

10. 10 Let M= (m i,j ) n*n be a symetric matrix, where m i,j with i  j represents the manimum frequency separation required to satisfy the constraint between vertices vi and vj Let P = (p i,j ) n*n be a transition matrix, where p i,j represents a probability of reaching a state j starting in state i, For i:= 1 to n Do sum := 0 For j:= 1 to n Do sum := sum + m i,j End For j:= 1 to n Do p i,j := m i,j /sum End * Transition matrix is calculated by using the algorithm given as following: Markov Decision Processes

11. 11 : represents the minimal gap in frequency required to satisfy the constraint between vertices v s and v s’ weight associate to the constraint m s,s’  s,s’ :  s,s’  R(s, a )  s,s’  * If  s,s’ m  (s’) a  0 s,s’  If  s,s’ m  (s’) a  a  s,s’ m  m Markov Decision Processes * The Reward function is defined as follows:

12. 12

13. 13 Markov Decision Processes: Probabilistic tabu search stop := false tabu_list :=  let A be a best solution While not stop do with a probability p t, chose c in the best neighbouring configuration forbidden by tabu list let d be the best neighbouring configuration which is not forbidden by the tabu list. This configuration is choosen by using a uniform probability choose the best configuration B between c and d wich minimizes an objective function A := B adjust tabu_list stop := stopping criterion done L. Idoumghar and R. Schott and M. Alabau, New Hybrid Genetic Algorithm for the Frequency Assignment Problem, Journal of IEEE Transactions on Broadcasting, Mars 2002, Vol. 48, N. 1, 27-34.

14. 14 Experimental results Comparison of the results obtained by differents approachs that are used to solve instance1 (more than 970 transmitters and more than 12900 constraints) of the frequency assignment problem.

15. 15 Comparison of the results obtained by differents approachs that are used to solve instance2 (more than 970 transmitters and more than 25550 constraints) of the frequency assignment problem. Experimental results

16. 16 Comparison of the results obtained by differents approachs that are used to solve instance3 (more than 970 transmitters and more than 43800 constraints) of the frequency assignment problem. Experimental results

17. 17 Experimental results

18. 18 Experimental results: Quality plan estimationin regard of radio system characteristics coverage area service area percentage = Ca : Coverage Area Ca Sa E Sa : Service Area E : Transmitter

19. 19 65 25 % of transmitters have a service area at most equal to 65 % of the cover area Or 75 % of transmitters have a service area at less equal to 65 % of the cover area 0100 0 25 70 75 % of transmitters have a service area at less equal to 70 % of the cover area % accumulated of transmitters Ratio service area / coverage area Experimental results: Quality plan estimationin regard of radio system characteristics

20. 20 Instance 1 Experimental results: Quality plan estimationin regard of radio system characteristics

21. 21 Instance 2 Experimental results: Quality plan estimationin regard of radio system characteristics

22. 22 Instance 3 Experimental results: Quality plan estimationin regard of radio system characteristics

23. 23 Conclusion Using Markov decision processes to solve the frequency assignment problem, Improvement of the quality of the solutions and the time complexity

24. 24 Future works Parallelization of our algorithms Markov Decision Processes: Decomposition

25. 25 End

26. 26

27. 27 Instance 3

28. 28 Instance 2