1. Source-Destination Routing Optimal Strategies Eric Chi EE228a, Fall 2002 Dept. of EECS, U.C. Berkeley

2. Basic Routing Problem Network with links of finite capacity Connection requests for various node-pairs arrive one by one A decision is made to either –deny the request or –admit the connection along a given route An admitted call simultaneously holds some capacity along all links along the route for some amount of time before departing Objective: Make decisions that minimize blocking probability

3. Approaches Suboptimal: Greedy algorithms –Always admit if there is space. –Choose good heuristics for where to place calls. Maximize spare capacity Minimize “Interference” Optimal: Dynamic programming –Balances Immediate gains Long term opportunity costs

4. Markov Decision Process State specified by a Markov Chain –Request arrivals are Poisson –Calls holding times are exponentially distributed Rewards (Costs) associated with –Residing in a state –Making a transition Transition probabilities depend on policies for a given state.

5. Discrete Time MDP

6. Bellman Principle of Optimality Given an optimal control for n steps to go, the last n-1 steps provide optimal control with n-1 steps to go. Example: Dijstkra’s Shortest Path Algorithm

7. Solving MDPs: Value Iteration Solve the fixed point equation. Then

8. Solving MDPs: Policy Iteration

9. Example: Symmetric Y/C X/C ’ Optimal Policy: Route to least loaded

10. Proof (Sketch) Prove that load balancing is optimal for any finite time to go n. (Monotone convergence allows us to take the limit.) Prove inductively that for all n, , a

11. Example: Unbalanced Y/C X/C   2

13. Example: Unbalanced Y/C X/C ’ Optimal Policy: Route to lower link until full. If full route to top link.

14. Comparison

15. Example: Alternate Routing Policy A: Route up 1 st, Route down 2 nd Policy B: Route down 1 st, Route up 2 nd Y/C X/C  2

16. Comparison Two policies

17. Literature K. R. Krishnan and T. J. Ott, "State-dependent routing for telephone traffic: theory and results," in 25th IEEE Control and Decision Conf., Athens, Greece, Dec. 1986, pp. 2124-2128. A. Ephremides, P. Varaiya, and J. Walrand. A simple dynamic routing problem. IEEE Transactions on Automatic Control, 25(4):690-693, August 1980. R.J. Gibbon and F.P. Kelly. Dynamic routing in fully connected networks. IMA journal of Mathematical Control and Information, 7:77--111, 1990. Marbach, P., Mihatsch, M., Tsitsiklis, J.N., "Call admission control and routing in integrated service networks using neuro-dynamic programming," IEEE J. Selected Areas in Comm., v. 18, n. 2, pp. 197--208, Feb. 2000. K. Kar, M. Kodialam, and T.V. Lakshman, “Minimum Interference Routing of Bandwidth Guaranteed Tunnels with Applications to MPLS Traffic Engineering,” IEEE JSAC, 1995, Special Issue on Advances in the Fundamentals of Networking, pp. 1128-36.