1. Adaptive Routing Proshanto Mukherji CSC 457: Computer Networks University of Rochester

2. Introduction  Networks are not static. They are subject to three classes of change: Topologies change as nodes are added and removed Traffic patterns change cyclically Overall network load changes  So, routing algorithms that assume that the network is static don’t work in this setting

3. Rotten Algorithms 1: DV ABC HostNextCost BB20 CC50 1020 HostNextCost AA20 CC10 “I can reach C with cost 10” 30B

4. Rotten Algorithms 2: LS ABC 50 1020 “I see C (cost 10), and A (cost 20)” “I see B (cost 10), and A (cost 50)” CB30 BB20 HostNextCost B A 50 10 20 C

5. Analyses  Distance Vector: No. of messages: O( |E|.diam ) Size of each message: O( n )  Link State No. of messages: O( n.|E| ) Size of each message: O( e ) Diameter of graph Number of edges Number of hosts Edges per host  Each generates O( n.|E| ) = O( n 3 ) bytes of traffic!  Perhaps ok for single execution, but not every time the load on the network changes  When will you run the algorithm?

6. A New Model  We want instead to think of the routing protocol as always active  Model the network as a dynamical system in the topology and link costs

7. Q-Routing  Q x (d, y) is the time that node x estimates it will take to deliver a packet to node d via its neighbor y  When y receives the packet, it sends back a message (to node x), containing its (i.e. y’s) best estimate of the time remaining to get the packet to d, i.e. t = min(Q y (d, z)) over all z  neighbors( y )  x then updates Q x (d, y) by: [Q x (d, y)] NEW = [Q x (d, y)] OLD + .(s+q+t - [Q x (d, y)] OLD ) new estimateold estimate RTT: x  y Time spent in queue at x

8. Q-Routing (contd.) xy to d min(Q y (d, z i )) = 13; RTT = s = 11 message Q y (d, z 1 ) = 25 DestNext Q x (d, y) … dy20 dw21 … Q y (d, z 2 ) = 17 Q y (d, z e ) = 70 [Q x (d, y)] += (0.25).[(11+17) - 20 ] 22 w estimated RTT = 3 message to d

9. Q-Routing (contd.) xy message DestNext Q x (d, y) … dy22 dw21 … w Use this path next time Q w (d, z 1 ) Q w (d, z 2 ) Q w (d, z e ) to d message to d

10. Ants

11. Ants Online: Routing Tables  Each router x in the network maintains for each destination node d a list of the form:,, …, >, where y 1, y 2, …, y e are the neighbors of x, and p 1 + p 2 + …+ p e = 1  This is a parallel (multi-path) routing scheme  This also multiplies the number of degrees of freedom the system has by a factor of |E| p i = P(msg to d forwarded via y i )

12. Ants Online: Generation Every destination host h d periodically generates an “ant” to a random source host h s An “ant” is a 3-tuple of the form:  cost is a counter of the cost of the path the ant has covered so far

13. Ants Online: Updation When a router x receives an ant from neighbor y i, it: 1. Updates cost by the cost of traversing the link from x to y i (i.e. the cost of the link in reverse) 2. Updates entry for host (,, …, > ) p i = p i +  p  p = k / cost, for some k for j  i, p j = p j 1 +  p normalizing sum of probabilities to 1

14. Ants Online: Propagation (1)  Two sub-species of ant: Regular Ants: P( ant sent to y i ) = p i Uniform Ants: P( ant sent to y i ) = 1 / e  Regular ants use learned tables to route ants  Uniform ants explore randomly

15. Ants Online: Propagation (2) Regular AntsUniform Ants Explore best paths very thoroughly; others hardly at all Explore all paths equally Propagate “bad news” extremely quickly, “good news” extremely slowly Propagate “good” and “bad” news equally fast Tends to find shortest pathsNatural parallel (multi-path) routers Algorithm converges to Q- Routing in a static network Does not converge to Q- Routing

16. Q-Routing vs. Ants  Q-Routing only changes its currently selected route when the cost of that route increases, not when the cost of an alternate route decreases  Q-Routing involves overhead linear in the volume of traffic in the network; ants are effectively free in moderate traffic  Q-Routing cannot route messages by parallel paths; uniform ants can

17. Summary  Routing algorithms that assume a static network don’t work well in real-world networks, which are dynamic  Adaptive routing algorithms avoid these problems, at the cost of a linear increase in the size of the routing tables  Q-Routing is a straightforward application of Q- Learning to the routing problem  Routing with ants is more flexible than Q-Routing

18. Finis that’s all, folks