1. EE382C Lecture 6 Adaptive Routing 4/14/11 What is tornado traffic?
Adaptive routing algorithms aim to solve global load balancing. If you narrow it down locally, it does worse.
O1turn optimal oblivious algorithm for mesh networks.
What is RLB?
EE 382C - S11 - Lecture 6

2. Question of the day
What is the worst-case traffic pattern for CQR routing on a torus?
Like we did for the other routing algorithms in the previous lecture.
EE 382C - S11 - Lecture 6

3. Review of Routing Basics
Deterministic, Oblivious, or Adaptive Routing
Deterministic – simple, maintains ordering, only choice if no path diversity
Oblivious
VAL – achieves optimum throughput on worst-case traffic
But destroys locality, same throughput on all traffic
Linearity lets us find worst-case traffic pattern for arbitrary oblivious algorithm
Bipartite maximal-weight matching problem
VAL: its properties are because it makes every pattern into uniform random.
Bipartite graph: sources on left, destinations on right. Each edge has a weight equal to the load that it will induce on the channel we are trying to congest. The goal is to find the maximal-weight matching.
EE 382C - S11 - Lecture 6

4. Report Card from Last Time
Algorithm
Locality
Uniform traffic
Tornado traffic
Greedy A
1 (A)
0.33 (F)
Random F
0.5 (F)
0.4 (C)
RLB C
0.76 (C)
0.53 (A)
VAL
0.5 (A-)
Valiant: same performance on both patterns.
EE 382C - S11 - Lecture 6

5. Topology & Routing
Introduce adaptivity. Google maps -> adaptive routing.
EE 382C - S11 - Lecture 6

6. Adaptive routing
Path from source A to destination B based on network conditions
Advantages
Can provide low latency at low loads and good load balancing at high loads
Disadvantages
Making good routing decisions with approximation information can be difficult
Packet reordering
EE 382C - S11 - Lecture 6

7. Minimal adaptive routing
Two phases
Route “up” adaptively towards a random top level node
Route “down” deterministically to destination
“Turn around” at first common ancestor
Keeps route minimal
Green and red are the two alternate routes packets can take.
This is good at locally balancing load, but poor at globally balancing load.
Minimal adaptive performs just as well with O1turn.
EE 382C - S11 - Lecture 6

8. How is minimal adaptive on a Clos different than minimal Oblivious?
In terms of results: performance of the network, latency characteristics, etc.
EE 382C - S11 - Lecture 6

9. Transient Imbalance
This is load per time.
EE 382C - S11 - Lecture 6

10. With Adaptive Routing
This is load per time.
EE 382C - S11 - Lecture 6

11. Minimal adaptive routing
Minimal Adaptive (MIN AD) [Gravano ’93]
Always route in minimal quadrant
Adaptively routing in each hop
Makes bad global decisions.
Illustrates the lack of global knowledge and path diversity.
It may choose a channel which is slightly less congested but would put it in a congested region. Without the minimal restriction, it could get out. With global knowledge, it would not make that choice.
Optimal Qben; Poor Qadv
EE 382C - S11 - Lecture 6

12. Minimal adaptive drawback
Left: congestion is avoided by minimal adaptive routing.
Right: Congestion cannot be avoided.
EE 382C - S11 - Lecture 6

13. Fully adaptive routing
Potential livelock.
Any issues with misrouting?
EE 382C - S11 - Lecture 6

14. Livelock issues
A packet is always misrouted from 02 -> 12 and 12 -> 11.
EE 382C - S11 - Lecture 6

15. Global Adaptive Routing
Global - Make overall routing decision (e.g., which quadrant to traverse in a torus).
Adaptive - Based on network state
Need to estimate network state at packet source.
Need a method to partition routes - so we can choose between them based on network state.
EE 382C - S11 - Lecture 6

16. Reconsider an 8-node Ring
Congested clockwise channels
Receiver
Sender
Sender cannot sense congestion locally
Two partitions of the channels. Clockwise and counter-clockwise.
Show why global state is important: adjacent channels may be free, but distant channels in one direction may be congested.
Two partitions: clockwise and counter-clockwise
EE 382C - S11 - Lecture 6

17. Channel Queue Routing CQR
1. Check queue lengths for outgoing channels at source
2. Compute quadrant congestion for each quadrant - q1, q2, q3, q4
3. There are 4 quadrants with hop counts H1, H2, H3, H4
4. Choose quadrant i corresponding to the smallest Hiqi
This algorithm still perform bad under localized traffic.
Routes minimally adaptive within each quadrant.
The goal of this is to equalize latency (hops times congestion is an estimate for latency).
5. Route minimally adaptively within chosen quadrant
EE 382C - S11 - Lecture 6

18. Switches to non-minimal at 0.12
CQR: TOR Throughput
Shows how many packets go minimally and how many minimally.
Switches to non-minimal at 0.12
EE 382C - S11 - Lecture 6

19. CQR: TOR Latency
Shows that the latency of minimal and non-minimal packets is roughly the same.
CQR tries to balance latency both ways which it does.
EE 382C - S11 - Lecture 6

20. Report card Throughput (frac of capacity) Latency (Cycles) Algo
Θ benign UR traffic
Θ adv traffic
Θ avg
Tlow (0.2 UR)
Tint (0.4 TOR)
VAL
0.5 (F)
0.5 (A)
0.5 (C)
10.5 (F)
20.5 (B)
DOR
1.0 (A)
0.25 (F)
0.31 (F)
4.5 (A)
 (F)
RLB
0.76 (C)
0.31 (C)
0.51 (C)
6.5 (C)
5.5 (A)
MIN AD
1.0 (A)
0.33 (C)
0.63 (B)
4.5 (A)
 (F)
GOAL
0.76 (C)
0.5 (A)
0.68 (A)
5.5 (C)
5.5 (A)
Min ad does a little better than DOR, but the same practically as O1turn (not shown).
GOAL: obliviously chooses a quadrant (weighing the short distance more heavily), and then does minimal adaptive within the quadrant.
It’s like MIN AD except it doesn’t care for the queue lengths when choosing a quadrant.
GOAL: globally oblivious adaptive locally.
CQR has the best properties of the other routing algorithms.
CQR
1.0 (A)
0.5 (A)
0.73 (A)
4.5 (A)
5.5 (A)
EE 382C - S11 - Lecture 6

21. Routing Mechanics
EE 382C - S11 - Lecture 6

22. Routing Mechanics
The mechanism used to implement any routing algorithm: deterministic, oblivious, or adaptive
Table-based routing
Source routing
Node table routing
Algorithmic routing
EE 382C - S11 - Lecture 6

23. Source table routing Advantages: Very low routing delay
Routers can be used in any topology.
Disadvantages:
Can’t be used for adaptive routing.
Packet overhead
Any others?
Each source has a table. It appends the route to the packet according to the destination.
Routers can be used in any topology because they simply do what they see in the packet. Only the sources care about the topology
EE 382C - S11 - Lecture 6

24. Building routing tables
There may be several routes for each destination
These may be selected using a pseudo random generator as a selector
For ordering, hash using a flow identifier
Having multiple routes help in
Fault tolerance
If routes are edge disjoint
Load balance
Distribute load over several network paths
Distribution
When destination is a logical destination to a “service” rather than a physical destination node. In this case, the multiple routes may in fact lead to multiple destinations
EE 382C - S11 - Lecture 6

25. Encoding variable length header
Variable length header needs special flags to indicate what is missing and so on.
P is simply a placeholder (phit continuation).
X: destination reached.
Each line is a phit. Each rectangle is a flit. Read from right to left
EE 382C - S11 - Lecture 6

26. Impl variable length src table
Table now has pointers to memory where the path can be found (left to right in this case)
X terminates a route.
Avoids the storage waste associated with rounding up all routes to a maximum length
EE 382C - S11 - Lecture 6

27. Node table routing - Livelock avoidance (00 to 11 takes S (10) then N)
- Latency at each hop
+ Local adaptivity (bolded letters are misroutes)
+ Space saving
Stores the table in the routers instead of the sources.
Livelock: Have to guarantee that routers won’t put packets into an infinite loop.
Source table routing guarantees that because entries are not of infinite size.
How can you do minimal adaptive or oblivious routing in this case?
Is more appropriate for adaptive routing.
Significantly reduces space required because each router only needs to hold which is the next hop.
Cannot give packets to same destination from different sources different routes.
Scalability also suffers.
EE 382C - S11 - Lecture 6

28. Hierarchical Node Table
Each nearby node gets its own next hop entry.
Small groups of remote nodes share entries.
Larger groups of distant nodes share entries.
Etc.
The more Xs the larger the group.
Can be implemented with a CAM.
Table for node --- of an 8 ary 3 cube
EE 382C - S11 - Lecture 6

29. Algorithmic routing in a torus?
Choose first productive dimension from left to right
Randomly select a productive dimension
Select a productive dimension based on queue lengths
Allow selection of unproductive dimensions if productive dimension queue lengths are above threshold
CHOICES: Min OB, Min Ad, Full Ad, DOR
Same output port for the same packet, just with combinational logic instead.
Route selection: determines the type of routing performed by this circuit.
Match the choices to the description text.
Algorithm is specific to routing and topology.
Sx: sign bit for X dimension.
The circuit checks for which dimension the packet is at the right coordinate.
It produces where it must go (for example increase X and decrease Y). “Exit” is 1 if it has arrived and must be ejected.
EE 382C - S11 - Lecture 6

30. Question of the day
What is the worst-case traffic pattern for CQR routing on a torus?
Can’t be answered. The problem is too hard for adaptive routing.
If you try to load a channel as the method we saw last week, the adaptive routing algorithm would route packets another way.
EE 382C - S11 - Lecture 6

31. Summary Adaptive routing Routing mechanics Use network state
Is non-linear
Minimal adaptive - make local decisions
Non-minimal
Global adaptive (example: CQR)
Sensing global state
Partitioning routes
Routing mechanics
Tables
Source, Node, Associative
Algorithmic
Associate are the routing tables to reduce space for short routes.
Oblivious routing gives linear channel load functions. Adaptive does not. Therefore worst-case traffic pattern is hard to calculate.
Non-linear: The more you load, the resulting load won’t be the sum.
EE 382C - S11 - Lecture 6