© Sudhakar Yalamanchili, Georgia Institute of Technology (except as indicated) Routing Algorithms

ECE 8813a (2) Overview Routing Algorithm fixed by  Routing function  Selection function Determinant of key properties  Connectivity  Deadlock and routing freedom  Fault tolerance  adaptivity A single function for deterministic protocols

ECE 8813a (3) Taxonomy Routing Algorithms Unicast multicast CentralizedSourceDistributedMulti-phase Table LookupFinite State Machine DeterministicAdaptive ProgressiveBacktracking ProfitableMisrouting CompletePartial # Destinations Routing Decisions Implementation Adaptivity Progressiveness Minimality Number of Paths

ECE 8813a (4) Generic Router Architecture

ECE 8813a (5) Elaboration Source Routing  Static paths defined at the source  Extensions to street-sign routing Multiphase routing  Used for reliable routing  Load balancing Irregular networks typically need table look-up or source routing Interval routing  Making table-based routing more efficient

ECE 8813a (6) Interval Routing Each output link corresponds to an interval of nodes  Union of intervals at a node is the set of all destination nodes  Must be able to distinguish invalid intervals (at the edges of a mesh) Use overlapping intervals for fault tolerance ChannelNodeInterval X X-60-3 Y+64-5 Y-67

ECE 8813a (7) Some Finer Points Oblivious vs. deterministic protocols  Typically referring to use of network state  Deterministic protocols are oblivious oConverse may not be true Fully, maximal, and true fully adaptive  Fully: maximize alternative physical paths  Maximal: maximize all routing options  True: no constraints on VC usage oDeadlock recovery-based methods

ECE 8813a (8) Classes of Routing Algorithms Deterministic Routing Partially Adaptive Fully Adaptive Maximally Adaptive Non-minimal Routing Routing in MINs

ECE 8813a (9) Deterministic Routing Algorithms Strictly increasing or decreasing order of dimension Acyclic channel dependencies Mesh Binary Hypercube Deterministic routing

ECE 8813a (10) Tori Dimension order routing Create an acyclic channel dependency graph with the following routing function c 0i when j i n0n1 n3n2 c0c0 c1c1 c2c2 c3c3 c 13 n0n1 n3n2 c 10 c 11 c 02 c 12 c 01 c 03 c 00 c 10 c 11 c 12 c 01 c 02 c 03 Deterministic routing

ECE 8813a (11) Deterministic Routing Algorithms: Implementation Issues Relatively, the most inexpensive to implement  Absolute addressing  Relative Addressing Header update  Necessary to maintain uniformity of implementation Source routing  Street sign addressing: only encode the turns Deterministic routing

ECE 8813a (12) Partially Adaptive Routing Algorithms Trade-off between hardware resources and adaptivity  Maximize adaptivity for given resources  Minimize resources required for a given level of adaptivity Typically exploit regular topologies Partially Adaptive Routing

ECE 8813a (13) Planar Adaptive Routing Packets are routed adaptively in a series of two dimensional planes  Order of planes (dimensions) is arbitrary Routing in two dimension uses two virtual networks  Increasing and decreasing networks Fully adaptive Planar adaptive A0 A1 A0 A1 A2 Partially Adaptive Routing

ECE 8813a (14) Adaptive Routing in Two Dimensions Increasing Network Decreasing Network D i+1 DiDi Partially Adaptive Routing

ECE 8813a (15) PAR in Multidimensional Networks Routing is fully adaptive in a plane When can you skip a plane? Dimension i Dimension i+1 Dimension i+2 A1 A0 Partially Adaptive Routing

ECE 8813a (16) PAR Properties Each plane is comprised of the following channels A i = d i,2 + d i+1,0 + d i+1,1 Three virtual channels/link in meshes and six virtual channels/link in Tori Partially Adaptive Routing

ECE 8813a (17) The Turn Model What is a turn?  From one dimension to another : 90 degree turn  To another virtual channel in the same direction: 0 degree turn  To the reverse direction: 180 degree turn Turns combine to form cycles Goal: prohibit the least number of turns to break all possible cycles abstract cyclesXY routingwest-first routing Partially Adaptive Routing

ECE 8813a (18) Turn Constraints Choice of prohibited turns is not arbitrary Alternative designs  Three combinations unique (within symmetry) oThree algorithms: west-first, north-last, negative-first equivalent Partially Adaptive Routing

ECE 8813a (19) West First Routing Fully adaptive to the east, deterministic to the west (for non-minimal routing) Non-minimal is partially adaptive to the west deterministic region fully adaptive region Partially Adaptive Routing

ECE 8813a (20) Generalization of the Turn Model Identify channel classes and prohibit turns between them Cycles are infeasible within a channel class Transitions between channel classes are acyclic Partially adaptive: number of shortest paths are reduced Application to Binary hypercubes Base is e-cube routing Identify up channels and down channels Adaptively route though one class and then the other Turns prohibited from one class to the other Partially Adaptive Routing

ECE 8813a (21) Fully Adaptive Routing Using resources in the form of buffers, channels and networks Migration from initial proposals to more efficient implementations Fully Adaptive Routing

ECE 8813a (22) Structured Buffer Pools Positive hop algorithm  D+1 buffers at each node (D = diameter)  Nodes request/use buffers in strictly increasing order  Minimal path algorithm, valid for any topology  Large buffer requirements: O(Diameter) buffer 0 buffer 1 buffer D-1 Fully Adaptive Routing

ECE 8813a (23) Structured Buffer Pools: Extension subset S-1subset 1subset 0 Negative hop algorithm  Partition nodes into non-adjacent subsets  Order subsets  Down transitions request higher numbered buffer, else same numbered buffer  Number of buffers required in each node is given by + 1 set 0 set 1 Fully Adaptive Routing

ECE 8813a (24) Extensions to Wormhole Switching replace central buffers with equivalent number of virtual channels across each physical channel Acyclic central buffer dependencies are transformed into acyclic virtual channel dependencies Basic version produces unbalanced used of virtual channels  distance rarely equal to diameter Extension: bonus cards.  Number is equal to unused hops: diameter - #req  Use this number to increase the number of choices of virtual channels at any node  Not the same as adaptivity Fully Adaptive Routing

ECE 8813a (25) Virtual Networks Establish multiple virtual networks  Each network works for a specific destination set  Routing functions are acyclic, but typically not connected Establish constraints between virtual networks Fully Adaptive Routing

ECE 8813a (26) Virtual Networks in a Mesh Y X X+Y+X-Y+ Each virtual network is constructed to have acyclic channel dependencies  Routing function in a virtual network is not connected Packets are injected into the appropriate virtual network Fully adaptive, no transitions between networks 2 n virtual networks with (n.2 n ) virtual channels/node Fully Adaptive Routing

ECE 8813a (27) Optimize Reduce the number of networks by 50% with one additional channel/network Extensions to k-ary n-cubes by introducing levels in each virtual network  One level for each wrap-around channel  (n+1).(n+1).2 (n-1) virtual channels/node for dimensions >0  See Figure Virtual network 0Virtual network 1 Fully Adaptive Routing

ECE 8813a (28) Extensions to Tori Addition of layers (virtual networks) for each wrap- around connection Number of layers increases by #dimensions Fully Adaptive Routing

ECE 8813a (29) Deadlock Avoidance via Message Dependencies Two virtual channel classes across each physical link  Adaptive channels & deterministic channels Fully adaptive use of adaptive channels  Keep track of #dimension reversals for each message oMoving from a dimension p to a lower dimension q  Label each channel with the DR# of the message  Messages cannot block on a channel with lower DR# oIf no channel available, permanent transition to the deterministic channel  Dependencies between messages are acyclic Fully Adaptive Routing

ECE 8813a (30) Summary of Design Techniques Ordered use of topological features  Dimensions  Packaged components  Paths Ordered use of resources  Buffers  Channels Order the message population  Each message is uniquely identified by some attribute, e.g., number of wrap-around channel crossings  Order blocking based on message population membership

ECE 8813a (31) Design Methodology Start with a network, set of channels C 1, and routing function R 1  R 1 is connected and deadlock free and may be deterministic/adaptive, minimal/non-minimal Split each physical channel into a set of additional virtual channels and define the new routing function  Set of channels includes escape channels and adaptive channels  Selection function can be defined in many ways For wormhole switching, verify that the extended channel dependency graph is acyclic: likely if R is restricted to minimal paths

ECE 8813a (32) Example Binary Hypercube Start with dimension order e- cube algorithm Add additional channels for adaptive routing

ECE 8813a (33) Maximally Adaptive Routing Establish a relationship between routing freedom and resources  Maximize adaptivity for fix resources  Minimize resources for target adaptivity Relationship between adaptivity and performance  Not obvious  Unbalanced use of physical or virtual channel resources Maximally Adaptive Routing

ECE 8813a (34) In 2D Meshes: Double Y Maximally Adaptive Routing increasing networkdecreasing network X+ X-Y1- Y1+ Y1- X- X+ Y2+ Y2- X-X+ X- Y2+ X+Y2- Y1Y2 6 Y1

ECE 8813a (35) In 2D Meshes: Mad-Y X+ X-Y1- Y1+ Y1- X- X+ Y2+ Y2- X-X+ X- Y2+ X+Y2- Permit turns  From the Y1 channels to the X+ channels  From X- channels to Y2 channels Remove unnecessary turn restrictions Still overly restrictive! permitted 6 Y2 Y1 No coupling between Y2 and Y1 Maximally Adaptive Routing

ECE 8813a (36) In 2D Meshes: Opt Y X+ X-Y1- Y1+ Y1- X- X+ Y2+ Y2- X-X+ X- Y2+ X+Y2- Y2+Y1+ Y2+ Y2-Y1- Y2- Further reduce the number of restrictions  Only restrict turns from Y1 to X-  Turns from X- to Y1 and 0-degree turns in Y only when X offset is 0 or positive Extensions to multidimensional meshes Basic idea: fully adaptive routing in one set of channels, and dimension order in the other set until specified lower dimension traversals are complete Maximally Adaptive Routing

ECE 8813a (37) Routing with Minimum Buffer Requirements Key Idea:  Organize packet traffic into disjoint groups that use separate buffers in each node  Place acyclic routing restrictions in buffer usage Based on node orderings Maximally Adaptive Routing

ECE 8813a (38) Node Labeling for 2D Torus right increasing node ordering left increasing node ordering outside increasing node orderinginside increasing node ordering Maximally Adaptive Routing

ECE 8813a (39) Algorithm 1 Algorithm  Packet moves from the injection queue to the A queue can  Stay in the A queues as long as we can move to right along at least one dimension along a minimal path  Transition to the B queues under same rule for left traversals  Transition to the C queue and remain there until packet is delivered Note the de-coupling of node labeling from buffer labeling Maximally Adaptive Routing

ECE 8813a (40) Application Orderings analogous to virtual planes  Note the orderings are acyclic Extensions to edge buffers  Check Algorithm 2 Maximally Adaptive Routing

ECE 8813a (41) True Fully Adaptive Routing Adaptivity extends across physical and virtual channels Deadlock recovery vs. deadlock avoidance Maximally Adaptive Routing

ECE 8813a (42) Non-minimal Routing Non-minimal routing  Wormhole degrades performance while VCT has less secondary effects  Fault tolerance is the main motivator Classes  Backtracking  Randomized routing and the Chaos router Non-Minimal Routing

ECE 8813a (43) Backtracking Protocols Backtracking search + resource reservation Constrain the search  Minimal paths vs. #misroutes Non-Minimal Routing

ECE 8813a (44) Optimization Sensitive to choice of switching technique  Naturally suited to circuit switching and pipelined circuit switching  Overhead is large with SAF Deadlock is avoided by not blocking on busy channels Livelock is avoided by maintaining and using search history  In the header: large headers  In the routers: local state, headers comparable to e-cube Protocol variations  Multi-links  k-family  exhaustive: profitable and misrouting  limited misrouting  multi-phase Non-Minimal Routing

ECE 8813a (45) Routing in MINs “Square” vs. non-square MINs vs. dilated MINs Permutation routing and centralized control Message routing, distributed control, and contention Basic elements of a log k N network and topological equivalence  Single source destination path  Logarithmic delay Routing in MINs

ECE 8813a (46) The Blocking Condition s n-1,s n-2,…,s 2,s 1,s 0,d n-1,d n-2,…,d 2,d 1,d 0 r n-1,r n-2,…,r 2,r 1,r 0,t n-1,t n-2,…,t 2,t 1,t 0 Addresses of ports at intermediate switches are computed from the source-destination addresses  Blocking condition: two paths collide iff they compete for the same link at any stage Routing in MINs

ECE 8813a (47) Self Routed MINs Routing in MINs

ECE 8813a (48) Self Routed MINs (cont.) Self routing property only  Routing decision a function of only the destination Computation of destination routing tag t n-1,t n-2,…,t 2,t 1,t 0  t i = d i+1 0 <= i <= n-2, and t n-2 = d 0 for butterfly MINs  t i = d n-i-1 0 <= i <= n-1 for Omega and Cube networks Routing in MINs

ECE 8813a (49) Self Routed MINs: example Routing in MINs

ECE 8813a (50) Routing in Bidirectional MINs Routing uses the function FirstDifference(S,D)  Identifies the nearest stage to “turnaround” Multiple choices of switches at last stage  Can randomize forward path selection Routing in MINs

ECE 8813a (51) Relevance of Fat Trees Popular interconnect for commodity supercomputing Active research problems  Efficient, low latency routing  Fault tolerant routing  Scalable protocols (coherence) Routing in MINs

ECE 8813a (52) Segment Based Routing Topology dependent vs. topology agnostic routing  Reliability  Increasing important on-chip Restriction-based approach  Multiple restriction options oSelect restrictions based on performance goals  Source based routing oRouting table generation Topology Agnostic Routing

ECE 8813a (53) Key Idea: Segments & Subnets Partition topology into subnets and then segments in a subnet Goal: islands of regularity Topology Agnostic Routing

ECE 8813a (54) Requirements Avoid deadlock in a segment Avoid deadlock when traversing multiple segments Ensure routing connectivity when physical connectivity exists Avoiding congestion in path construction Topology Agnostic Routing

ECE 8813a (55) Construction of Segments Search for initial segment + “regular” segments  Unitary segments Add one bidirectional restriction in each segment Source routing Starting node Terminal node Unitary segment bridge segment Topology Agnostic Routing Failed links

ECE 8813a (56) Deadlock Freedom One routing restriction per segment  No cycles in a segment Every cycle contains a segment  Hence cannot be “closed” to create deadlock No cycle from the start node back to itself  Cannot create cycles across subnets Think of a subnet as a union of 1-D segments Topology Agnostic Routing

ECE 8813a (57) Performance From A. Mejia, J. Flich, J. Duato, Sven-Arne Reinomo and Tor Skeie, “Segment Based Routing: An Efficient Fault-Tolerant Routing Algorithm for Meshes and Tori,” Proceedings of the International Parallel and Distributed Processing Symposium, April Topology Agnostic Routing

ECE 8813a (58) Region-Based Routing Problem: reduce the size of routing tables for on-chip routers Recognize that finite state machine routers implicitly check for region membership  Think meshes Generalize the idea of regions  Can naturally be adapted for fault tolerant routing Topology Agnostic Routing

ECE 8813a (59) Example of Regions What are the characteristics of these regions?  Note #regions = f(routing options) { node set } From J. Flich, A. Mejia, P. Lopez, and J. Duato, “Region-Based Routing: An Efficient Routing Mechanism to Tackle Unreliable Hardware in Networks on Chip,” Proceedings of the First International Symposium on Networks on Chip, May 2007 Topology Agnostic Routing

ECE 8813a (60) Approach Observe that table-based routing is really region based  Each entry identifies a region Merge entries into compact region specifications at each switch Region construction is based on the paths  Any set of paths  fault tolerant routing  No virtual channels Topology Agnostic Routing

ECE 8813a (61) Region Construction Start with routing restrictions Compute deadlock free-paths Compute regions Coalesce regions (pack if necessary to cap #regions) Construct routing table From J. Flich, A. Mejia, P. Lopez, and J. Duato, “Region-Based Routing: An Efficient Routing Mechanism to Tackle Unreliable Hardware in Networks on Chip,” Proceedings of the First International Symposium on Networks on Chip, May 2007 Topology Agnostic node

ECE 8813a (62) Creating Regions Coalesce routing options based on inputs and outputs Represents a compact routing table From J. Flich, A. Mejia, P. Lopez, and J. Duato, “Region-Based Routing: An Efficient Routing Mechanism to Tackle Unreliable Hardware in Networks on Chip,” Proceedings of the First International Symposium on Networks on Chip, May 2007 Topology Agnostic Routing

ECE 8813a (63) Implementation Initialization of region registers and parallel evaluatiom of all regions From J. Flich, A. Mejia, P. Lopez, and J. Duato, “Region-Based Routing: An Efficient Routing Mechanism to Tackle Unreliable Hardware in Networks on Chip,” Proceedings of the First International Symposium on Networks on Chip, May 2007 Topology Agnostic Routing

ECE 8813a (64) Hardware Overheads Each region requires  Four registers that define the region  Mask registers to define input and output ports  Logic to determine routing options Hardware cost grows as the number of regions  Growth as f(network_size) is much slower From J. Flich, A. Mejia, P. Lopez, and J. Duato, “Region-Based Routing: An Efficient Routing Mechanism to Tackle Unreliable Hardware in Networks on Chip,” Proceedings of the First International Symposium on Networks on Chip, May 2007 Topology Agnostic Routing

ECE 8813a (65) Microarchitecture Issues Routing algorithm performance is sensitive to resource allocation schemes in the router Key resource management functions include  Routing function  Selection function  Arbitration/scheduling Mismatch can lead to poor performance

ECE 8813a (66) Resource Allocation: Selection Functions Selection function may be oblivious or informed  Common to favor minimal paths and lightly loaded links Examples:  Meshes: minimum congestion, maximum flexibility, straight lines Unlike routing functions, selection function must be serialized  Result updates the channel status – a centralized resource VC status/control VC buffer Selection function Routing function Input VC Output VCs

ECE 8813a (67) Selection Functions Favor adaptive channels  Improve probability of escape channel availability  Time dependent selection functions: give adaptivity a chance Selection functions for real time traffic  Separate best effort and guaranteed packets via VCs or virtual networks Note the impact on bisection utilization Selection functions for cache coherent systems?

ECE 8813a (68) Resource Allocation: Arbitration Tradeoffs: channel bandwidth vs. message sizes and types  Mix of buffering strategies across message types All three strategies must be co-designed for a tuned system arbitration Flow controlMessage size

ECE 8813a (69) Routing, Selection & Arbitration Input driven vs. output driven scheduling  Output driven scheduling requires replication of routers amongst inputs Lessons from the microprocessor world  Impact of complexity, workloads, and concurrency Impact of….  Symmetry of the topology  Locality of traffic  Packet size Locality, uniformity Irregular, hot spot deterministic routing adaptive routing

ECE 8813a (70) Summary Best routing algorithm driven by multiple considerations Deterministic vs. adaptive Uniform vs. non-uniform traffic Packet sizes Power envelope On-chip vs. off-chip Locality of traffic Hot spots Compatible micro-architecture Symmetric vs. asymmetric topology