1. © Sudhakar Yalamanchili, Georgia Institute of Technology (except as indicated) Deadlock

2. ECE 8813a (2) Reading Assignment Chapter 3

3. ECE 8813a (3) Overview: The Problem Guaranteeing message deliveryGuaranteeing message delivery  Fundamentally an issue of finite resources (buffers) and the manner in which they are allocated oHow are distributed structural hazards handled? Waiting for resources can lead to  Deadlock: configuration of messages that cannot make progress  Livelock: messages that never reach the destination  Starvation: messages that never received requested resources oEven though the requested resource does periodically become available

4. ECE 8813a (4) Ensuring Packet Delivery Deadlock  Prevention  Avoidance  Recovery Livelock  Minimal paths  Restricted non-minimal paths  Probabilistic Starvation  Resource allocation

5. ECE 8813a (5) Focus We are primarily concerned with deadlock  Starvation is addressed via “fair” sharing  Livelock is addressed (at the moment) by constraining messages to minimal paths Relationship between resources, traffic and conditions  Some conditions can produce others oDeadlocked configuration  livelock for other messages

6. ECE 8813a (6) Deadlocked Configuration of Messages Routing in a 2D mesh What can we infer from this figure?  Routing  Dependencies or Wait-for relationships

7. ECE 8813a (7) Some Definitions Routing function – R – delivers a set of channels  Memoryless, coherent, and connected  Note we define the domain as Selection function – picks a channel  Determines performance properties

8. ECE 8813a (8) Router and Network Model Arbitrary topology Fair access to routing and arbitration unit Edge buffers  Theory can be extended to central buffers (we will not) Wait for any channel in a set For wormhole routing, allocated channel must be empty  See Appendix A for detailed list of assumptions Consumption Assumption sisi cici didi Blocked packets wait for first available channel Fair access to channel bandwidth

9. ECE 8813a (9) Concepts and Definitions No assumptions about the paths delivered by R  Expect that minimal paths are always included in R Coherent routing function  If a routing function supplies a path, it also supplies all sub-paths of the path Configuration  Assignment of packets to a set of queues/buffers Connected routing function S D Channels delivered by a routing function

10. ECE 8813a (10) Message Configurations Canonical message configurations  All messages deadlocked Legal, and reachable message configurations  A legal configuration must be possible with the routing function oE.g., non-minimal paths  Since routing functions are memoryless, all legal configurations are reachable

11. ECE 8813a (11) Deadlock A canonical, deadlocked, reachable, legal configuration is one which no message can advance  SAF and VCT oAll packets are in transit oAll adjacent buffers are full  Wormhole oMessages are in transit oHeader flits and data flits cannot advance since all adjacent buffers are full Framework for proving deadlock freedom

12. ECE 8813a (12) Channel Dependencies When a packet holds a channel and requests another channel, there is a direct dependency between them Channel dependency graph D = G(C,E) For deterministic routing: single dependency at each node For adaptive routing: all requested channels produce dependencies, and dependency graph may contain cycles dependency SAF & VCT dependency Wormhole dependency Header flit Data flit

13. ECE 8813a (13) Some Observations Deadlock is not synonymous with the occurrence of cyclic dependencies  Note the goal of guaranteeing message delivery  Are there exits from cycles? The set of dependencies that actually occur (wait-for) are a subset of the statically determined dependencies that can occur as determined by the routing function

14. ECE 8813a (14) Breaking Cycles in Rings/Torii n0n1 n3n2 n0n1 n3n2 c0c0 c1c1 c2c2 c3c3 c 10 c 11 c 02 c 12 c 01 c 03 c 13 c 00 The configuration to the left can deadlock Add (virtual) channels  We can make the channel dependency graph acyclic via routing restrictions (via the routing function) Routing function is c 0i when j i

15. ECE 8813a (15) Breaking Cycles in Rings/Torii (cont.) Channels c 00 and c 13 are unused Routing function breaks cycles in the channel dependency graph n0n1 n3n2 c 10 c 11 c 02 c 12 c 01 c 03 c 00 c 10 c 11 c 12 c 01 c 02 c 03 c 13

16. ECE 8813a (16) Extension to Adaptive Routing n0n1 n3n2 c 10 c 11 c 02 c 12 c 01 c 03 c 00 c 10 c 11 c 12 c 01 c 02 c 03 Note the acyclic subgraph Routing function: is c 0i and c 1i when j >i How can we formalize this? c 00 Additional dependencies due to adaptive routing

17. ECE 8813a (17) Key Idea Informally: The deterministic routing sub-function is used to provide an escape path for an adaptive routing protocol  Packets are guaranteed to escape cyclic dependencies Formally the escape path is defined by a connected routing sub-function defined over a subset of channels  Routing sub-function supplies only escape channels  In practice, this concept only used as a proof mechanism  R 1 (x,y) R(x,y) for all x, y and C 1 is the set of channels supplied by R 1

18. ECE 8813a (18) Theorems Theorem: A connected routing function R for an interconnection network I is deadlock free iff there exists a routing sub-function R 1 that is connected and has no cycles in its extended channel dependency graph Holds for both adaptive and deterministic routing functions  In the case of the latter, R = R1 and the extended channel dependency graph = channel dependency graph  Dally & Seitz 1987 for deterministic routing  Duato 1996 for adaptive routing

19. ECE 8813a (19) Properties Use of escape channels by a message is not unidirectional  If a message enters the escape network it can move back to the adaptive network Do not need to wait on a specific channel  Can wait on any channel including escape channels - the latter guaranteed to show up eventually if allocation is fair Application: start with a deterministic routing function and add channels to be used for adaptive routing Additional restriction required for wormhole (coherent) while this holds for SAF and VCTs

20. ECE 8813a (20) Robustness of the Model No assumptions are made about paths  At least one minimal path is necessary for performance The use of NxN routing functions are practical  What are the alternatives? No assumptions about packet generation rate, packet length and destinations (other than connectivity) For wormhole, only one message can occupy a buffer  Buffers at both ends are single buffer  Thus head flits always block occupying the head of the queue

21. ECE 8813a (21) Revisit Some Properties Canonical legal configuration Which legal configurations are reachable from an empty network for a given routing function? What is the importance of a legal configuration?  Can model a legal configuration as one that occurred by packet injection at the blocked node (VCT/SAF) or the packet with the tail node (wormhole)

22. ECE 8813a (22) Deadlock Avoidance in SAF &VCT Networks Routing sub-function R 1 is c Ai when j i A closer look at escape channel dependencies  c A1 is an escape channel for n0 but not n3  c H2 is an escape channel for n3 Extended channel dependency graph must be acyclic n0n1 n3n2 c H0 c H1 c A2 c H2 c A1 c A3 c A0 c H0 c H1 c H2 c A1 c A2 c A3 direct dependency direct cross dependency escape channel for n0 Direct cross dependency

23. ECE 8813a (23) Example: Constructing Deadlock Free Routing Algorithms R is deadlock free if R1 is deadlock free Making the North-last algorithm fully adaptive  Add an extra adaptive channel in the North Direction  The extra channel has no turn restrictions N1N2 Extra virtual channel in the north direction

24. ECE 8813a (24) Some More Observations Can have one escape network for all packets or multiple escape networks Construct dependency graph on escape channels Deadlock freedom: no cycles in the extended channel dependency graph of R 1  Must consider all direct and indirect dependencies oThere still remain dependencies we have not yet considered!

25. ECE 8813a (25) Indirect Channel Dependencies Direct and indirect dependencies between escape channels Channel c i is an escape channel for some destination Now we can focus all of our attention on the escape channels  Add other channels for adaptive routing  Define the extended channel dependency graph for escape channels oCapture the use of all channels  for adaptive routing request Network cici ckck

26. ECE 8813a (26) Deadlock Avoidance in Wormhole Networks Routing function R is c Ai or c Bi c Hi when j >I Routing sub-function R 1 is c Ai when j i Note the addition of indirect dependencies for Wormhole switching  Cross and direct dependencies between escape channels transmitted through adaptive channels n0n1n2 n5n4n3 c A0 c B0 c H0 c A1 c B1 c H1 indirect dependency

27. ECE 8813a (27) Deadlock Avoidance in Wormhole Networks (cont.) Routing function R is c Ai or c Bi c Hi when j >I Routing sub-function R 1 is c Ai when j i Note the addition of indirect dependencies for Wormhole switching  Cross and direct dependencies between escape channels transmitted through adaptive channels n0n1n2 n5n4n3 c A0 c B0 c H0 indirect cross dependency c A1 c B1 c H1

28. ECE 8813a (28) Deadlock Avoidance in Wormhole Networks (cont.) Routing function: is c Ai and c Hi when j >i Routing sub-function R 1 is c Ai when j i c H0 c H1 c H2 c A1 c A2 c A3 direct dependency direct cross dependency indirect dependency n0n1 n3n2 c H0 c H1 c A2 c H2 c A1 c A3 c A0

29. ECE 8813a (29) Example: Fully Adaptive Routing Each physical channel has a fully adaptive channel a, and DOR channel b Look at the channels supplied by R at node S S D Adaptive channel supplied by R Deterministic channel supplied by R 1

30. ECE 8813a (30) Extensions to Irregular Networks Create an appropriate routing sub-function Example:  Identify a distinguished node and construct a spanning tree  Label each channel as up/down depending on proximity of end points to root (break ties with unique node number)  Routing restriction: a valid path traverses zero or more links in the up direction followed by zero or more links in the down direction How does this avoid cycles? Note the absence of virtual channels! 3 4 26 7 5 1 0

31. ECE 8813a (31) Review The extended channel dependency graph must capture all dependencies  Direct and indirect cross dependencies For wormhole switching routing function must be coherent, otherwise the preceding theorem provides only a sufficient condition Complete the list of dependencies accommodated by the theorem

32. ECE 8813a (32) Extensions: Channel Classes Create equivalence classes of channels  For example, with no intra-class dependencies If the extended class dependency graph is acyclic, routing is deadlock free (sufficient condition) 012 345 678 channel Class X0+X1+X0-X1- Y1+ Y0+ Y0- Y1-

33. ECE 8813a (33) Extensions Extending the domain of the routing function  Including source, channel, or history information Extension to central buffers  Channels are really queues oAssociate routing functions with central queues and use queue dependencies to analyze deadlocks  Domain of the routing function may be or NxN or QxN oExample 2-D mesh: four queues/node, one corresponding to each direction in each dimension.  Queue dependency graph is acyclic  Mixed resource types (edge buffers and central queues) utilizes similar analysis techniques  the extended resource dependency graph

34. ECE 8813a (34) Summarize Solution Philosophies Defined permitted dependencies (prevention)  Correct by construction (acyclic)  Minimal routing freedom Weaken the preceding requirement (avoidance)  Ensure cycles cannot lead to deadlock  Improve routing freedom  Some creative avoidance techniques based on the theory (next) Detect and recover (Next)  Predicated on unlikelihood of occurrence  Maximal routing freedom