The network-on-chip protocol Packetized communication PAYLOAD HEADER TAIL Packet FLIT … What does a switch do when a packet arrives? swA swB The way packets (or packet flits) reserve buffer and link bandwidth is called the “switching strategy”

Packet-based switching Store-and-forward A B After A finishes receiving the entire packet, A asks B if B is ready for the entire packet. (2) B  A, ack (3) A sends the packet to B. Buffering for the largest packet is needed at each switch Extremely high latency Full packet storage latency experienced at each hop Packet not decomposed into flits It is actually broken down into words (phits), but there is no special control for them Used for Time-Division Multiplexing in real-time NoCs

Packet-based switching Virtual-cut-through A B While A receives a part of the packet, A asks B if B is ready for the entire packet. (2) B  A, ack A starts to send the packet to B even when A has not yet received the entire packet. (4) If NACK from B, the entire packet is stored in A Same buffering requirements as for store-and-forward Cuts down on latency Packet propagation is started as soon as possible Packet not decomposed into flits Deadlock freedom easier to enforce

Flit-buffer switching Wormhole switching Pipelining on a flit basis A B flit size < packet size Smaller data space is needed than store-and-forward After A receives a flit of the packet, A asks B if B is ready to receive a flit (2) B  A, ack (3) A sends a flit to B. Reduced buffering requirements Flit-based allocation of buffers and channels Reduced latency wrt store-and-forward May lead to starvation of other packets Packet strung out over the network if head flit blocks

X X The blocking problem Wormhole switching suffers from packet blocking problems Idle Idle Blocked X X Blocked! A B C C cannot send it and has no enough space for a new flit D An idle channel cannot be used because it is owned by a blocked packet…. Although another packet could use it!!!

Virtual-Channels Performance improvement using virtual channels Multiple virtual channels multiplexed on a single physical link Node 1 Node 2 Node 3 Node 4 Node 5 Destination of B Node 1 Node 2 Node 3 Node 4 Node 5 Destination of B A B Block Body flits use the VC acquired by the head flit Tail flit releases VC

Flow control ACK-NACK Flow control ON-OFF Flow control In buffered switching, it determines the way the downstream node communicates buffer availability to the upstream node Block A B C Backpressure Don’t send Buffer full Don’t send Buffer full ACK-NACK Flow control ON-OFF Flow control Credit based Flow control

ACK-NACK Flow control Makes optimistic assumptions for transmission ACK/NACK propagation ACK and buffering Memory deallocation NACK Retransmission Go-back-N Makes optimistic assumptions for transmission In case of long-lasting blockage, waste of power! Long buffer occupancy at the upstream switch Natively supports link-level error control

Credit based flow control No of credits 2 credit Rx Buffer B Receiver gives N credits to sender Sender decrements count Stops sending if zero Receiver sends back credit as it drains its buffer Bundle credits to reduce overhead 1 1 H credit 1 B T H 1 credit B H Round trip time between buffer empty and flit arrival More efficient buffer usage Large amount of upstream signaling (e.g., short packets) Error control pushed at a higher layer

On-Off flow control Incoming flits Outgoing flits Fon threshold GO! Foff threshold STOP! Switch B Potentially reduces the amount of upstream signalling Upstream state: 1 control bit Permission to send (on) or not (off) Upstream signalling only to change this state Buffering requirements to prevent overflow or idling

Simple on/off implementation: stall/go This could be an input or output buffer of the switch Every stage of the pipelined data path should have two slots Not to loose the flit in flight while stall is propagated upstream

Link Pipelining A link pipeline stage is not just a retiming stage but also a flow control stage, hence requires a 2-slot buffer (in general, the number of additional slots is flow control dependent)