Do We Need Wide Flits in Networks-On-Chip? Junghee Lee, Chrysostomos Nicopoulos, Sung Joo Park, Madhavan Swaminathan and Jongman Kim Presented by Junghee Lee

2 Introduction Increasing number of cores  Communication-centric  Packet-based Networks-on-Chip Unit –Packet: a meaningful unit of the upper-layer protocol –Flit: the smallest unit of flow control maintained by NoC If a packet is larger than a flit, a packet is split into multiple flits The flit size usually matches with the physical channel width

3 Motivation What is the optimal flit size in Networks-on-Chip for general purpose computing? 64 or 128 Research papers 256 or 512 Research papers 144 Intel Single- Chip Cloud 160 Tilera 256 Intel Sandy Bridge

4 Multifaceted Factors Flit Size Global Wires Cost of Router WorkloadLatencyThroughput A first attempt in drawing balanced conclusion

5 Assumed NoC Router Architecture d v c p

6 Packet and Flit HeaderPayload

7 Simulation Environment ParameterDefault Value SimulatorSimics + GEMS (Garnet) BenchmarkPARSEC Number of processors64 Operating systemLinux Fedora L1 cache size32 KB L1 cache number of ways4 L1 cache line size64 B L2 cache (shared)16 MB, 16-way, 128-B line MSHR size32 for I- and 32 for D- cache Main memory2 GB SDRAM Cache coherence protocolMOESI directory Topology2D mesh

8 Default NoC Parameters ParameterDefault Value Number of virtual channels3 Buffer depth8 flits per virtual channel Number of pipeline stages4 Number of ports5 Header overhead16 bits

9 Key Questions Can we afford wide flits as technology scales? Is the cost of wide-flit routers justifiable? How much do wide flits contribute to overall performance? Do memory-intensive workloads need wide flits? Do we need wider flits as the number of processing elements increases?

10 #1) Global Wires Can we afford wide flits as technology scales? Technology scaling does not allow for a direct widening of the flits because the power portion of the global wires increases as technology scales ItemUnitValue Technologynm Chip size*mm Transistors*MTRs Global wiring pitch*nm Power index*W/GHz cm Total chip power*W Normalized power portion * International Technology Roadmap for Semiconductors (ITRS) 2009 and 2011

11 #2) Cost of Router Is the cost of wide-flit routers justifiable? Cost of buffers  Flit size  Buffer depth  Number of virtual channels Cost of switch  (Flit size) 2  (Number of ports) 2 Flit size Cost Switch Buffer Flit size  2  cost of router  2.97 Flit size  4  cost of router  Flit size  2  cost of router  2.97 Flit size  4  cost of router  If the performance improvement does not compensate for the increase in the cost, widening of the flit size is hard to justify

12 #3) Latency How much do wide flits contribute to overall performance? The network traffic usually consists of packets of different sizes –l s : The size of shortest packet –l l : The size of longest packet Flit size Latencyl s +hl l +h Suggested rule of thumb: Flit size = shortest packet size + header overhead Suggested rule of thumb: Flit size = shortest packet size + header overhead

13 #4) Workload Characteristics ApplicationCache misses / Kcycle / node Injected packets / Kcycle / node Blackscholes Bodytrack Ferret Fluidanimate Freqmine Streamcluster Swaptions Vips X Do memory-intensive workloads need wide flits? The injection rate of real applications is far less than the typical saturation point of NoC  Self-throttling effect [34] The injection rate of real applications is far less than the typical saturation point of NoC  Self-throttling effect [34] Up to 64 cores, we can keep the rule of thumb because of the low injection rate

14 #5) Throughput Widening the flit is not a cost-effective way because of fragmentation If widening the physical channel is the only option for increasing the throughput, we suggest using physically separated networks Do we need wider flits as the number of processing elements increases? Flit size Latency One 80-bit network One 160-bit network Two 80-bit networks

15 Conclusions Can we afford wide flits as technology scales? Is the cost of wide-flit routers justifiable? How much do wide flits contribute to overall performance? Do memory-intensive workloads need wide flits? Do we need wider flits as the number of processing elements increases? No, unless the power budget for NoC increases No, the cost increases sharply with the flit size Until the flit size reaches the shortest packet size No, because of self-throttling effect No, because of fragmentation

16 Final Conclusion Suggested rule of thumb: Flit size = shortest packet size + header overhead This paper provides a comprehensive discussion on all key aspects pertaining to the NoC’s flit size This exploration could serve as a quick reference for the designers/architects of general-purpose multi-core microprocessors who need to decide on an appropriate flit size for their design.

17 Thank you!

18 Questions? Contact info Junghee Lee Electrical and Computer Engineering Georgia Institute of Technology