Guangdeng Liao, Xia Zhu, Steen Larsen, Laxmi Bhuyan, Ram Huggahalli University of California, Riverside Intel Labs

 Motivation  Network Processing  Experiment Setup  Power Studies ◦ Intel Nehalem Server ◦ Niagara 2 Server  System Architecture Implications

 Network speed increases at a rate of 10X and rapidly transits to 10Gbps and beyond.  Existing studies on network processing focused on performance.  Power becomes increasingly important due to environmental and economic concerns.  Detailed power studies of network processing over high speed networks guide us to design a more power-efficient platform.

 Unlike traditional CPU/Memory-intensive apps, network processing involves various platform components.

 Focused on mainstream servers: Intel Nehalem servers interconnected with 10GbE.  Used Iperf to generate network traffic.  Used Data Acquisition System (DAQ) to measure power consumption on individual hardware components.  Studied power benefits of integrating 10GbE NICs into CPUs by using a Niagara 2 server.

 Sensing resistors are added to +12V, +5V, +3.3V power supply, as well as each DIMM and the 10GbE PCI-E NIC card. AC/DC power supply 120V AC CPU power and regulation Memory DIMM power PCIe NIC power Current + Voltage sensing data acquisition

 NIC Idling Power: ~9 Watts.  ~25 Watts and ~17 Watts are dissipated for small and large I/O sizes, respectively. CPU is the major power consumer, followed by memory.

 Small packets have very low power-efficiency due to inefficiency of processing small packets on current systems. CPU utilization breakdown shows the overhead is mainly from OS kernel.

 ~22 Watts and ~18 Watts are dissipated by small and large I/O sizes. CPU is the major consumer, followed by memory.

 Small packets have very low power-efficiency. Unlike the receive side, CPU utilization breakdown shows the overhead is mainly from SoftIRQ.

 Sun Niagara 2 integrates two 10GbE NICs into CPU die.  Conducted experiments on an integrated NIC (INIC) and a discrete NIC (DNIC), and compared power efficiency of network processing.

 DNIC has ~20 watts idling power with dual port, INIC has ~17 watts with dual port. Saving is from PCI-E interface.  INIC has better power efficiency than DNIC, mainly due to less CPU cycles consumed for processing received packets.

 INIC has similar power efficiency to DNIC.

 High speed NICs have high idling power and are not energy proportional at all.  Small packets have low power-efficiency. Optimizations lie in OS kernel such as buffer management, context switches etc.  CPU is the major contributor of power consumption of network processing, followed by memory.  Integrating NICs has a little bit better power efficiency.

 Motivation  Network Processing  Experiment Setup  Power Studies ◦ Intel Nehalem Server ◦ Niagara 2 Server  System Architecture Implications

 Reducing CPU power cost ◦ Small in-order cores have better power efficiency for network processing. ◦ Heterogeneous CPUs incorporating small cores for network processing. core Core Cache/Interconnect core…. atom NHM

 Reducing memory power cost ◦ In the receive side, there are two memory access for each packet. ◦ Solution: packets are delivered to caches and a new instruction is added to invalidate packets in caches.

 Reducing memory power cost ◦ In the transmit side, there are packet write-backs. ◦ Solution: transmitted packets are fed from caches and invalidated after data transfer.

 Reducing NIC power cost (Idling power) ◦ Integrate NICs into CPU to avoid PCI-E power consumption. ◦ Apply rate-adaption scheme into NICs to save power with low traffic rates.

Q & A