CAECW Salt Lake City -- Veazey & Gaither Varying Memory Size with TPC-C Performance and Resource Effects Jay Veazey and Blaine Gaither Hewlett-Packard

CAECW Salt Lake City -- Veazey & Gaither Motivation --- why is this interesting? More memory increases performance → How much? →Why exactly? → Reveal and quantify the underlying causes Focus is R&D tradeoffs →Performance, cost, schedule, power →How much memory to design into a commercial server? →Is memory latency more important than memory size?

CAECW Salt Lake City -- Veazey & Gaither Experimental Design Vary memory GBytes → Measure Throughput Resource utilization –CPU, disk I/O, memory BW, CPI, OS context switches HP Integrity rx6600 → Itanium CPUs (2S/4C) → About 750 disk drives TPC-C → Resource intensive → Standard, “coin of the realm”…easy to communicate → Unofficial results

CAECW Salt Lake City -- Veazey & Gaither Throughput Increase of 48% in throughput

CAECW Salt Lake City -- Veazey & Gaither Resource Utilization I/O reduction accounts for 20% of the 48% throughput improvement. Where’s the rest of it? Disk I/O and CPU utilization GB Memthruput CPU Util.IOs / sec Relative thruput approx. % insts. devoted to I/O 32149, %71, % 64173, %58, % 96184, %50, % , %44, % , %29, %

CAECW Salt Lake City -- Veazey & Gaither CPI and Memory As memory is added, CPU cycles are used more efficiently But this is an effect, not a cause---why does CPI fall?

CAECW Salt Lake City -- Veazey & Gaither CPI and Memory Bandwidth CPI can change for many reasons, most irrelevant here Memory accesses are relevant – When a load misses cache, the delay counts toward CPI

CAECW Salt Lake City -- Veazey & Gaither Caches Stabilize with Increasing Memory Units normalized for throughput –accesses (or misses) / sec / CPU / tpmC L1 accesses imply that the registers also stabilize memory L1 accesses L1 misses L2 misses L3 misses

CAECW Salt Lake City -- Veazey & Gaither OS Thread Switches and Memory Reduced thread switches probably cause of register / cache stabilization --- working sets stay around longer

CAECW Salt Lake City -- Veazey & Gaither Summary and Conclusions Adding memory increases performance significantly I/O is reduced, as well as I/O instruction pathlength Context switches are reduced as a result of less I/O –Fewer memory accesses –Lower CPI –More stable caches and registers