1. MemGuard: Memory Bandwidth Reservation System for Efficient Performance Isolation in Multi-core Platforms Apr 9, 2012 Heechul Yun +, Gang Yao +, Rodolfo Pellizzoni *, Marco Caccamo +, Lui Sha + + University of Illinois at Urbana-Champaign * University of Waterloo

2. Real-Time Applications 2 Resource intensive real-time applications – Multimedia processing(*), real-time data analytic(**), object tracking Requirements – Need more performance and cost less  Commercial Off-The Shelf (COTS) – Performance guarantee (i.e., temporal predictability and isolation) (*) ARM, QoS for High-Performance and Power-Efficient HD Multimedia, 2010 (**) Intel, The Growing Importance of Big Data and Real-Time Analytics, 2012

3. Modern System-on-Chip (SoC) More cores – Freescale P4080 has 8 cores More sharing – Shared memory hierarchy (LLC, MC, DRAM) – Shared I/O channels 3 More performance Less cost But, isolation?

4. Problem: Shared Memory Hierarchy Shared hardware resources OS has little control Core1 Core2 Core3 Core4 DRAM Part 1 Part 2 Part 3Part 4 4 Memory Controller (MC) Shared Last Level Cache (LLC) Space contention Access contention

5. Memory Performance Isolation Q. How to guarantee worst-case performance? – Need to guarantee memory performance Part 1 Part 2 Part 3Part 4 5 Core1 Core2 Core3 Core4 DRAM Memory Controller LLC

6. Inter-Core Memory Interference Significant slowdown (Up to 2x on 2 cores) Slowdown is not proportional to memory bandwidth usage 6 Core Shared Memory foreground X-axis Intel Core2 L2 Slowdown ratio due to interference (1.6GB/s)(1.5GB/s) (1.4GB/s) Core background 470.lbm Runtime slowdown Core

7. Bank 4 Background: DRAM Chip Row 1 Row 2 Row 3 Row 4 Row 5 Bank 1 Row Buffer Bank 2 Bank 3 activate precharge Read/write State dependent access latency – Row miss: 19 cycles, Row hit: 9 cycles (*) PC6400-DDR2 with 5-5-5 (RAS-CAS-CL latency setting) READ (Bank 1, Row 3, Col 7) Col7

8. Background: Memory Controller(MC) Request queue – Buffer read/write requests from CPU cores – Unpredictable queuing delay due to reordering 8 Bruce Jacob et al, “Memory Systems: Cache, DRAM, Disk” Fig 13.1.

9. Background: MC Queue Re-ordering Improve row hit ratio and throughput Unpredictable queuing delay 9 Core1: READ Row 1, Col 1 Core2: READ Row 2, Col 1 Core1: READ Row 1, Col 2 Core1: READ Row 1, Col 1 Core1: READ Row 1, Col 2 Core2: READ Row 2, Col 1 DRAM Initial Queue Reordered Queue 2 Row Switch 1 Row Switch

10. Challenges for Real-Time Systems Memory controller(MC) level queuing delay – Main source of interference – Unpredictable (re-ordering) DRAM state dependent latency – DRAM row open/close state State of Art – Predictable DRAM controller h/w: [Akesson’07] [Paolieri’09] [Reineke’11]  not in COTS 10 Core 1 Core 2 Core 3 Core 4 DRAM Memory Controller DRAM Predictable Memory Controller

11. Our Approach OS level approach – Works on commodity h/w – Guarantees performance of each core – Maximizes memory performance if possible 11 Core 1 Core 2 Core 3 Core 4 DRAM Memory Controller DRAM OS control mechanism

12. Operating System MemGuard 12 Core1 Core2 Core3 Core4 PMC DRAM DIMM MemGuard Multicore Processor Memory Controller Memory bandwidth reservation and reclaiming BW Regulator BW Regulator BW Regulator BW Regulator 0.9GB/s0.1GB/s Reclaim Manager

13. Memory Bandwidth Reservation Idea – OS monitor and enforce each core’s memory bandwidth usage 13 1ms2ms 0 Dequeue tasks Enqueue tasks Dequeue tasks Budget Core activity 2121 computationmemory fetch

14. Memory Bandwidth Reservation 14

15. Guaranteed Bandwidth: r min 15 (*) PC6400-DDR2 with 5-5-5 (RAS-CAS-CL latency setting)

16. Memory Access Pattern Memory access patterns vary over time Static reservation is inefficient 16 Time(ms) Memory requests Time(ms) Memory requests

17. Memory Bandwidth Reclaiming Key objective – Redistribute excess bandwidth to demanding cores – Improve memory b/w utilization Predictive bandwidth donation and reclaiming – Donate unneeded budget predictively – Reclaim on-demand basis 17

18. Reclaim Example Time 0 – Initial budget 3 for both cores Time 3,4 – Decrement budgets Time 10 (period 1) – Predictive donation (total 4) Time 12,15 – Core 1: reclaim Time 16 – Core 0: reclaim Time 17 – Core 1: no remaining budget; dequeue tasks Time 20 (period 2) – Core 0 donates 2 – Core 1 do not donate 18

19. Best-effort Bandwidth Sharing Key objective – Utilize best-effort bandwidth whenever possible Best-effort bandwidth – After all cores use their budgets (i.e., delivering guaranteed bandwidth), before the next period begins Sharing policy – Maximize throughput – Broadcast all cores to continue to execute 19

20. Evaluation Platform Intel Core2Quad 8400, 4MB L2 cache, PC6400 DDR2 DRAM – Prefetchers were turned off for evaluation – Power PC based P4080 (8core) and ARM based Exynos4412(4core) also has been ported Modified Linux kernel 3.6.0 + MemGuard kernel module – https://github.com/heechul/memguard/wiki/MemGuard https://github.com/heechul/memguard/wiki/MemGuard Used the entire 29 benchmarks from SPEC2006 and synthetic benchmarks 20 Core 0 ID L2 Cache Intel Core2Quad Core 1 ID Core 2 ID L2 Cache Core 3 ID Memory Controller DRAM

21. Evaluation Results C0 Shared Memory C2 Intel Core2 L2 462.libquantum (foreground) memory hogs (background) Normalized IPC Foreground (462.libquantum) C2 53% performance reduction

22. Evaluation Results C0 Shared Memory C2 Intel Core2 L2 462.Libquantum (foreground) Normalized IPC Foreground (462.libquantum) 1GB /s memory hogs (background) C2.2GB /s Reservation provides performance isolation Guaranteed performance

23. Evaluation Results C0 Shared Memory C2 Intel Core2 L2 462.Libquantum (foreground) Normalized IPC Foreground (462.libquantum) 1GB /s memory hogs (background) C2.2GB /s Reclaiming and Sharing maximize performance Guaranteed performance

24. SPEC2006 Results 24 Normalized IPC Normalized to guaranteed performance Guarantee(soft) performance of foreground (x-axis) – W.r.t. 1.0GB/s memory b/w reservation C0 Shared Memory C2 Intel Core2 L2 X-axis (foreground) 1GB /s 470.lbm (background) C2.2GB /s

25. SPEC2006 Results Improve overall throughput – background: 368%, foreground(X-axis): 6% 25 Normalized IPC Normalized to guaranteed performance C0 Shared Memory C2 Intel Core2 L2 X-axis (foreground) 1GB /s 470.lbm (background) C2.2GB /s

26. Conclusion Inter-Core Memory Interference – Big challenge for multi-core based real-time systems – Sources: queuing delay in MC, state dependent latency in DRAM MemGuard – OS mechanism providing efficient per-core memory performance isolation on COTS H/W – Memory bandwidth reservation and reclaiming support – https://github.com/heechul/memguard/wiki/MemGuard https://github.com/heechul/memguard/wiki/MemGuard 26

27. Thank you. 27

28. Effect of Reclaim IPC improvement of background (lbm@0.2GB/s) is 3.8xlbm@0.2GB/s IPC reduction of foreground (SPEC@1.0GB/s) is 3%SPEC@1.0GB/s 28

29. Reclaim Underrun Error 29

30. Effect of Spare Sharing IPC of background (lbm@0.2GB/s) improves 40%lbm@0.2GB/s IPC of foreground (SPEC@1.0GB/s) also improves 9%SPEC@1.0GB/s 30

31. Isolation and Throughput Effect of r min 4 core configuration 31

32. Isolation Effect of Reservation Sum b/w reservation < r min (1.2GB/s)  Isolation – 1.0GB/s(X-axis) + 0.2GB/s(lbm) = r min 32 Isolation Core 0: 1.0 GB/s for X-axis Core 2: 0.2 – 2.0 GB/s for lbm Solo IPC@1.0GB/s

33. Effect of MemGuard Soft real-time application on each core. Provides differentiated memory bandwidth – weight for each core=1:2:4:8 for the guaranteed b/w, spare bandwidth sharing is enabled 33

34. Hard/Soft Reservation on MemGuard Hard reservation (w/o reclaiming) – Can guarantee memory bandwidth B i regardless of other cores at each period – Wasted if not used Soft reservation (w/ reclaiming) – Misprediction can caused missed b/w guarantee at each period – Error rate is small---less than 5%. Selectively applicable on per-core basis 34