1. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Garbage Collection Without Paging Matthew Hertz, Yi Feng, Emery Berger University of Massachusetts Amherst

2. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Garbage Collection Performance Garbage collection now performs reasonably well High throughput Low pause times Given large heap and sufficient memory But: what happens when there’s not enough RAM?

3. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science GC Performance While Paging RAM Hard Disk Heap: most pages in RAM, one on disk GC begins

4. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science GC Performance While Paging Collector touches an evicted page... RAM Hard Disk

5. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science GC Performance While Paging... bringing the page into memory... RAM Hard Disk

6. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science GC Performance While Paging RAM Hard Disk... but triggers another page eviction.

7. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science GC Performance While Paging RAM Hard Disk... but triggers another page eviction.

8. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science GC Performance While Paging Collector touches newly-evicted page... RAM Hard Disk

9. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science GC Performance While Paging RAM Hard Disk Collector touches newly-evicted page...

10. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science GC Performance While Paging... leading to the eviction of yet another page… RAM Hard Disk

11. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science GC Performance While Paging … which eventually triggers more paging. RAM Hard Disk

12. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Program Throughput Paging causes a 40 to 62-fold increase in runtime

13. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Outline Motivation GC without paging Cooperative garbage collection Bookmarking Results

14. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science The problem Garbage collector: VM-oblivious Examines all reachable objects Lacks knowledge of page residency Treats evicted and resident pages identically Virtual memory: GC-oblivious GC & application have different access patterns Likely to evict pages needed by GC

15. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Cooperative Garbage Collection Bookmarking collector Extended virtual memory manager Select victim(s) Update residency page eviction notification victim page(s) Page replacement In response to notifications, BC: Adjusts heap size to fit in main memory Prevents eviction of important pages Avoids touching non-resident pages

16. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Avoiding Page Evictions When notified, avoid a pending eviction… 0000 RAM Hard Disk 0

17. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Avoiding Page Evictions …find a page BC knows to be empty… 0000 RAM Hard Disk 0

18. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Avoiding Page Evictions … and discard it… 0000 RAM Hard Disk 0

19. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Avoiding Page Evictions … eliminating the need to evict a page. 0000 RAM Hard Disk

20. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Limit of Heap Sizing Could collect, compact, compress, etc. Eventually: Will run out of pages to discard Going to have to evict non-empty pages Result: Paging Can we avoid this?

21. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Bookmarks We introduce bookmarks: Summaries of connectivity info on evicted pages References from objects on the page These summaries enable GC w/o paging

22. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0000 Bookmarking RAM Hard Disk Process page before eviction…

23. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0000 Bookmarking B B... by following pointers & bookmark-ing targets... RAM Hard Disk

24. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0000 Bookmarking B B... increment the referring page count... RAM Hard Disk 1

25. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 1000 Bookmarking BB BB B B B... conservatively bookmark objects on the page... RAM Hard Disk

26. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 000 Bookmarking... then tell extended VM to evict the page. RAM Hard Disk 0 BB BB B B B 1

27. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Bookmarking Details Cheap summary of connectivity One bit per object: free One word per page: referring page count Bookmarks cleared when count = zero Use bookmarks as secondary roots during garbage collection

28. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 BB BB B 100 RAM Hard Disk Collection with Bookmarks Process objects as usual, but... B B roots

29. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 BB BB B 100 RAM Hard Disk Collection with Bookmarks... ignore any references to evicted pages. B B roots

30. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 BB BB B 100 RAM Hard Disk Collection with Bookmarks Use bookmarks to recreate evicted references... B B roots

31. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 BB BB B 100 RAM Hard Disk Collection with Bookmarks... and continue collection. roots B B

32. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 BB BB B 100 RAM Hard Disk Collection with Bookmarks B B roots Result: Garbage collection without paging!

33. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 BB BB B 100 RAM Hard Disk Collection with Bookmarks B B roots Note: can waste space on evicted pages.

34. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Bookmarking Incompleteness Space waste not just on evicted pages Collection with bookmarks is necessarily incomplete Not guaranteed to reclaim all memory

35. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 BB BB B 100 RAM Hard Disk Bookmarking Incompleteness B B When a reference to an evicted object changes…

36. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 BB BB B 100 RAM Hard Disk Bookmarking Incompleteness B B When a reference to an evicted object changes…

37. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 10 0 0 RAM Hard Disk Bookmarking Incompleteness BB BB B B B …it can make evicted objects unreachable.

38. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 BB BB B 100 RAM Hard Disk Bookmarking Incompleteness B B But bookmarks cannot be removed…

39. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 BB BB B 100 RAM Hard Disk Bookmarking Incompleteness B B …retaining unreachable heap objects.

40. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Bookmarking Completeness Worst-case: completeness requires duplicating evicted pages See paper for more info How can we preserve completeness?

41. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 BB BB B 100 RAM Hard Disk Bookmarking Completeness B B If the heap becomes full…

42. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 000 RAM Hard Disk Bookmarking Completeness …BC removes all bookmarks…

43. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 000 ??? Bookmarking Completeness …and performs a VM-oblivious collection...

44. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 0 000 ??? Bookmarking Completeness …reclaiming all unreachable objects.

45. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 00 0 0 ??? Bookmarking Completeness …reclaiming all unreachable objects. BC’s worst case is other collectors’ common case

46. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science BC Performance Optimizations Uses generational design similar to GenMS Yields good performance when not paging Compacts heap to reduce pressure Prevents single object from tying down a page

47. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Outline Motivation GC without paging Cooperative garbage collection Bookmarking Results

48. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Experimental Methodology Extended Linux kernel 2.4.20 Eviction notification vm_relinquish() Added only 600 LOC Jikes RVM 2.3.2 & MMTk Compare BC to MarkSweep, SemiSpace, CopyMS, GenCopy, GenMS

49. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Throughput w/o Memory Pressure BC runtime comparable to GenMS

50. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Throughput while Paging BC throughput closely follows ideal curve

51. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science BMU Curve w/ Memory Pressure Bookmarking crucial for good performance 0.0% 1566 ms 71.9% 12231 ms 0.0% 81591 ms 39.1% 467944 ms

52. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Summary of Results When not paging: BC as fast as GenMS When paging: vs. GenMS (fastest when not paging) 41x faster, avg pause up to 218x smaller vs. CopyMS (next fastest when paging) 5x faster, avg pause up to 45x smaller

53. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Conclusion Bookmarking collector available at: http://www.cs.umass.edu/~hertz

54. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Thank you

55. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Other Pointer Summarizations Pointer Compression Save space by compressing pointers Tracing pointers becomes expensive Remembered Sets Add evicted pointers to buffers Requires space upon pages eviction Per Object Referring Page Counts Increases BC overheads

56. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Related Work VM-Sensitive Garbage Collection Linearizing LISP lists [Bobrow/Murphy] Does not know or use which heap pages are memory resident Independent heap regions [Bishop] Cannot avoid page faults when region contains evicted pages Ephemeral garbage collection [Moon] Must access evicted pages when they contain pointers into generations being collected

57. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Related Work VM-Cooperative Garbage Collection Discarding empty pages [Cooper, et al.] No mechanism for evicting non-empty pages Shrinking heap size [Alonso/Appel] Responds to memory pressure changes only after a collection Automatic heap sizing [Yang, et al.] Orthogonal approach that determines proper heap size

58. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Throughput w/ Memory Pressure  BC outperforms fixed nursery collectors

59. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Multiprogramming Performance  BC performs well in many environments

60. U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science Experimental Methodology Benchmarks: SPECjvm98, ipsixql, jython, and pseudoJBB “Opt-and-reset” methodology First iteration optimizes all code Record results from second run