1. Automatic Memory Management/GC
CS 3214 Computer Systems
Automatic Memory Management/GC

2. Memory Management Part 2
Some of the following slides are taken with permission from
Complete Powerpoint Lecture Notes for Computer Systems: A Programmer's Perspective (CS:APP)
Randal E. Bryant and David R. O'Hallaron
Part 2
Memory Management

3. Dynamic Memory Allocation
Application
Dynamic Memory Allocator
Heap Memory
Explicit vs. Implicit Memory Allocator
Explicit: application allocates and frees space
E.g., malloc and free in C
Implicit: application allocates, but does not free space
E.g. garbage collection in Java, ML or Lisp
Allocation
The memory allocator provides an abstraction of memory as a set of blocks or, in type-safe languages, as objects
Doles out free memory blocks to application
Will discuss automatic memory allocation today

4. Implicit Memory Management
Motivation: manually (or explicitly) reclaiming memory is difficult:
Too early: risk access-after-free errors
Too late: memory leaks
Requires principled design
Programmer must reason about ownership of objects
Difficult & error prone, especially in the presence of object sharing
Complicates design of APIs

5. Manual Reference Counting
Idea: keep track of how many references there are to each object in a reference counter stored with each object
Copy a reference to an object globalvar = q
increment count: “addref”
Remove a reference p = NULL
decrement count: “release”
Uses set of rules programmers must follow
E.g., must ‘release’ reference obtained from OUT parameter in function call
Must ‘addref’ when storing into global
May not have to use addref/release for references copied within one function
Programmer must use addref/release correctly
Still somewhat error prone, but rules are such that correctness of the code can be established locally without consulting the API documentation of any functions being called; parameter annotations (IN, INOUT, OUT, return value) imply reference counting rules
Used in Microsoft COM & Netscape XPCOM

6. Automatic Reference Counting
Idea: force automatic reference count updates when pointers are assigned/copied
Most common variant:
C++ “smart pointers” – C++ allows programmer to interpose on assignments and copies via operator overloading/special purpose constructors.
Disadvantage of all reference counting schemes is their inability to handle cycles
But great advantage is immediate reclamation: no “drag” between last access & reclamation

7. Garbage Collection
Determine which objects may be accessed in the future
Don’t know which ones will, but can determine those who can’t be accessed because there are no pointers to them
Requires that all pointers are identifiable (e.g., no pointer/int conversion)
Invented 1960 by McCarthy for LISP

8. Reachability Graph Root set
All tasks allocate from the same shared pool.
Reference graph traversal.
Thread A
Thread B
Thread C

9. Reachability Graph Root set
All tasks allocate from the same shared pool.
Reference graph traversal.
Thread A
Thread B
Thread C

10. Reachability Graph Root set
All tasks allocate from the same shared pool.
Reference graph traversal.
Thread A
Thread B
Thread C

11. Reachability Graph Root set
All tasks allocate from the same shared pool.
Reference graph traversal.
Thread A
Thread B

12. Reachability Graph Root set
All tasks allocate from the same shared pool.
Reference graph traversal.
Thread A

13. Reachability Graph Root set
All tasks allocate from the same shared pool.
Reference graph traversal.
Thread A

14. GC Design Choices Determining which objects are reachable
“marking” live objects, or
“evacuating”/”scavenging” objects – copying live objects into new area (if objects are movable)
Deallocating unreachable objects
“sweeping” – essentially calling “free()” on all unreachable objects
more efficient if it’s possible to evacuate all live objects from an area
cost generally
proportional to
amount of live
objects in area
considered
cost proportional to amount of dead
objects (garbage)
in theory, constant cost;
in practice, dominated by need to zero memory

15. Memory Allocation Time-Profile
Start time – ts
Time
Allocated
Memory
Amax
live
garbage
For GC’ed systems, we extend it to include live  difference is garbage
This picture assumes the task has infinite memory during its execution.
End time – te

16. Modeling Memory Allocation
Allocated
Memory
Amax
allocation rate
garbage
For GC’ed systems, we extend it to include live  difference is garbage
This picture assumes the task has infinite memory during its execution.
live
Time
Start time – ts
End time – te

17. Execution Time vs. Memory
Max Heap
time ts te

18. Execution Time vs. Memory
Max Heap
time ts te

19. Execution Time vs. Memory
Max Heap
time ts te

20. Execution Time vs. Memory
Max Heap
time ts te

21. Execution Time vs. Memory
Max Heap
time
The answer is never – if you have enough memory for your profile.
But what if you don’t? ts te

22. Heap Size vs. GC Frequency
All else being equal, smaller maximum heap sizes necessitate more frequent collections
Old rule of thumb: need between 1.5x and 2.5x times the size of the live heap to limit collection overhead to 5-15% for applications with reasonable allocation rates
[Hertz 2005] finds that GC outperforms explicit MM when given 5x memory, is 17% slower with 3x, and 70% slower with 2x
Performance degradation occurs when live heap size approaches maximum heap size

23. Infant Mortality
Source:

24. Generational Collection
Observation: “most objects die young”
Allocate objects in separate area (“nursery”, “Eden space”), collect area when run out of space
Will typically have to evacuate few survivors
“minor garbage collection”
But: must treat all pointers into Eden as roots
Typically, requires cooperation of the mutator threads to record assignments: if ‘b’ is young, and ‘a’ is old, a.x = b must add a root for ‘b’.
Aka “write barrier”

25. When to collect “Stop-the-world” Incremental Concurrent/Parallel
All mutators stop while collection is ongoing
Incremental
Mutators perform small chunks of marking during each allocation
Concurrent/Parallel
Garbage collection happens in concurrently running thread – requires some kind of synchronization between mutator & collector

26. Example: G1GC
See Oracle tutorial and InfoQ.
Source: [Beckwith 2013]

27. Precise vs. Conservative Collectors
Precise collectors keep only objects alive that are in fact part of reachability graph
Conservative collectors may keep objects alive that aren’t
Reason typically that they do not know where pointers are stored, must conservatively guess
In-between forms: some systems assume precise knowledge of heap objects, but not stack frame layouts
Can be expensive to keep track of where references are stored on the stack, particularly in fully preemptive environments
Conservatism makes GC usable for languages such as C, but prevents moving/compacting of objects

28. Application Programmer’s Perspective
Dealing with Memory Leaks
Avoiding bloat
Avoiding churn
Tuning garbage collection parameters
Garbage collection in mixed language environments
C code must coordinate with the garbage collection system in place

29. Programmer’s Perspective
Your program is running out of memory. What do you do?
Possible reasons:
Leak
Bloat
Your program is running slowly and unpredictably
Churn
“GC Trashing”

30. Memory Leaks
Objects that remain reachable, but will not be accessed in the future
Due to application semantics
Will ultimately lead to out-of-memory condition
But will degrade performance before that
Common problem, particularly in multi-layer frameworks
Containers are a frequent culprit
Heap profilers can help

31. Bloat and Churn
Bloat: use of inefficient, pointer-intensive data structures increases overall memory consumption [Chis et al 2011]
E.g. HashMaps for small objects
Churn: frequent and avoidable allocation of objects that turn to garbage quickly
Caches:
How to implement and size caches