1. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 20041 A Rapid Introduction to Garbage Collection Richard Jones Computing Laboratory University of Kent at Canterbury mm-net Garbage Collection & Memory Management Summer School Tuesday 20 July 2004 © Richard Jones, 2004. All rights reserved.

2. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 20042 PART 1: Introduction Motivation for garbage collection What to look for Motivation for garbage collection What to look for

3. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 20043 Why garbage collect? Finite storage requirement computer have finite, limited storage Language requirement many OO languages assume GC, e.g. allocated objects may survive much longer than the method that created them Problem requirement the nature of the problem may make it very hard/impossible to determine when something is garbage

4. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 20044 Why automatic garbage collection? Because human programmers just can’t get it right. Either too little is collected leading to memory leaks, or too much is collected leading to broken programs. Explicit memory management conflicts with the software engineering principles of abstraction and modularity. It’s not a silver bullet Some memory management problems cannot be solved using automatic GC, e.g. if you forget to drop references to objects that you no longer need. Some environments are inimical to garbage collection –embedded systems with limited memory –hard real-time systems

5. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 20045 PART 2: The Basics What is garbage? The concept of liveness by reachability The basic algorithms The cost of garbage collection What is garbage? The concept of liveness by reachability The basic algorithms The cost of garbage collection

6. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 20046 What is garbage? Almost all garbage collectors assume the following definition of live objects called liveness by reachability: if you can get to an object, then it is live. More formally: An object is live if and only if: it is referenced in a predefined variable called a root, or it is referenced in a variable contained in a live object (i.e. it is transitively referenced from a root). Non-live objects are called dead objects, i.e. garbage.

7. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 20047 Roots Objects and references can be considered a directed graph. Live objects are those reachable from a root. A process executing a computation is called a mutator — it simply modifies the object graph dynamically. Determining roots of a computation is, in general, language-dependent. In common language implementations roots include words in the static area registers words on the execution stack that point into the heap.

8. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 20048 The basic algorithms Reference counting: Keep a note on each object in your garage, indicating the number of live references to the object. If an object’s reference count goes to zero, throw the object out (it’s dead). Mark-Sweep: Put a note on objects you need (roots). Then recursively put a note on anything needed by a live object. Afterwards, check all objects and throw out objects without notes. Mark-Compact: Put notes on objects you need (as above). Move anything with a note on it to the back of the garage. Burn everything at the front of the garage (it’s all dead). Copying: Move objects you need to a new garage. Then recursively move anything needed by an object in the new garage. Afterwards, burn down the old garage (any objects in it are dead)!

9. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 20049 Update(left(R), S) Reference counting The simplest form of garbage collection is reference counting. Basic idea: count the number of references from live objects. Each object has a reference count (RC) when a reference is copied, the referent’s RC is incremented when a reference is deleted, the referent’s RC is decremented an object can be reclaimed when its RC = 0

10. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200410 Advantages of reference counting Simple to implement Costs distributed throughout program Good locality of reference: only touch old and new targets' RCs Works well because few objects are shared and many are short-lived Zombie time minimized: the zombie time is the time from when an object becomes garbage until it is collected Immediate finalisation is possible (due to near zero zombie time) OHPOHP

11. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200411 Disadvantages of reference counting  Not comprehensive (does not collect all garbage): cannot reclaim cyclic data structures  High cost of manipulating RCs: cost is ever-present even if no garbage is collected  Bad for concurrency — need Compare&Swap  Tightly coupled interface to mutator  High space overheads  Recursive freeing cascade OHPOHP

12. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200412 Mark-Sweep Mark-sweep is a tracing algorithm — it works by following (tracing) references from live objects to find other live objects. Implementation: Each object has a mark-bit associated with it. There are two phases: Mark phase: starting from the roots, the graph is traced and the mark-bit is set in each unmarked object encountered. At the end of the mark phase, unmarked objects are garbage. Sweep phase: starting from the bottom, the heap is swept –mark-bit not set:the object is reclaimed –mark-bit set:the mark-bit is cleared

13. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200413 A simple mark-sweep example 0 1 2

14. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200414 Comprehensive: cyclic garbage collected naturally No run-time overhead on pointer manipulations Loosely coupled to mutator Does not move objects does not break any mutator invariants optimiser-friendly requires only one reference to each live object to be discovered (rather than having to find every reference) Advantages of mark-sweep OHPOHP

15. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200415 Disadvantages of mark-sweep  Stop/start nature leads to disruptive pauses and long zombie times.  Complexity is O(heap) rather than O(live) every live object is visited in mark phase every object, alive or dead, is visited in sweep phase  Degrades with residency (heap occupancy) the collector needs headroom in the heap to avoid thrashing  Fragmentation and mark-stack overflow are issues  Tracing collectors must be able to find roots (unlike reference counting) OHPOHP

16. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200416 Fast allocation? Problem: Non-moving memory managers fragment the heap mark-sweep reference counting A compacted heap offers better spatial locality, e.g. better virtual memory and cache performance allows fast allocation –merely bump a pointer

17. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200417 Copying garbage collection Divide heap into 2 halves called semi-spaces and named Fromspace and Tospace Allocate objects in Tospace When Tospace is full flip the roles of the semi-spaces pick out all live data in Fromspace and copy them to Tospace preserve sharing by leaving a forwarding address in the Fromspace replica use Tospace objects as a work queue OHPOHP

18. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200418 copy root and update pointer, leaving forwarding address scan A' copy B and C, leaving forwarding addresses scan B' copy D and E, leaving forwarding addresses scan C' copy F and G, leaving forwarding addresses scan D' and E' nothing to do scan F' use A's forwarding address scan G' nothing to do scan=free so collection is complete Copying GC Example

19. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200419 Advantages of copying GC Compaction for free Allocation is very cheap for all object sizes out-of-space check is pointer comparison simply increment free pointer to allocate Only live data is processed (commonly a small fraction of the heap) Fixed space overheads free and scan pointers forwarding addresses can be written over user data Comprehensive: cyclic garbage collected naturally Simple to implement a reasonably efficient copying GC OHPOHP

20. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200420 Disadvantages of copying GC  Stop-and-copy may be disruptive Degrades with residency  Requires twice the address space of other simple collectors touch twice as many pages trade-off against fragmentation  Cost of copying large objects Long-lived data may be repeatedly copied  All references must be updated Moving objects may break mutator invariants  Breadth-first copying may disturb locality patterns

21. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200421 Mark-compact collection Mark-compact collectors make at least two passes over the heap after marking to relocate objects to update references (not necessarily in this order) Issues how many passes? compaction style –sliding: preserve the original order of objects –linearising: objects that reference each other are placed adjacently (as far as possible) –arbitrary: objects moved without regard for original order or referential locality

22. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200422 Cost metrics Many cost metrics can be interesting (albeit not necessarily at the same time). These cost metrics cover different types of concerns that may apply. The metrics are partially orthogonal, partially overlapping, and certainly also partially contradictory. In general it is not possible to identify one particular metric as the most important in all cases — it is application dependent. Because different GC algorithms emphasise different metrics, it is also, in general, not possible to point out one particular GC algorithm as “the best”. In the following, we present the most important metrics to consider when choosing a collector algorithm.

23. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200423 GC Metrics Execution time total execution time distribution of GC execution time time to allocate a new object Memory usage additional memory overhead fragmentation virtual memory and cache performance Delay time length of disruptive pauses zombie times Other important metrics comprehensiveness implementation simplicity and robustness

24. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200424 Execution time metrics Total execution time relevant for applications such as batch processing. can be less important for some applications, e.g. where there is much idle time (interactive applications). Distribution of GC execution time the absolute amount of execution time consumed may be less important than the amortisation of that cost over the mutator’s execution. The time to allocate a new object for some applications it may be important to be able to allocate new objects fast. NotesonlyNotesonly

25. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200425 Delay time metrics Length of disruptive pauses for applications requiring rapid response, e.g. most interactive applications, the length of the disruptive pauses introduced by the collector may be the most relevant metric. Zombie times the delay time from when an object becomes garbage until the memory allocated to it is actually collected. Long zombie times require more memory to be available (to house the dead, as yet uncollected, objects) NotesonlyNotesonly

26. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200426 Memory metrics Amount of extra memory consumed some algorithms work better (or simply require) large amounts of extra memory Memory fragmentation some algorithms result in much fragmentation of memory, while others actually reduce fragmentation Virtual memory and cache performance the interaction between virtual memory, caches, and the garbage collector can be quite important. Some algorithms touch all parts of allocated memory (live as well as dead objects and even unallocated memory) while others touch only limited amounts (e.g. live objects only). NotesonlyNotesonly

27. © Richard Jones, Eric Jul, 1999-2004mmnet GC & MM Summer School, 20-21 July 200427 Other important metrics Comprehensiveness does it find all garbage? Some collectors are comprehensive: they collect all garbage while others are conservative: they leave some garbage uncollected, for example, some do not collect cyclic object structures, while others retain some dead objects because these collectors cannot clearly identify all garbage. Implementation simplicity and robustness at times, simplicity of implementation is most important (get the job done!) is the garbage collector robust? How tightly coupled to the mutator is it? NotesonlyNotesonly