1. Memory Management and Exceptions COMP 640 Week 5

2. COMP 640 – CWB Topics Memory Management Stack Heap Management Explicit: Allocate and Free Garbage Collection Exceptions

3. The Structure of Run-Time Memory Figure 5.1 Source: T & N, Figure 5.1

4. Stack Frame Source: T & N, Figures 5.3, 5.4

5. COMP 640 – CWB Heap Memory Management Explicit malloc/free, new/delete Issues: memory leaks, dangling references, orphaned data Reference counting Pools Garbage Collection new/ Issues: "stop-the-world" pauses, "forgotten" data

6. COMP 640 – CWB Malloc Design Issues Speed Fragmentation Memory efficiency "Locality" of request sizes Trimming

7. COMP 640 – CWB Strategies First fit Best fit Binning LRU – improve consolidation

8. COMP 640 – CWB The Doug Lea Memory Allocator Large requests (> 512 bytes): best-fit Small requests (< 64 bytes): caching allocator In between: compromise Very large (> 1 mb): use system memory mapping http://gee.cs.oswego.edu/dl/html/malloc.html

9. COMP 640 – CWB Memory Chunk Contents Size of previous chunk, if allocated Size of this chunk Plus previous-in-use flag The data itself If unallocated, forward and backward pointers for the free list Size of this chunk Boundary tags

10. COMP 640 – CWB Bins Each contains memory chunks in doubly linked list Small chunks – bins have equal sized chunks Medium chunks – bins have similarly sized chunks, sorted by size to facilitate best-fit search Large chunks – consolidated into one bin Very large chunks – not in bins Same-sized chunks allocated LRU to facilitate consolidation Fastbins: Preliminary "cache" Unordered, MRU lists of very recently freed small chunks Effective when lots of the same sized-chunks are being rapidly allocated and released

11. COMP 640 – CWB Bins 8 16 24 … 32 bins with fixed size chunks 256-512 512-1023 1024-2047 … 63 bins with varying sized chunks 524,288 up 1 bin with very large chunks 10 fastbins 8 16 24 … Each normal bin is a doubly linked list. Fastbins are singly linked.

12. COMP 640 – CWB malloc Algorithm (Approximate) If no free chunks, get more (consolidate or grow memory) Get from a fastbin if available, MRU If "small", get from bin if available, LRU If "large", consolidate all fastbins (!) Look at recently released, not-yet binned, free chunks If exact fit, return it If not exact fit, put it into its bin Get best fit in preferred bin (split if necessary) This is actually exact fit for small chunks. Search larger bins Last resort: Consolidate and try again If still fails, allocate more memory from system

13. COMP 640 – CWB Memory Leaks and Dangling References Source: T & N Figure 5.13 Leaked memory (orphan) Dangling reference

14. COMP 640 – CWB Reference Counting Use reference count in each object When assigning to a pointer Decrement ref count of object previously pointed to, if any Zero ref count  de-allocate the object Increment the ref count of object newly pointed to, if any C++ operator overloading Issue: inaccessible circular chains

15. Node Structure and Example Heap for Reference Counting Figure 5.14 Source: T & N Figure 5.14 What if this reference is changed?

16. COMP 640 – CWB Garbage Collection Reference Counting Used alone  inaccessible circular chains Mark-Sweep Copy Collection Generational Algorithms

17. COMP 640 – CWB Mark-Sweep GC Use mark-bit for each object When free list is empty Mark all objects that are accessible through pointers in the "root set" Return all unmarked objects to free list Extra memory per object Two passes: Greater stop-the-world pauses

18. COMP 640 – CWB Root Set Pointers in static memory Pointers in stack variables

19. COMP 640 – CWB Mark-Sweep GC Initial state: After Pass 1:After Pass 2: Source: T & N Figures 5.15, 16, 17

20. COMP 640 – CWB Copy Collection GC Divide heap into 2 equal spaces Begin using first space Whenever free list is empty when memory is requested: Flip spaces Set free list to the entire new space Recursively copy all referenced objects from old space to allocated space in the new space Single pass: potentially shorter pauses Requires copying Wasteful: allows use of only half of heap memory

21. COMP 640 – CWB Generational Algorithms and the Train Algorithm References: http://www-106.ibm.com/developerworks/java/library/j- jtp11253/ http://www-106.ibm.com/developerworks/java/library/j- jtp11253/ http://www.daimi.au.dk/~beta/Papers/Train/train.html

22. COMP 640 – CWB Generational Algorithms Observations: Most objects "die young" – they're freed shortly after being allocated Copying garbage collectors work well when most objects die young since they don't have to be copied at all. Mark-sweep works better for the more stable older objects Therefore, use copying on the younger generation of objects and mark-sweep on the older Promote young objects to the older generation when they survive n GC cycles (n >= 1)

23. COMP 640 – CWB Generations Younger Objects Older Objects Root Set Promotion

24. COMP 640 – CWB The Root Pointer Set In single generation algorithms, root pointers are stack and static variables For younger generation GC, root set includes Root pointers to younger objects Pointers from the older generation to younger objects Record pointers in objects that are promoted Record changes to pointers in older objects

25. COMP 640 – CWB Train Algorithm Divide older objects into several sets of objects (trains) Each train has several fixed-size subsets (cars) Each car can be GC'ed separately  very short pauses Higher overhead maintaining pointers to younger cars and trains

26. COMP 640 – CWB Train Algorithm Younger Generation Older Generation Trains Goal: cluster interlinked garbage into the same train Each car has a "remembered set" – a list of pointers into the car from later cars and trains Promotion

27. COMP 640 – CWB Train Algorithm Collect one train, T, at a time If no links into the train, entire train is garbage Evacuate one car, C, at a time Objects that are referenced from objects in another train are moved to that train (If train fills, add a car) Objects referenced from the young generation are moved to any train (other than T) Transitive closure: check evacuated object for pointers back to car C and move those to the same train Remaining objects in C (which are referenced only within train T) are evacuated to the last car in T (with transitive closure) Continued …

28. COMP 640 – CWB Train Algorithm, cont. When younger objects are promoted They are moved to any train (except the one currently being collected) All affected remembered sets are updated When pointers are changed: The pointer mutator (runtime code) maintains the remembered set

29. COMP 640 – CWB Why It Works Any garbage structure (inaccessible circular chain), G, is eventually forced into a single train and thus released. Why? G cannot grow or change (inaccessible) Pieces of G are moved into other (later) trains that contain other pieces of G If all the trains are eventually processed, then G is forced into one train See http://www.daimi.au.dk/~beta/Papers/Train/train.html for a modification needed to make this assumption validhttp://www.daimi.au.dk/~beta/Papers/Train/train.html

30. COMP 640 – CWB Popular Objects Objects with large numbers of references to them disrupt the train algorithm by inducing large remembered sets. Thus, for a car, C, with popular objects: Evacuate from C only the unpopular objects If C contains multiple popular objects (logically) split the car, creating C', C'', etc. Move the C (now containing only a single popular object) to the end of the highest train that contains an object with a reference to C. Similarly move C', C'', etc., if any.

31. COMP 640 – CWB Exceptions Traditional techniques No exception mechanism in language (C) unusual method value by-reference method arguments global error indicator Global signals

32. COMP 640 – CWB Modern Exception Handler Language Features Source: T & N Figure 6.1 Retry

33. COMP 640 – CWB Retry in Java while(true) try { some code that may fail break; } catch(CorrectableException ce) { code to fix things } catch(UncorrectableException ue) { error reporting throw something; } Must be re-startable as a unit after failure What about Resumption in Java?

34. COMP 640 – CWB Java Exception Handling Specification " … the Java virtual machine abruptly completes, one by one, any expressions, statements, method and constructor invocations, initializers, and field initialization expressions that have begun but not completed execution in the current thread. …" "This process continues until a handler is found …" If none is found, The handler set by setUncaughtExceptionHandler in the current thread is invoked, if any. Otherwise, the uncaughtException method is called in the thread group owning the current thread http://java.sun.com/docs/books/jls/second_e dition/html/exceptions.doc.html#44044 http://java.sun.com/docs/books/jls/second_e dition/html/exceptions.doc.html#44044

35. COMP 640 – CWB Cleanup During an Exception Unlock synchronization locks Release local variables (if necessary for the memory management scheme)

36. COMP 640 – CWB Checked Exceptions in Java Certain exception classes are designated as "checked" The Exception class and its subclasses But not: the RuntimeException class and its subclasses A call to a method throwing a checked exception, E, must either Be in a try block that has a catch for E or a superclass of E Be in a method that throws E or a superclass of E http://java.sun.com/j2se/1.5.0/docs/api/

37. COMP 640 – CWB Preciseness of Exceptions When exception is thrown: The effects of "all statements executed or expressions evaluated" before the exception must occur. The effects of statements and expressions after the exception must not occur. (Speculative optimization must be undone.)

38. COMP 640 – CWB Synchronous and Asynchronous Exceptions Sync: occur at precisely defined points in the code Async: occur an any point in code Java: stop invoked by another thread JVM errors Non-interpreted languages: Signals

39. COMP 640 – CWB Exceptions in Non- interpreted Languages Come-from tables Lists of ranges of code where a particular exception might be expected A function may have any number of non- overlapping ranges Range is mapped to a catch clause and type Exception handling must invoke local destructors or other code expected to occur at the end of a function call ("stack unwinding")

40. COMP 640 – CWB Next Week Imperative Languages – Chapter 4 Abstract syntax for non-Jay statements Code Generation Selected Topics References: http://flint.cs.yale.edu/cs421/lectureNotes/c09.pdf http://flint.cs.yale.edu/cs421/lectureNotes/c10.pdf http://flint.cs.yale.edu/cs421/lectureNotes/c11.pdf http://flint.cs.yale.edu/cs421/lectureNotes/c10.pdf http://flint.cs.yale.edu/cs421/lectureNotes/c11.pdf http://www.scifac.ru.ac.za/compilers/cha15s.htm#C15-1 http://en.wikipedia.org/wiki/Compiler_optimization#Type s_of_Optimizations (and links) http://en.wikipedia.org/wiki/Compiler_optimization#Type s_of_Optimizations