Heap liveness and its usage in automatic memory management Ran Shaham Elliot Kolodner Mooly Sagiv ISMM’02 Unpublished TVLA inside

Motivation An object could be collected once it is no longer needed –Yet, run-time garbage collectors (RTGCs) are typically based on reachability Profiling tools can detect when objects are needed The compiler can: –Statically identify a subset of unneeded objects –Issue a free instruction (compile-time Garbage Collection) –Issue a warning when a potentially needed object is reclaimed –Inform run-time garbage collector that a reference to an object is not further used

A Pathological C Program a = malloc(…) ; b = a; free (a); c = malloc (…); if (b == c) printf(“unexpected equality”);

Inefficient Java Class public Class Stack { private Object stack[]; private int top; public Stack(int len) { stack = new Object[len]; top = 0; } public synchronized Object pop() { top= top-1; return stack[top]; } public synchronized void push(Object o) { stack[top]=o; top= top+1; } public synchronized void print() { for (int i=0; i<top; i++) { System.out.println(stack[i]); } GC does not reclaim the memory stack[top+1]

Needed Location p p’ l is needed a reference to l is used l is allocated

Needed Reference Expression p p’ e is needed a reference to l is used e references l e is not needed  free(e) is valid l is allocated

A Pathological C Program a = malloc(…) ; b = a; free (a); c = malloc (…); if (b == c) printf(“unexpected equality”); a is needed

Location Liveness p p’ l is live l is not assigned l is used

Reference Expression Liveness p p’ e is live e denotes a location l l is not assigned l is used Generalizes liveness of program variables when l is &x

Inefficient Java Class public Class Stack { private Object stack[]; private int top; public Stack(int len) { stack = new Object[len]; top = 0; } public synchronized Object pop() { top= top-1; return stack[top]; } public synchronized void push(Object o) { stack[top]=o; top= top+1; } public synchronized void print() { for (int i=0; i<top; i++) { System.out.println(stack[i]); } stack[top+1] is not live

Typical GC Limits class Node { Node left, right; int data; } class C { void main(…) { Node root = createTree(); processTree(root.right); } root

Typical GC Limits class Node { Node left, right; int data; } class C { void main(…) { Node root = createTree(); processTree(root.right); } root

Typical GC Limits class Node { Node left, right; int data; } class C { void main(…) { Node root = createTree(); processTree(root.right); } root

Liveness Analysis Aids GC class Node { Node left, right; int data; } class C { void main(…) { Node root = createTree(); processTree(root.right); } root root.right is live, root.left is dead

Liveness Analysis Aids GC class Node { Node left, right; int data; } class C { void main(…) { Node root = createTree(); processTree(root.right); } root root.right is live, root.left is dead

Liveness Analysis Aids GC class Node { Node left, right; int data; } class C { void main(…) { Node root = createTree(); processTree(root.right); } root root.right is live, root.left is dead

Liveness Analysis Aids GC class Node { Node left, right; int data; } class C { void main(…) { Node root = createTree(); processTree(root.right); } root root.right is live, root.left is dead

Typical GC Limits Program Variables a b c d e f

Outline Dynamic liveness measurements –Complete location liveness –Assign-null interface Static analysis algorithms –(Exact) assign null (improve GC) –(Exact) free (CTGC)

Dynamic Liveness Measurements Estimating the potential of static analysis –Find upper bounds on expected savings Can be used in as an assistant tool Liveness information kinds –Stack reference liveness –Global reference liveness –Heap reference liveness

Main Results Dynamic measurements for 10 benchmarks Shallow information  Small Potential –local variables 2% –global variables 5% –local + global 9% Deep information  Larger Potential –heap liveness 39% complete location liveness 15% assign-null interface

Dynamic measurements –Implemented via an instrumented JVM Complete location liveness measurements –Single-run algorithm Assign-null liveness interface –Determines the liveness of expressions –Assign null to dead reference expressions –Requires two runs of the program

Complete Liveness Measurements An Observation The last use of the references to an object determines the time an object could be collected assuming liveness information

Heap Liveness Example I Stack y z g Static x f f2f2 f1f1 f f Heap L = t … use z.f f Heap L  = t

Heap Liveness Example I Stack y z g Static x f f2f2 f1f1 f f Heap L = t+2 … use y.f2 f Heap L = t  = t+2

Heap Liveness Example II Stack y z g Static x f f2f2 f1f1 f f Stack R = t’’ f Stack R = Directly stack reachable (computed during GC) Collection time = max(t’, t’’) Collection time(obj) = max(Heap L (obj), Stack R (obj), Static R (obj), Other R (obj))  = t’’ Heap L = t’

Stack Liveness Example I Stack y z g Static x f f2f2 f1f1 f f Stack L = t … use z [Current Time = t] f Stack L

Stack Liveness Example I Stack y z g Static x f f2f2 f1f1 f f Stack L = t f … use y [Current Time = t+2] Stack L = t+2

Stack Liveness Example I Stack y z g Static x f f2f2 f1f1 f f Stack L = t f Stack L* = path starting with a live stack root (computed during GC trace phase) Stack L = t+2

Stack Liveness Example I Stack y z g Static x f f2f2 f1f1 f f Stack L* = t f Stack L* = path starting with a live stack root (computed during GC trace phase) Stack L* = t+2

Stack Liveness Example II Stack y z g Static x f f2f2 f1f1 f f Stack L* = t f Static R* = path starting with a static root (computed during GC trace phase) Stack L* = t+2 Static R* = t’ Stack L* = t+2 Static R* = t’ Collection time = max(t+2, t’) Collection time(obj) = max(Stack L* (obj), Static R* (obj), Other R* (obj))

PropertyIntended Meaning heap L (obj)obj is referenced by a live heap reference at time heap L (obj) stack L (obj)obj is referenced by a live stack root at time stack L (obj) stack L* (obj)obj is reachable along a path starting from a live stack root at time stack L* (obj) stack R (obj)obj is referenced by a stack root at time stack R (obj) stack R* (obj)obj is reachable along a path starting from a stack root at time stack R* (obj) static L (obj)obj is referenced by a live static root at time static L (obj) static L* (obj)obj is reachable along a path starting from a live static root at time static l* (obj) static R (obj)obj is referenced by a static root at time static R (obj) static R* (obj)obj is reachable along a path starting from a static root at time static R* (obj) other R (obj)obj is referenced by an other root at time other R (obj) other R* (obj)obj is reachable along a path starting from an other root at time other R* (obj)

Complete Liveness Summary Mutator –Tracks the last use of references to an object Collector –Propagation needed for stack/static liveness –Propagates reachability information –Propagates path liveness Object Collection/Program Termination –Maximum of liveness/reachability properties of an object –Depends on liveness scheme (heap liveness etc.)

Experimental Results Instrumented Sun’s classic JVM (1.2) 10 benchmarks (5 SPECjvm) Time is measured bytes allocated by the mutator so far in program Total space savings (integral) Maximum heap size savings (footprint)

Potential Savings – Total Space

Restricted GC Interface GC Interface –Should be simple/effective/efficient –Feasible heap liveness representation Assign null to dead heap references –Simple –Effective? Partially answered by our experiments –Efficient? Will be answered by static analysis

Null Assignable Program Points Normalized statements (Java Bytecode) –Manipulate at most one heap reference x = y.f is null assignable –Could be followed by y.f = null Dynamic algorithm –First run Determine null assignable program points –Assume all program points are null assignable –Detect non-null-assignable program points during the run –Second run Assign null in null assignable program points

Doubly-Linked List Example – First Run x n p dd n p dd // processing list elements in pairs pt 1 : y = x.n; // x.n = null; x = null; while (y != null) { pt 2 : t = y.p; // y.p = null; pt 3 : d1 = t.d; // t.d = null pt 4 : d2 = y.d; // y.d = null process(d1, d2); pt 5 : t = y.n; // y.n = null; y = t; }

Doubly-Linked List Example – First Run x n p dd n p dd [pt 1 ] // processing list elements in pairs pt 1 : y = x.n; // x.n = null; x = null; while (y != null) { pt 2 : t = y.p; // y.p = null; pt 3 : d1 = t.d; // t.d = null pt 4 : d2 = y.d; // y.d = null process(d1, d2); pt 5 : t = y.n; // y.n = null; y = t; } y

Doubly-Linked List Example – First Run n p dd n p dd y t [pt 2 ] [pt 1 ] // processing list elements in pairs pt 1 : y = x.n; // x.n = null; x = null; while (y != null) { pt 2 : t = y.p; // y.p = null; pt 3 : d1 = t.d; // t.d = null pt 4 : d2 = y.d; // y.d = null process(d1, d2); pt 5 : t = y.n; // y.n = null; y = t; }

Doubly-Linked List Example – First Run n p dd n p dd y t [pt 2 ] [pt 3 ] d1 [pt 1 ] // processing list elements in pairs pt 1 : y = x.n; // x.n = null; x = null; while (y != null) { pt 2 : t = y.p; // y.p = null; pt 3 : d1 = t.d; // t.d = null pt 4 : d2 = y.d; // y.d = null process(d1, d2); pt 5 : t = y.n; // y.n = null; y = t; }

Doubly-Linked List Example – First Run n p dd n p dd y t [pt 2 ] [pt 3 ] d1 [pt 4 ] d2 [pt 1 ] // processing list elements in pairs pt 1 : y = x.n; // x.n = null; x = null; while (y != null) { pt 2 : t = y.p; // y.p = null; pt 3 : d1 = t.d; // t.d = null pt 4 : d2 = y.d; // y.d = null process(d1, d2); pt 5 : t = y.n; // y.n = null; y = t; }

Doubly-Linked List Example – First Run n p dd n p dd y t [pt 2 ] [pt 3 ] d1 [pt 4 ] d2 [pt 5 ] [pt 1 ] // processing list elements in pairs pt 1 : y = x.n; // x.n = null; x = null; while (y != null) { pt 2 : t = y.p; // y.p = null; pt 3 : d1 = t.d; // t.d = null pt 4 : d2 = y.d; // y.d = null process(d1, d2); pt 5 : t = y.n; // y.n = null; y = t; }

Doubly-Linked List Example – First Run n p dd n p dd t y [pt 2 ] [pt 3 ] d1 [pt 4 ] d2 [pt 5 ] [pt 2 ] [pt 1 ] // processing list elements in pairs pt 1 : y = x.n; // x.n = null; x = null; while (y != null) { pt 2 : t = y.p; // y.p = null; pt 3 : d1 = t.d; // t.d = null pt 4 : d2 = y.d; // y.d = null process(d1, d2); pt 5 : t = y.n; // y.n = null; y = t; }

[pt 4 ] Doubly-Linked List Example – First Run n p dd n p dd t y [pt 2 ] [pt 3 ] d1 [pt 3 ] d2 [pt 5 ] [pt 1 ] // processing list elements in pairs pt 1 : y = x.n; // x.n = null; x = null; while (y != null) { pt 2 : t = y.p; // y.p = null; pt 3 : d1 = t.d; // t.d = null pt 4 : d2 = y.d; // y.d = null process(d1, d2); pt 5 : t = y.n; // y.n = null; y = t; }

Doubly-Linked List Example – First Run n p dd n p dd [pt 2 ] [pt 3 ] d2 [pt 3 ] d1 [pt 5 ] [pt 3 ] [pt 1 ] [pt 3 ] // processing list elements in pairs pt 1 : y = x.n; // x.n = null; x = null; while (y != null) { pt 2 : t = y.p; // y.p = null; pt 3 : d1 = t.d; // t.d = null pt 4 : d2 = y.d; process(d1, d2); pt 5 : t = y.n; // y.n = null; y = t; }

Doubly-Linked List Example - Second Run x n p dd n p dd // processing list elements in pairs pt 1 : y = x.n; x.n = null; x = null; while (y != null) { pt 2 : t = y.p; y.p = null; pt 3 : d1 = t.d; t.d = null pt 4 : d2 = y.d; process(d1, d2); pt 5 : t = y.n; y.n = null; y = t; }

n p dd n p d d ty // processing list elements in pairs pt 1 : y = x.n; x.n = null; x = null; while (y != null) { pt 2 : t = y.p; y.p = null; pt 3 : d1 = t.d; t.d = null pt 4 : d2 = y.d; process(d1, d2); pt 5 : t = y.n; y.n = null; y = t; } Doubly-Linked List Example - Second Run d1 d2

// processing list elements in pairs pt 1 : y = x.n; x.n = null; x = null; while (y != null) { pt 2 : t = y.p; y.p = null; pt 3 : d1 = t.d; t.d = null pt 4 : d2 = y.d; process(d1, d2); pt 5 : t = y.n; y.n = null; y = t; } Doubly-Linked List Example - Second Run d n p d d ty d1 d2

15% average savings for context = 2 –11% assigning null to instance fields –10% assigning null to array elements Results are valid across runs –Detecting null assignable program points on a second input –Running the program with the first input –null assignable program points are those detected for both inputs

Related Work On the Usefulness of Liveness for Garbage Collection and Leak Detection [HDH01] –Does not handle heap liveness –Algorithm requires two runs First run: record uses and defs Analyze log backwards for liveness information Second run: use liveness results Garbage Collection and Local Variable Type-Precision and Liveness in Java Virtual Machines [ADM98] –Stack liveness static analysis –Actual trends match our upper bounds On the Effectiveness of GC in Java [SKS00] –Drag information Slightly larger potential than heap liveness information Not clear how to automate space savings HUP tool (PLDI’01 + M. Pan)

Dynamic liveness measurements - Conclusion Liveness Information has large potential Assign null savings “achievable” by static analysis Stack liveness information –Small potential Stack+static liveness information –Medium potential Heap liveness information –Is feasible Recording history on heap is a powerful mechanism –Larger potential Depends on static analysis precision Depends on GC interface

Static Analysis Combine history with shape analysis –a-La- Horwitz, Pfeiffer, and Reps 1989 Assign null –Assign null to a dead reference expression –GC exploits information Free –free an unneeded object

Assign Null Analysis –Insert “x.fld = null” after statements in which the expression x.fld becomes dead –Limitations Only one reference is assigned null All the paths to the statement must agree –Detects last-use –Technically llastu[pt,x.fld](v) – The last use of the location denoted by x.fld occurs at pt null[pt,x.fld]() –It is safe to insert “x.fld = null” after pt

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n n null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x,y n n null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x,y n n null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x,y n llastu[pt 3,y.n] n n t null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n llastu[pt 3,y.n] n n y,t null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n llastu[pt 3,y.n] n n y,t null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n llastu[pt 3,y.n] n n y n t null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n llastu[pt 3,y.n] n n n y,t null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n llastu[pt 3,y.n] n n n y,t null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n llastu[pt 3,y.n] n n n y n t null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n n llastu[pt 3,y.n] n n y,t n null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n n llastu[pt 3,y.n] n n y,t n

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n llastu[pt 3,y.n] n n n y n t n null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n llastu[pt 3,y.n] n n n y,t n null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n llastu[pt 3,y.n] n n n y,t n null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n llastu[pt 3,y.n] n y n null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n llastu[pt 3,y.n] n null[pt 3,y.n]

Assign Null Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n llastu[pt 3,y.n] n null[pt 3,y.n]

Static Reachability Information + Static Liveness Information CT garbage detection Issue “free(e)” for unneeded e Compile-Time Garbage Collection

Exact Free Analysis –Insert free(x) at program points when the x becomes unneeded Only one location is freed All the paths to the statement must agree on the garbage –History predicates lastu[pt,x](v) – the last use of the location pointed-to by x occurs unneeded [pt,x]() –“free(x)” is safe after pt

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n n unneeded[pt 1,x] unneeded[pt 2,y] unneeded[pt 3,y] unneeded[pt 4,t]

Exact unneeded Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x,y n n unneeded[pt 1,x] unneeded[pt 2,y] unneeded[pt 3,y] unneeded[pt 4,t] lastu[pt 1,x]

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x,y n n unneeded[pt 2,y] unneeded[pt 3,y] unneeded[pt 4,t] lastu[pt 2,y]

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x,y n unneeded[pt 3,y] unneeded[pt 4,t] lastu[pt 3,y] n n t

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n unneeded[pt 3,y] unneeded[pt 4,t] lastu[pt 3,y] n n y,t lastu[pt 4,t]

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n unneeded[pt 3,y] lastu[pt 3,y] n n y,t lastu[pt 2,y]

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n unneeded[pt 3,y] lastu[pt 3,y] n n y n t

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n unneeded[pt 3,y] lastu[pt 3,y] n n n lastu[pt 4,t] y,t

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n unneeded[pt 3,y] lastu[pt 3,y] n n n lastu[pt 2,y] y,t

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n unneeded[pt 3,y] lastu[pt 3,y] n n n y n t

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x unneeded[pt 3,y] n n lastu[pt 3,y] n n y,t lastu[pt 4,t] n

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x unneeded[pt 3,y] n n lastu[pt 3,y] n n y,t lastu[pt 2,y] n

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n unneeded[pt 3,y] lastu[pt 3,y] n n n y n t n

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n unneeded[pt 3,y] lastu[pt 3,y] n n n y,t n lastu[pt 4,t]

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n unneeded[pt 3,y] lastu[pt 3,y] n n n y,t n lastu[pt 2,y]

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n unneeded[pt 3,y] lastu[pt 3,y] n n y

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n unneeded[pt 3,y] lastu[pt 3,y] n

Exact Free Example // traversing list elements pt 1 : y = x; pt 2 : while (y != null) { pt 3 : t = y.n; pt 4 : y = t; } x n unneeded[pt 3,y] lastu[pt 3,y] n

Preliminary Conclusions Heap Liveness is useful Can be dynamically computed for large programs More data is needed to confirm scalability of static analysis Recording local history is a powerful tool –Refines the abstraction