Michael Bond Kathryn McKinley The University of Texas at Austin

 Driverless truck  10,000 lines of C#  Leak: past obstacles remained reachable  No immediate symptoms “This problem was pernicious because it only showed up after 40 minutes to an hour of driving around and collecting obstacles.”  Quick “fix”: restart after 40 minutes  Environment sensitive  More obstacles in deployed setting  Unresponsive after 28 minutes

 Driverless truck  10,000 lines of C#  Leak: past obstacles remained reachable  No immediate symptoms “This problem was pernicious because it only showed up after 40 minutes to an hour of driving around and collecting obstacles.”  Quick “fix”: restart after 40 minutes  Environment sensitive  More obstacles in deployed setting  Unresponsive after 28 minutes Different environments & inputs  different behavior Uncertainty in deployed systems Hard to fix before deployment

 Deployed systems have leaks  Critical systems need immediate help  Leak pruning tolerates bad effects  Reclaims memory automatically  High precision & low overhead  Bounds resources  Preserves semantics

 Why tolerate leaks  Why leaks are so bad  How leak pruning works  How leak pruning predicts leaked objects  Evaluation

 Memory leaks are a real problem  Managed languages do not eliminate them

 Memory leaks are a real problem  Managed languages do not eliminate them Reachable Unreachable

 Memory leaks are a real problem  Managed languages do not eliminate them Live ReachableDead

 Memory leaks are a real problem  Managed languages do not eliminate them Live Reachable Dead

 Memory leaks are a real problem  Managed languages do not eliminate them Live Reachable Dead

 Memory leaks are a real problem  Managed languages do not eliminate them  Slow & crash real programs Live Dead

 Memory leaks are a real problem  Managed languages do not eliminate them  Slow & crash real programs  Unacceptable for some applications  Fixing leaks is hard  Leaks take time to materialize  Failure far from cause

 Is it a memory leak or a case of stale memory ?  Is it correct to always consider stale memory as equivalent to unreachable memory ?

 Why tolerate leaks  Why leaks are so bad  How leak pruning works  How leak pruning predicts leaked objects  Evaluation

 Garbage collection based on liveness Live Reachable Dead

 Garbage collection based on liveness  Take action when memory exhausted Live Reachable Dead

 Garbage collection based on liveness  Take action when memory exhausted Live Reachable Dead

 Garbage collection based on liveness  Take action when memory exhausted Live Dead Throw OOM error Out of memory!

 Garbage collection based on liveness  Take action when memory exhausted Live Dead Out of memory! Throw OOM error Reclaim some objects

 Garbage collection based on liveness  Reclaim predicted dead objects Reclaimed Live

 Garbage collection based on liveness  Reclaim predicted dead objects Reclaimed Live A A B B

 Garbage collection based on liveness  Reclaim predicted dead objects Live A A ?

 Garbage collection based on liveness  Reclaim predicted dead objects  Poison references to reclaimed objects Live A A X

 Garbage collection based on liveness  Reclaim predicted dead objects  Poison references to reclaimed objects Live A A X Throw InternalError with OOMError attached

 Garbage collection based on liveness  Reclaim predicted dead objects  Poison references to reclaimed objects Live A A X Throw InternalError with OOMError attached Worst case: defers fatal errors Best case: keeps leaky programs running much longer or indefinitely

 Instead of returning the error can we return some other object, to keep program going ? Eg DieHard  What about semantics in that case ?

INACTI VE OBSER VE SELECTPRUNE Heap not full <50% Heap filled >50% Heap not full Heap full

 Why tolerate leaks  Why leaks are so bad  How leak pruning works  How leak pruning predicts leaked objects  Evaluation

 Predicting the future  Leaked objects  not used again  Highly stale objects  likely leaked  Alternative: offload to disk [Melt, Bond & McKinley ’08] [LeakSurvivor, Tang et al. ’08] [Panacea, Goldstein et al. ’07, Breitgand et al. ’07]  Tolerates mispredictions  Eventually exhausts disk (if disk at all)

 Predicting the future  Leaked objects  not used again  Highly stale objects  likely leaked  Alternative: offload to disk [Melt, Bond & McKinley ’08] [LeakSurvivor, Tang et al. ’08] [Panacea, Goldstein et al. ’07, Breitgand et al. ’07]  Tolerates mispredictions  Eventually exhausts disk (if disk at all) Need high precision One misprediction: program terminates Need high precision One misprediction: program terminates

 Identify references to prune  Roots of leaked data structures  Categorize by reference type ParserInfo PreparedStatement

 Identify references to prune  Roots of leaked data structures  Categorize by reference type  Criteria  Highly stale references ▪ More stale than most stale instance accessed previously ParserInfo PreparedStatement MaxS&U 2-4 GCs

 Identify references to prune  Roots of leaked data structures  Categorize by reference type  Criteria  Highly stale references ▪ More stale than most stale instance accessed previously  Reference type keeping most bytes reachable ParserInfo PreparedStatement MaxS&U 2-4 GCs Size 132MB

 Identify references to prune  Roots of leaked data structures  Categorize by reference type  Criteria  Highly stale references ▪ More stale than most stale instance accessed previously  Reference type keeping most bytes reachable Piggyback on GC: two-phase transitive closure ParserInfo PreparedStatement MaxS&U 2-4 GCs Size 132MB

 Based on colleague’s JDBC application  Leak: SQL statements remain in set

HashSet Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[]

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[]

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[]

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] enqueued HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] enqueued HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U enqueued

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] enqueued HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes enqueued

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] enqueued HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes enqueued

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] enqueued HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes 60 enqueued 0 0

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] enqueued HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes 572 enqueued 0 0

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes 572 enqueued 0 0

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes 572 enqueued 244 0

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes 420M 180M 0

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes 420M 180M 0

HashSet … …… Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] Parser Info Parser Info Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] byte[][] byte[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes 420M 180M 0

HashSet … …… Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes 420M 180M 0

HashSet … …… Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes 0 0 0

HashSet … …… Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes enqueued

HashSet … …… Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes 50M 220M 0

HashSet … …… Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes 50M 220M 0

HashSet … …… Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] Result Set Result Set Hash Entry Hash Entry Prepared Statement Prepared Statement Field[] HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes 50M 220M 0

HashSet … … Hash Entry Hash Entry Prepared Statement Prepared Statement Hash Entry Hash Entry Prepared Statement Prepared Statement Hash Entry Hash Entry Prepared Statement Prepared Statement HashEntry PreparedStatement ParserInfo PreparedStatement ResultSet MaxS&U Bytes 50M 220M 0

HashSet … … Hash Entry Hash Entry Prepared Statement Prepared Statement Hash Entry Hash Entry Prepared Statement Prepared Statement Hash Entry Hash Entry Prepared Statement Prepared Statement Algorithm efficiently selects references to consistently unused data structures based on type

 Why tolerate leaks  Why leaks are so bad  How leak pruning works  How leak pruning predicts leaked objects  Evaluation

 Leak pruning added to Jikes RVM  Performance stress test  Non-leaking programs: DaCapo & SPEC benchmarks  Low overhead ▪ 3% (Core 2) or 5% (Pentium 4) ▪ Primarily read barriers

Low footprint & often helps a lot  all-the-time deployed use Low footprint & often helps a lot  all-the-time deployed use

INACTI VE OBSER VE SELECTPRUNE Heap not full <50% Heap filled >50% Heap not full Heap full

 What about bursty leaks ?  Can we have dynamically adjustable limit for switching on Leak Pruning ?  Can we dynamically switch off Leak Pruning ?

 What if stale objects interleaved with active objects ?  Is it useful to find source of these leaks ? If yes then how ?