1. Diagnosing Unbounded Heap Growth in C++ Problem Description Types of Unbounded Heap Growth –Reference Lost (Leak) Reference lost to memory without freeing it Well studied, wide variety of tools that identify leaks –Reference Retained But no longer needed On shutdown, memory freed, so not reported as leak If growing, it’s accumulating in a data structure Few tools exist to deal with this problem in C++ –Valgrind, IBM Rational Purify, Insure++ do not detect this Generally also referred to as a leak 1

2. Diagnosing Unbounded Heap Growth in C++ Problem Description Memory Tumor –Data structure with unbounded growth –Examples: Storing unlimited history info in memory Incorrect removal functions Simplest example: 2 void main() { std::vector tumorV; while( inputKey != ESC ) tumorV.push_back(1); }

3. Diagnosing Unbounded Heap Growth in C++ Detection Approach 3 Aspects of Tumor Detection –Container tracking Hold references to all data structures in the system –Growth tracking Track size changes per data structure over time Report those with unbounded growth –Automated Test Created by user to exercise all code paths 3

4. Diagnosing Unbounded Heap Growth in C++ Review Container tracking –CAT (Central Aggregate Tracker) Maintains references to all aggregates in the system –Create wrappers for each aggregate type in system Templated constructors, multiple inheritance Add to CAT on construction, remove on destruction –Namespace replacement to enable wrappers Find and replace to apply new namespace Wrappers disabled with compile time flag Example: trak::std::vector 4

5. Diagnosing Unbounded Heap Growth in C++ Detection Approach Growth Tracking –Take periodic samples of the CAT –Exponentially increasing interval sizes Reduces false positives & negatives over time –Report growing aggregates at each sample 5

6. Diagnosing Unbounded Heap Growth in C++ Testing Automated Test –Detecting growth requires a test that exhibits the growth Good complete test design is important –Need cyclic tests that cover all code paths May require multiple tests –Slow growing tumors may grow quickly with different usage patterns Eg. WebKit MiniBrowser pop-up windows 6

7. Diagnosing Unbounded Heap Growth in C++ Detection Approach Growth Tracking –Heuristic Take periodic samples of the CAT Two Interval Analysis –1 st interval establishes aggregate age, gives time to stabilize –2 nd interval proves stability, non-tumors shouldn’t grow –2 nd interval becomes the 1 st for next more accurate test 7

8. Diagnosing Unbounded Heap Growth in C++ Detection Approach Growth Tracking –Two interval analysis 8 time memory 1 23 4

9. Diagnosing Unbounded Heap Growth in C++ Detection Approach Growth Tracking –Two interval analysis 9 time memory 1 23 4 Reported as tumor (false positive) Not reported (growth stabilized)

10. Diagnosing Unbounded Heap Growth in C++ Detection Approach Growth Tracking –Heuristic Take periodic samples of the CAT Two Interval Analysis –1 st interval establishes aggregate age, gives time to stabilize –2 nd interval proves stability, non-tumors shouldn’t grow –2 nd interval becomes the 1 st for next more accurate test Exponentially increasing interval sizes –Reduces false positives & negatives over time 10

11. Diagnosing Unbounded Heap Growth in C++ Detection Approach Growth Tracking –Exponentially increasing interval size 11 time memory 1 23 4 In this example: constant intervals would not report growth half the time

12. Diagnosing Unbounded Heap Growth in C++ Detection Approach Growth Tracking –Heuristic Take periodic samples of the CAT Two Interval Analysis –1 st interval establishes aggregate age, gives time to stabilize –2 nd interval proves stability, non-tumors shouldn’t grow –2 nd interval becomes first for next more accurate test Exponentially increasing interval sizes –Reduces false positives & negatives over time Monitor size maximums –Reduces size fluctuation false positives 12

13. Diagnosing Unbounded Heap Growth in C++ Detection Approach Growth Tracking –Max size variable 13 1 23 time memory ceiling 4 Growth would be reported without max size

14. Diagnosing Unbounded Heap Growth in C++ Detection Approach Growth Tracking –Heuristic Take periodic samples of the CAT Two Interval Analysis –1 st interval establishes aggregate age, gives time to stabilize –2 nd interval proves stability, non-tumors shouldn’t grow –2 nd interval becomes the 1 st for next more accurate test Exponentially increasing interval sizes –Reduces false positives & negatives over time Monitor size maximums –Reduces size fluctuation false positives At each interval report all aggregates that: –Increased their size maximum –Have existed for two full intervals Sort results by size & reporting frequency to prioritize investigation 14