1. Valgrind, the anti-Alzheimer pill for your memory problems
Philippe Waroquiers
FOSDEM 2017 valgrind devroom

2. Talk content
Discuss/demo a new functionality, that provides an easier way to visualise memory usage and other types of data.
If time permits:
Discuss/demo memory pool and leak heuristics

3. How much memory, allocated where ?
Massif : records the evolution of the memory
By taking snapshots regularly
Some snapshots are detailed, showing the allocation stack traces
The peak snapshot is also detailed
In 3.13 (SVN), new ' tree view' memory feature xtree-memory=none|allocs|full to produce a memory report at the end of execution
Can also be produced on demand, using vgdb

4. Demo : Who allocated how much memory ?
valgrind --tool=massif --xtree-memory=full \ soffice.bin --invisible \ convert-to pdf fosdem2017_optim_xt_hg.odp
# If specifying a .ms extension, produces a Massif format: valgrind --tool=massif --xtree-memory=full \ xtree-memory-file=xtmemory.ms.%p \ mfg # Default xtmemory.kcg.%p produces a callgrind format: valgrind --tool=massif --xtree-memory=full \ mfg

5. Demo : ms_print output

6. Demo : massif-visualizer

7. Demo : kcachegrind xtree memory full

8. Demo : massif-visualize xtree memory full

9. Massif format and --xtree-memory=full
valgrind --tool=massif --xtree-memory=full \ xtree-memory-file=xtmemory.ms.%p \ soffice.bin --invisible \ convert-to pdf fosdem2017_optim_xt_hg.odp
Massif format :
Not designed for huge set of data
Not designed for more than 1 'data kind'
=> the above generates a 6.3 Gb file
=> very (too?) heavy e.g. for massif-visualizer
=> for xtree, kcachegrind format is better

10. Xtree and memcheck/helgrind
Like with Massif, --xtree-memory=allocs|full will produce a report at the end of execution
To produce report during execution: vgdb xtmemory <filename>
Supported by massif/memcheck/helgrind.
Can be done even when --xtree-memory=none => it will produce a “=allocs report”

11. How to best visualise memory ?
massif+ms_print : not that easy for big apps
massif+massif-visualizer : nice evolution graph, digging in stack traces not that easy
massif-visualizer+--xtree-memory-file=full : not really appropriate
kcachegrind+--xtree-memory-file=full : easy to use, but file can become huge
Maybe kcachegrind and callgrind format should be expanded to present evolution of data ?

12. Xtree and memcheck leak
valgrind --tool=memcheck --xtree-leak=yes \ soffice.bin --invisible \ convert-to pdf fosdem2017_optim_xt_hg.odp
--xtree-leak=yes : final leak report as xtree
report is automatically a “full” leak report
report file name controlled by xtree-leak-file=xtleak.kcg.%p
vgdb monitor command 'leak_check' can also produce xtree leak reports
Note: only the callgrind kcg format is supported for xtree leak reports

13. Xtree for syscalls experimental, not (yet?) in SVN
valgrind --tool=massif --xtree-syscall=yes \ soffice.bin --invisible \ convert-to pdf fosdem2017_optim_xt_hg.odp
--xtree-syscall=yes currently keeps track of
Nr of syscalls
Nr of failed syscalls
Syscall time (micro seconds)
Bytes read
Bytes written

14. kcachegrind xtree syscalls

15. kcachegrind cycles Cycles in a callgraph are created by
Recursive calls, e.g. fnA calls fnB that calls fnA
“Superposition” of 2 non recursive stack traces
main calls fnA that calls fnB
main calls fnB that calls fnA
Cycles and kcachegrind visualisation
Inclusive costs cannot be added
kcachegrind represents all the functions of a cycle by a “Cycle x” function
Complex cycles not easy to analyse/grasp

16. Xtree and kcachegrind cycles
Xtree stacktraces :
Code addresses are translated to function names
Function names are displayed by kcachegrind
What if there is no symbol for a code address ?
Currently, xtree uses a “UnknownFn???” Fn name
Consequence: many (artificial) cycles
=> some more ideas and work needed here
Maybe heuristic to guess a function start address ?
Maybe use callgrind-like callstack follow-up logic ?

17. Xtree is work in progress ...
In 3.13 SVN, xtree provides already an easy way to visualise memory usage and leaks
Some further work :
Xtree for system calls
Better handling of unknown function names
Optionally keeping the leaf function (malloc/free/...)
Allow to visualise evolution of data
Kcachegrind improvements : virtual top function, show exact callstack, ...
… Your ideas/suggestions here …

18. Questions about xtree?

19. Valgrind and address info
Address information based on
Knowledge of malloc/free calls
Debug info of variables, when using read-var-info=yes
Mempool information
...
Valgrind can give information about an address
Complementary to (sometimes better) than gdb

20. mempool (most) valgrind tools need to understand memory allocation
=> such tools are replacing malloc/free
Some applications have specific allocators
E.g. for small blocks fast allocation + release all blocks in one operation
How to “valgrind” these specific allocators ?
Either a compile time option to use malloc/free
Or describe these specific allocators using 'client requests' (documented in User manual and valgrind.h)

21. Memcheck leak search heuristics
“Possibly lost” : block only found via “inner pointer” (pointing inside the block)
Such inner pointers is a common pattern
e.g. in C++ library implementation
Memcheck has heuristics to detect some patterns :
C++ stdstring, C++ multiple inheritance, C++ array of objects
length64 (sqlite)
Note : these heuristics depends on the ABI

22. Questions?