1. 1 Data layouts for object-oriented programs Martin Hirzel IBM Research SIGMETRICS 6/16/2007

2. 2 Problem Object-oriented programs put data in objects. Caches and TLBs put data in blocks. Scattering objects over blocks causes cache/TLB misses. Misses cost time. o2o2 o1o1 o3o3 o4o4 o 11 o9o9 o 10 o5o5 o8o8 o6o6 o7o7 cache line TLB page

3. 3 Solution Most object-oriented languages use garbage collection. Garbage collection can move objects. To avoid misses, move objects to the right cache/TLB blocks. Simple, right? o2o2 o1o1 o3o3 o4o4 o 11 o9o9 o 10 o5o5 o8o8 o6o6 o7o7 o 11 o9o9 o 10 o2o2 o1o1 o3o3 o4o4 o5o5 o8o8 o6o6 o7o7

4. 4 Cheney Copying GC scan free To-space Copied & not yet scanned Copied & scanned free scan free scan From-space scan=free o9o9 o 10 o2o2 o1o1 o3o3 o4o4 o5o5 o8o8 o6o6 o7o7

5. 5 BF: Breadth-first layout 1 23 4567 89101112131415 Why?Siblings How?Queue-based traversal Who?Cheney 1970

6. 6 DF: Depth-first layout Why?Child-parent How?Stack-based traversal Who?Fenichel/Yochelson 1969 1 23 4567 89101112131415

7. 7 HI: Hierarchical layout 1 23 4567 89101112131415 Why?Both siblings and child-parent How?Block-bounded breadth first Who?Moon 1984; Wilson et al. 1991

8. 8 AO: Allocation order layout 12d34d5d678d9 1c3c6c7c9c Why?Creation order matches usage How?Sliding compaction Who?Lisp 2 collector 1967

9. 9 SZ: Size segregation layout 123d45d7698d 1c6c2c7c9c4c Why?Efficiently finding allocation holes How?Segregated free lists Who?Comfort 1964

10. 10 TH: Thread local layout 1 2 3 8 11 12 9 10 5 4 6 7 001010 111 110 100 101 Why?Disjoint working sets How?Reachability from call stacks Who?Steensgaard 2000

11. 11 Problem AOAllocation order ASAllocation site BFBreadth-first DFDepth-first HIHierarchical POPopularity RARandom SZSize THThread TYType Which layout is best, and which is worst? How much does it matter in practice? How similar are the layouts? How much does it matter in the limit?

12. 12 Solutions? Appeal to intuition –They can’t all be right! Formal –Petrank/Rawitz showed hardness Simulation –Who would believe those numbers? Brute-force –Do you have a few person-years to spare?

13. 13 Avoiding Heisenberg Effects Time Garbage collector Application Garbage collector Application Garbage collector Exclude garbage collector performance Measure real effect of layout on application Implement the layouts with simple algorithms

14. 14 Object sorting garbage collection populate sortcopyfixup Sort keys: AO, AS, PO, RA, SZ, TH, TY

15. 15 32 Benchmarks

16. 16 % Mutator time overhead Performance impact increases with SMP Conclusions for AO, DF, TH, RA still hold All layouts sometimes best, sometimes worst RA is worst, as expected Low averages, but beware of worst cases! AO has best average (but not by much) DF has most best cases TH has most benign worst case

17. 17 % Mutator miss rate increases Layouts have large impact on miss rates Miss rates confirm overhead conclusions Miss rates can not replace time measurements

18. 18 Layout similarities and differences AO, PO, and TH are quite similar As expected, RA is far out

19. 19 Estimated limit mutator time Benchmarks: largest avg. overhead Baseline: best observed Linear regression: Limit time (no misses) + cache misses  cache latency + TLB misses  TLB latency = Total time (as measured)

20. 20 Related work

21. 21 Conclusions Layouts matter little on average, but: –Beware of the worst cases! –Layout importance increases with SMP All layouts are sometimes best, sometimes worst –AO has best average –DF has most best-cases –TH has best worst-cases