1. Qin Zhao (MIT) Derek Bruening (VMware) Saman Amarasinghe (MIT) Efficient Memory Shadowing for 64-bit Architectures ISMM 2010, Toronto, Canada June 6, 2010

2. Dynamic Program Analysis Understand Program Behavior –Optimization –Debugging –Security –Memory management Shadow Memory Tools –Maintain meta-data for every memory location –Update meta-data on every memory operation ISMM, Toronto, Canada, 6/6/2010 2

3. Examples Memory Error Detection –MemCheck [VEE’07] –Purify [USENIX’92] –Dr. Memory Dynamic Information Flow Tracking –LIFT [MICRO’39] –TaintTrace [ISCC’06] Multi-threaded Program Analysis –Eraser [TCS’97] –Helgrind Memory Usage Analysis –CETS [ISMM’10] –Staleness ISMM, Toronto, Canada, 6/6/2010 3

4. Shadow Memory System Shadow Memory Manager –Meta-data for application memory –Memory mapping scheme (addr A  addr S ) DMS (Direct Mapping) SMS (Segmented Mapping) Instrumentor –Every memory operation Address calculation Meta-data update –Expensive MemCheck (~25x) –~12x for addr A  addr S ISMM, Toronto, Canada, 6/6/2010 a.out stack libc Application Memory Shadow Memory heap 4

5. Direct Mapping Scheme (DMS) Single memory region for entire address space. Translation: Issue: address conflict between mem A and mem S ISMM, Toronto, Canada, 6/6/2010 lea [addr]  %r1 add %r1 disp  %r1 Slowdown relative to native execution Application Shadow 5

6. Slowdown relative to native execution Segmented Mapping Scheme (SMS) Shadow segment per application segment Translation: –Segment lookup (address indexing) –Address translation ISMM, Toronto, Canada, 6/6/2010 lea [addr]  %r1 mov %r1  %r2 shr %r2, 16  %r2 add %r1, disp[%r2]  %r1 addr A addr S App 1 Shd 1 Shd 2 App 2 Segment table 6

7. Kernel space Shadow Memory Mapping Scaling to 64-bit Architecture –DMS Infeasible due to memory layout ISMM, Toronto, Canada, 6/6/2010 a.out Unusable space stack User space vsyscall 2 47 2 64 7

8. Shadow Memory Mapping Scaling to 64-bit Architecture –DMS Infeasible due to memory layout –Single-Level SMS Too big (~4 billion entries) ISMM, Toronto, Canada, 6/6/2010 addr A 8

9. Shadow Memory Mapping Scaling to 64-bit Architecture –DMS Infeasible due to memory layout –Single-Level SMS Too big (~4 billion entries) –Multi-Level SMS Even more expensive ISMM, Toronto, Canada, 6/6/2010 Slowdown relative to native execution addr A 9

10. Umbra (CGO’10) Scaling to 64-bit Architecture –Single-Level SMS is too big but sparse Umbra (CGO’10) –Eliminate empty entries –Compact table –Walk the table to find the entry ISMM, Toronto, Canada, 6/6/2010 10

11. Slowdown relative to native execution Umbra (CGO’10) Reference Uni-Cache –Software cache per instr per thread Segment tag & displacement Check uni-cache before table walk 99.97% hit ratio ISMM, Toronto, Canada, 6/6/2010 11 tag = addr A & mask; if (cache  tag != tag) { … // table walk} addr S = addr A + cache  disp

12. EMS64: Key Idea Umbra EMS64 –Speculatively use a disp without check –Smart shadow memory placement Notified by memory access violation fault for incorrect displacement ISMM, Toronto, Canada, 6/6/2010 12 tag = addr A & mask; if (cache  tag != tag) { … // table walk (0.03%)} addr S = addr A + cache  disp

13. EMS64: Example A0 A2 S0 0: Application 2: Shadow 11: Application 12: Unavailable S2 10: Shadow 13: Unavailable 15: Unavailable 14: Unavailable 6: Shadow 7: Application A1 S1 Displacement: {-1, 2} ISMM, Toronto, Canada, 6/6/2010 13 9: Reserved 13: Unavailable/Reserved 15: Unavailable/Reserved

14. EMS64: Potential Problem A0 A2 S0 0: Application 2: Shadow 11: Application 12: Unavailable S2 10: Shadow 14: Unavailable 6: Shadow 7: Application A1 S1 Displacement: {-1, 2} ISMM, Toronto, Canada, 6/6/2010 14 9: Reserved 13: Unavailable/Reserved 15: Unavailable/Reserved

15. EMS64: Final Solution A0 A2 S0 0: Application 2: Shadow 11: Application 12: Unavailable S2 10: Shadow 13: Unavailable/Reserved 15: Unavailable/Reserved 14: Unavailable 6: Shadow 7: Application A1 S1 Displacement: {-1, 2} ISMM, Toronto, Canada, 6/6/2010 15 9: Reserved 4: Reserved 5: Reserved 1: Reserved 12: Unavailable/Reserved 8: Reserved

16. Slot Finding Problem Given n slots: –k Application slots –x Empty slots –y Reserved slots Find k S-slots. –For each slot A i, there is one associated slot S with displacement d i where d i = S i - A i. –For each slot A i and each existing displacement d j where d i ≠d j, slot ((A i + d j ) mod n) is an R-slot or an E-slot. –For each slot S and any existing valid displacement d i slot, slot ((S + d i ) mod n) is an R-slot or an E-slot. ISMM, Toronto, Canada, 6/6/2010 16 A0A0 A1A1 E0E0 E1E1 E2E2 E3E3 E4E4 R0R0 AiAi Application slot Shadow slot EiEi Empty slot RiRi Reserved slot SiSi S0S0 S1S1 R1R1 R2R2

17. Slot Finding Problem Given n slots: –k Application slots –x Empty slots –y Reserved slots Can We Find k S-slots? –Depends on layout! –Guarantee to find it, for 48-bit address space, if Application memory < 250 GB –Proof x ≥ 8k 2 +2k+1 We can always find an S i for A i if #E-slot > #conflicts ISMM, Toronto, Canada, 6/6/2010 17 AiAi Application slot Shadow slot EiEi Empty slot RiRi Reserved slot SiSi

18. Implementation & Optimization Implementation –Shadow memory allocation –Add signal handler –Remove reference uni-cache check Optimization –Restore uni-cache checks for instructions that access multiple segments, e.g., references from memcpy When number of access violation exceed 2 ISMM, Toronto, Canada, 6/6/2010 lea [addr]  %r1 add %r1, unicache  disp  %r1 18

19. Experimental Results Slowdown relative to native execution ISMM, Toronto, Canada, 6/6/2010 19

20. Thank You Download –http://people.csail.mit.edu/qin_zhao/umbra/http://people.csail.mit.edu/qin_zhao/umbra/ Q & A ISMM, Toronto, Canada, 6/6/2010 20

21. Slot Finding Example Can always find a solution –No AiAi Application slot SiSi Shadow slot EiEi Empty slot RiRi Reserved slot A0A0 A1A1 E0E0 E1E1 E2E2 E3E3 E4E4 R0R0 E-slotsS 0 (disp)ConflictS 1 (disp)Conflict E0E0 Х (1)E 0 + 1  A 1 X (7)E 0 + 7  A 0 E1E1 √ (3)√ (1) E2E2 X (4)E 2 + 4  A 0 X (2)A 0 + 2  A 1 E3E3 X (5)E 3 + 5  A 1 X (3)E 3 + 3  A 0 E4E4 X (6)A 1 + 6  A 0 X (4)E 4 + 4  A 1 ISMM, Toronto, Canada, 6/6/2010 21