Runtime Protection via Dataflow Flattening Bertrand Anckaert Ghent University/ Boston Consulting Group The Third International Conference on Emerging Security Information, Systems and Technologies SECURWARE 2009 June 18-23, 2009 – Athens/Glyfada, Greece Mariusz H. Jakubowski Ramarathnam Venkatesan Chit Wei (Nick) Saw Microsoft Research Redmond, WA (USA)
SECURWARE 2009June 18-23, Introduction Software protection –Complicate reverse engineering and tampering. –Enforce execution as intended by developer. –DRM, licensing, anti-malware, OS security, etc. Dataflow analysis –Track flow of data through program. –Locate and tamper “interesting” data. Goals of our work: –Develop methods against malicious dataflow analysis. –Study dataflow flattening as an element of comprehensive protection frameworks.
SECURWARE 2009June 18-23, Overview Introduction Background Dataflow flattening Implementation and experiments Applications Conclusion Protecting data operations
SECURWARE 2009June 18-23, Background Oblivious RAMs [Goldreich and Ostrovsky ’96] –Randomized memory-access patterns –Each fetch/store replaced by many fetch/stores –Cannot infer program operation from memory accesses Control-flow flattening –Program’s CFG converted to flat (two-level) graph –Cannot infer control-flow structure from execution on the flat graph
SECURWARE 2009June 18-23, Overview Introduction Background Dataflow flattening Implementation and experiments Applications Conclusion Protecting data operations
SECURWARE 2009June 18-23, Dataflow Flattening Two main aspects: –Making dataflow graph appear complete –Randomizing memory-access patterns Informally: –“Every variable affects every other variable.” –“Program accesses memory at random.”
SECURWARE 2009June 18-23, Flattening Dataflow Graphs Basic idea: Make dataflow graph appear complete. –Every variable affects all other variables. –Cannot infer useful variable dependencies. Data-centric analog of control-flow flattening AB X Y
SECURWARE 2009June 18-23, Flattening Dataflow Graphs Basic idea: Make dataflow graph appear complete. –Every variable affects all other variables. –Cannot infer useful variable dependencies. Data-centric analog of control-flow flattening AB X Y
Heap Memory Management Unit Program Dataflow Flattening via an MMU
SECURWARE 2009June 18-23, MMU Software-based Memory Management Unit: –Periodic reordering of heap data –Migration of variables from stack to heap –Pointer masking Variable references redirected through MMU
SECURWARE 2009June 18-23, MMU Operation Heap subdivided into encrypted pages (e.g, 4KB). Upon each access of a heap page: –Retrieve n extra pages with probability 1/p. –Randomly shuffle the (expected) 1+n/p pages. –Re-salt and re-encrypt each page.
SECURWARE 2009June 18-23, Security Analysis Security analyzed via practical metrics Confusion factor C as a metric –Define C as the number of possible places for a page in memory. –Let N = total number of memory pages. –Oblivious RAMs: C = N after each memory access. –Our approach: C converges to N as accesses occur.
SECURWARE 2009June 18-23, Practical Issues Most program variables are stack-based. –Solution: Migrate variables from stack to heap. –Explicitly allocate and free heap memory when entering and exiting stack frames. Pointers can reveal access patterns. –Solution: Scramble pointers. –Only MMU knows mapping between addresses and variables.
SECURWARE 2009June 18-23, Overview Introduction Background Dataflow flattening Implementation and experiments Applications Conclusion Protecting data operations
SECURWARE 2009June 18-23, Implementation Tool for transforming C programs –Based on Phoenix compiler infrastructure –Compiler backend plug-in –Instrumentation of Phoenix IR Transformations –Interception and custom implementation of heap operations (malloc, free, etc.) –Conversion of stack variables to heap variables –Pointer scrambling (encryption)
SECURWARE 2009June 18-23, Experimental Results Selected SPEC benchmarks (compression, optimization, QCD, chess, fluid dynamics ) Simple algorithms (pseudorandom-number generation, summing a list of integers) Performance impact (slowdown)
SECURWARE 2009June 18-23, Overview Introduction Background Dataflow flattening Implementation and experiments Applications Conclusion Protecting data operations
SECURWARE 2009June 18-23, Applications Software protection –Anti-malware systems –Licensing, DRM, product activation, etc. –Defenses against information-extraction and side- channel attacks More comprehensive tools –Element of broader protection strategies –Means of realizing “engineering assumptions” needed by some security models
SECURWARE 2009June 18-23, Conclusion Dataflow flattening –Makes dataflow graph appear complete. –Randomizes memory-access patterns. –Complicates inference of algorithms from their data operations. Future directions –Dataflow flattening as part of more comprehensive systems –Security analysis via models and metrics