1. The Superdiversifier: Peephole Individualization for Software Protection Mariusz H. Jakubowski Prasad Naldurg Chit Wei (Nick) Saw Ramarathnam Venkatesan Microsoft Research Matthias Jacob Nokia International Workshop on Security: IWSEC ’08 Kagawa, Japan November 25-27, 2008

2. 11/26/082 Introduction Software individualization –“Different-looking” but functionally equivalent code –Diversity as a defense against attacks –Important role in both biological and man-made systems Superoptimization –Brute-force search for shortest code sequences that implement a given function –Compiler optimization introduced by Massalin ‘87 Goals of our work: –Leverage and extend superoptimization to individualize instruction sequences –Study superdiversification in the context of more comprehensive protecton frameworks

3. 11/26/083 What Does This Do? unsigned __int64 nInput = _atoi64(argv[1]); __int64 n; n = nInput - ((nInput >> 1) & 033333333333333333333LL); n = n - ((nInput >> 2) & 011111111111111111111LL); n = n + (n >> 3); n = n & 07070707070707070707LL; n = n % 077; printf("%d\n", n);

4. 11/26/084 Overview Introduction Background –Individualization –Superoptimization Superdiversification Experimental results Applications Conclusion Instruction-level diversity via guided search

5. 11/26/085 Software Individualization Element of software security –Defends against BORE attacks (Break Once/Run Everywhere) –Forces duplication of effort to break systems –Alleviates “software monoculture” problem Many practical uses: –ASLR (Address Space Layout Randomization) –Secure DRM clients –Self-mutating malware –…

6. 11/26/086 Individualization Schemes Static: Individualization of program code –Algorithmic Bubble sort  quicksort Red-black trees  splay trees –Syntactic MOV EAX,0  XOR EAX,EAX MOV EAX,5; MOV EBX,1  MOV EBX,1; MOV EAX,5 Dynamic: Individualization of runtime behavior –Varying paths at runtime –Variable data encoding –Self-modifying code –Byte-codes with variable semantics –…

7. 11/26/087 Superoptimization Brute-force search for shortest equivalent instruction sequence [Massalin ‘87]: –“Startling programs have been generated, many of them engaging in convoluted bit fiddling bearing little resemblance to the source programs which defined the functions.” –“… like a typical superoptimized program, the logic is really convoluted.”

8. 11/26/088 Superoptimization Input: Instruction sequence implementing a function Algorithm outline: –Enumerate all possible sequences up to a given length (e.g., 10 instructions). –Check for equivalence to input sequence: Quick test: Test candidate sequence on several random inputs. Slow test: Check Boolean equivalence of sequences (if quick test passes). –Skip sequences longer than current shortest sequence. Quick test takes most of the computation time. Slow test guarantees equivalence to input sequence.

9. 11/26/089 Overview Introduction Background –Individualization –Superoptimization Superdiversification Experimental results Applications Conclusion Instruction-level diversity via guided search

10. 11/26/0810 The Superdiversifier Adapt and extend superoptimization to diversify code: –Restrict set of instructions and operands allowed in search. –Guide search based on instruction frequencies occurring in real-life programs. –Use pruning techniques to cut down search time. –Accept a secret key to control the above operations. Output any equivalent sequences, not necessarily only the shortest. –Secret key determines order of search. –Different keys may yield dramatically different equivalent sequences.

11. 11/26/0811 Equivalence Test Using a SAT Solver Input: Two Boolean functions, F(x) and G(x). Goal: Determine whether F(x) ≡ G(x). F(x) ≡ G(x) iff  x, F(x) = G(x). F(x) ≡ G(x) iff  x│F(x) ≠ G(x). Thus, simply run a SAT solver on F(x) ≠ G(x) represented as a Boolean (CNF) formula. F(x) ≡ G(x) iff F(x) ≠ G(x) is unsatisfiable.

12. 11/26/0812 Overview Introduction Background –Individualization –Superoptimization Superdiversification Experimental results Applications Conclusion Instruction-level diversity via guided search

13. 11/26/0813 Experimental Results Function: Swap registers Input code Sample equivalent versions

14. 11/26/0814 Experimental Results Function: Swap registers Input code Sample equivalent versions Only arithmetic and logical instructions allowed in search.

15. 11/26/0815 Experimental Results Function: Fragment of compiler-generated code Input code Sample equivalent versions Small set of constants allowed in search (may be harvested from real-life programs).

16. 11/26/0816 Empirical Taxonomy

17. 11/26/0817 Overview Introduction Background –Individualization –Superoptimization Superdiversification Experimental results Applications Conclusion Instruction-level diversity via guided search

18. 11/26/0818 Some Applications Defense against signature-based attacks Patch obfuscation –Patches reveal location of vulnerabilities. “Patch Tuesdays” often followed by exploits. Diffing tools locate vulnerable code quickly. –Superdiversification helps to hide patches. Maximize size of diff between unpatched and patched applications. For best results, diversify large sections of the patched binary, not just the patch code. An element of comprehensive individualization systems

19. 11/26/0819 Conclusion Main contribution: Guided search for instruction sequences to individualize binaries. Future work –Extend range of superdiversified code. Other types of instructions Control-flow constructs –Optimize for better speed. –Adapt to custom byte-codes. Modern instructions sets are geared towards generality and performance. Custom byte-codes may be designed for individualization and obfuscation. Instructions may perform arbitrary operations, not just serve as elementary building blocks.