Program Obfuscation: A Quantitative Approach Presented by: Mariusz Jakubowski Microsoft Research Third Workshop on Quality of Protection October 29 th, 2007 Bertrand Anckaert, Matias Madou, Bjorn De Sutter, Bruno De Bus, Koen De Bosschere, and Bart Preneel Ghent University and K.U.Leuven, Belgium
Obfuscation has many applications 2
There is a large gap between theoretical results 3 - On the (Im)possibility of Obfuscating Programs – Barak et al. (2001) - On the Impossibility of Obfuscation with Auxiliary Input – Goldwasser et al. (2005) - Positive Results and Techniques for Obfuscation – Lynn et al. (2004) - Towards Realizing Random Oracles: Hash Functions that Hide All Partial Information Canetti et al. (1997) + - Large gap Intuitively, obfuscation does help
We need a practical system for evaluating obfuscating transformations It should be easy to evaluate existing and future transformations => Automated The evaluation should convey difficulty of reverse-engineering => Build upon experience from complexity metrics 4
oIntro oMetrics oInstruction Count oCyclomatic Number oKnot Count o(De)Obfuscating transformations Outline 5
Four axes based on typical reverse- engineering scenario 6 Disassemble Flow graph construction Analyse Data Flow Interpret Data Code Control flow Data flow Data
+ No uncertainty about executed code + Always availabe - Only about covered part of the code Evaluated Complexity Metrics 7 Code Control flow Data flow Data Instruction Count Cyclomatic Number Knot Count Metrics are collected by a run-time instrumentation framework
Cyclomatic number and knot count Cyclomatic number: – #edges – #nodes + 2 – Intuitively: the number of decision points Knot count: – #crossings – Intuitively: the unstructuredness 8
oIntro oMetrics o (De)Obfuscating transformations oJump redirection [Linn et al. 2003] oControl flow flattening [Chenxi Wang et al. 2001] oOpaque predicates [Collberg et al. 1998] Outline 9
Jump redirection Redirect branches to function 10 1 Jmp call branch Branch Function 2 garbage assumed return site
Impact of Jump Redirection 11
Jump redirection - deobfuscation Identify Branch Function – signature based – run-time behavior Record (call,return) pairs under debugger Overwrite calls 12 1 call branch Branch Function 2 garbage assumed return site (1,2) (4,7) (9,5) … jmp 2
Success of De-obfuscation 13
Control flow flattening All original basic blocks have the same predecessor and successor switch
Control flow flattening significantly increases the complexity metrics 15
Success of De-obfuscation 16
Opaque predicates 17 1 Jmp Jmp if (2==2) 2fake Add fake decision statements
Impact of Opaque Predication 18
Conclusion A first step towards a unified quantitative evaluation of – obfuscating transformations – deobfuscating transformations Which leverages experience from the established field of complexity metrics 19
Program Obfuscation: A Quantitative Approach Presented by: Mariusz Jakubowski Microsoft Research Third Workshop on Quality of Protection October 29 th, 2007 Bertrand Anckaert, Matias Madou, Bjorn De Sutter, Bruno De Bus, Koen De Bosschere, and Bart Preneel Ghent University and K.U.Leuven, Belgium