1. Preventing Reverse Engineering by Obfuscating Bharath Kumar

2. Reverse Engineering  Process of backtracking through the software process  Obtaining source code from binary/ byte code.  Understanding programs to realize intent.  Intellectual property issues.

3. Example public class TreeEnum { Vector treesBasedOnNumber; private Vector getTrees(int numberOfNodes) { if (numberOfNodes == 0) return null; if (numberOfNodes <= treesBasedOnNumber.size()) { // System.out.println("Trying for " + (numberOfNodes - 1) + " with " + treesBasedOnNumber.size()); Object o = treesBasedOnNumber.get(numberOfNodes - 1); if (o instanceof Vector) return (Vector)o; else return null; } return null; }

4. Reverse engineered! public synchronized class TreeEnum { Vector treesBasedOnNumber; private Vector getTrees(int i) { if (i == 0) return null; if (i > treesBasedOnNumber.size()) return null; Object object = treesBasedOnNumber.get(i - 1); if (object instanceof Vector) return (Vector)object; else return null; }

5. Approaches against reverse engineering  Legal battles  Service based software  Thin mobile code  Code encryption  Distributing binaries  Obfuscation

6. Obfuscation  Obfuscate – “to confuse”  Alter code so as to confuse reverse engineer, but preserve functionality  Behavior preserving transformations on code that preserve function but reduce readability or understandability  How do we confuse the reader?

7. Software metrics  Program length  Complexity of program increases with the number of operators and operands in P.  Cyclomatic complexity  Complexity increases with the number of predicates in a function.  Nesting complexity  Complexity increases with the number of nesting level of conditionals in a program.  Data flow complexity  Complexity increases with the number of inter- block variable references.

8. Software metrics…  Fan-in/fan-out complexity  Complexity increases with the number of formal parameters to a function, and with the number of global data structures read or updated in the function.  Data structure complexity  Complexity increases with the complexity of the static data structures in the program. Variables, Vectors, Records.  OO Metrics  Complexity increases with  Level of inheritance  Coupling  Number of methods triggered by another method  Non-cohesiveness

9. A classification of obfuscations  Layout transformations  Change formatting information  Control transformations  Alter program control and computation  Aggregation transformations  Refactor program using aggregation methods  Data transformations  Storage and encoding information

10. Some metrics for obfuscations  Assume complexity of a program be E(P) (based on metrics)  Potency of a transformation is the level of complexity it introduces.  E(P`)/E(P) – 1  Resilience of a transformation measures how well it can deal with a deobfuscation ‘attack’  On a scale of trivial to one-way  Execution cost  Free, cheap, costly, dear  Quality of an obfuscation  A combination of potency, resilience, and execution cost

11. Control transformations  Opaque predicates  S1; S2;  S1; if (Pred) S1; S2; if (Pred) S2;  Opaque constructs – always evaluate one way (known to obfuscator), unknown to deobfuscator.  Trivial and weak opaque constructs.

12. Control transformations  Insert dead or irrelevant code  Extend loop conditions  Convert a reducible to a non-reducible flow graph  Redundant operands  Parallelize code  Replacing standard library routines by custom routines

13. Aggregation transformations  Inline and outline methods  Interleave methods  Clone methods  Loop transformations  Loop blocking  Loop unrolling  Loop fission  Ordering transformations

14. Data transformations  Change encoding  Pack variables into bigger variables  Pack variables into arrays  Convert static to procedural data  Restructure arrays  Altering inheritance hierarchies

15. Opaque constructs  The pointer aliasing problem  Shown to be NP-hard or even undecidable  Dynamic structures for producing opaque constructs.  Opaque constructs using threads.

16. Deobfuscation  Almost all obfuscating transforms have a deobfuscating transform  Essentially boils down to evaluating opaque constructs  Program slicing  Pattern matching  Statistical analysis  Data flow analysis  Theorem proving