1. jFuzz – Java based Whitebox Fuzzing
David Harvison
Adam Kiezun

2. Summary
Problem
Generating interesting test inputs for file reading programs takes time.
Approach
Create a smart fuzzer to generate inputs that cause programs to crash.
Results
jFuzz generates a many input files and creates a base for others to expand upon.

3. Problem Bugs in a program may cause crashes for specific input files.
A compiler with buggy code, a media player with a corrupt file, etc.
Creating input files by hand takes time.
Some of the files may exercise the same code.
Want a way to automatically generate input files that crash the program.

4. Idea
Generate inputs that cause crashes by generating many inputs that exercise unique execution paths.

5. Program Example File reading code. if (car0 == '-') { neg = true;
} else {
neg = false;
cnt++; }
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
val = car1;

6. Program Example File reading code.
Want to generate files which exercise different program paths.
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++; }
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
val = car1;

7. Program Example File reading code.
Want to generate files which exercise different program paths.
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++; }
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
val = car1;
car0 == '-'
car1 == '5'

8. Program Example File reading code.
Want to generate files which exercise different program paths.
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++; }
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
val = car1;
car0 == '-'
car1 == '7'

9. Program Example File reading code.
Want to generate files which exercise different program paths.
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++; }
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
val = car1;
car0 == '+'
car1 == '3'

10. Program Example File reading code.
Want to generate files which exercise different program paths.
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++; }
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
val = car1;
car0 == '+'
car1 == '9'

11. Related Tools Cute, EXE, SAGE, catchconv, Apollo JCute
Smart fuzzers - programs that generate interesting new inputs for programs.
Not for Java.
JCute
Smart fuzzer for Java.
Reinstruments code – Requires source files.
Has problems with the JDK.

12. Overall Idea Input Output Run the subject program in a modified JVM.
A compiled (into bytecode) Java program.
A valid input file.
Output
New input files which exercise unique control paths.
Run the subject program in a modified JVM.
A logic predicate, the Path Condition, is formed as the program executes.
Describes control flow of execution.
New inputs are created by manipulating the path condition.

13. Example good public void top(char[] input) { int cnt = 0;
if (input[0] == ‘b’) cnt++;
if (input[1] == ‘a’) cnt++;
if (input[2] == ‘d’) cnt++;
if (input[3] == ‘!’) cnt++;
if (cnt > 3) crash(); }
good

14. Example good Negate constraints in path condition.
public void top(char[] input) {
int cnt = 0;
if (input[0] == ‘b’) cnt++;
if (input[1] == ‘a’) cnt++;
if (input[2] == ‘d’) cnt++;
if (input[3] == ‘!’) cnt++;
if (cnt > 3) crash(); }
I0 != 'b'
I1 != 'a'
I2 != 'd'
I3 != '!'
path condition
good
Negate constraints in path condition.
Solve the new path condition to create new inputs.

15. Example goo! good public void top(char[] input) { int cnt = 0;
if (input[0] == ‘b’) cnt++;
if (input[1] == ‘a’) cnt++;
if (input[2] == ‘d’) cnt++;
if (input[3] == ‘!’) cnt++;
if (cnt > 3) crash(); }
I0 != 'b'
I1 != 'a'
I2 != 'd'
I3 == '!'
good
goo!

16. Example godd good public void top(char[] input) { int cnt = 0;
if (input[0] == ‘b’) cnt++;
if (input[1] == ‘a’) cnt++;
if (input[2] == ‘d’) cnt++;
if (input[3] == ‘!’) cnt++;
if (cnt > 3) crash(); }
I0 != 'b'
I1 != 'a'
I2 == 'd'
godd
good

17. All paths are explored systematically.
Example
bood
public void top(char[] input) {
int cnt = 0;
if (input[0] == ‘b’) cnt++;
if (input[1] == ‘a’) cnt++;
if (input[2] == ‘d’) cnt++;
if (input[3] == ‘!’) cnt++;
if (cnt > 3) crash(); }
gaod
godd
good
goo!
All paths are explored systematically.

18. Tool Used NASA Java PathFinder
Dynamic analysis framework for Java implemented as a JVM.
Allows backtracking including saving and restoring the whole state of the VM.
Can execute all thread interleavings.
Can execute a program on all possible inputs.

19. Attributes Additional state-stored information.
Associated with runtime values.
JPF propagates attributes across assignment, method calls, etc.
Allows us to keep track of how the variables relate to the input using symbolic expressions.

20. Concolic execution is both concrete and symbolic.
Concrete v. Concolic
Normal
With Attributes
exp0
exp1 3 4 3 4 + + 7
Sum
(exp0, exp1) 7
Concrete
Concolic execution is both concrete and symbolic.
Symbolic

21. Concrete v. Concolic Concrete Symbolic
public class IADD extends Instruction {
public Instruction execute ( ThreadInfo th) {
[1] int v1 = th.pop();
[2] int v2 = th.pop();
[3] th.push(v1 + v2, ...);
[4] return getNext(th); }
public class IADD extends ...bytecode.IADD {
public Instruction execute ( ThreadInfo th) {
[1] int v1 = th.pop();
[2] int v2 = th.pop();
[3] th.push(v1 + v2, ...);
[4] StackFrame sf = th.getTopFrame();
[5] IntExpr sym_v1 = sf.getOperandAttr();
[6] IntExpr sym_v2 = sf.getOperandAttr();
[7] if (sym_v1 == null && sym_v2 == null)
return getNext(th);
[7] IntExpr result = sym_v1._plus(sym_v2);
[8] sf.setOperandAttr(result);
[9] return getNext(th); }
Concrete
Symbolic

22. jFuzz Architecture
Subject
and
Input
Runs JPF many times on the subject program and input files.
Each run:
Collects the Path Condition (PC).
Negates each constraint, reduces, and solves.
Uses new PCs to generate new input files.
Keeps track of inputs which caused exceptions to be thrown.
jFuzz
JPF
Subject
and
Original
Input PC
Negated PC
Solver
New
Input
Inputs
which
cause
crashes

23. Creating New Inputs
For a given execution some parts of the input may not be read.

24. Creating New Inputs
For a given execution some parts of the input may not be read.

25. Creating New Inputs
For a given execution some parts of the input may not be read.
When the path condition is solved, only the read parts will have new values.

26. Creating New Inputs
For a given execution some parts of the input may not be read.
When the path condition is solved, only the read parts will have new values.
The changes are written over the original input, preserving the unused parts.

27. Reducing the Path Condition
Subject
and
Input
Path Conditions can be very long.
Not all constraints are effected by negating the PC.
Constraints not effected can be removed from the PC.
jFuzz
JPF
Subject
and
Original
Input PC
Negated PC PC
Minimizer
Solver
New
Input
Inputs
which
cause
crashes

28. Example Reduction Path Condition: [1] a + b < 10 [2] b > 6
[5] c + d > 7
[6] e != 1
[7] c – e = 5
[8] a == 2

29. Example Reduction Path Condition: [1] a + b < 10 [2] b > 6
Start with fuzzing the last constraint.
Path Condition:
[1] a + b < 10
[2] b > 6
[3] c < 15
[4] a < 3
[5] c + d > 7
[6] e != 1
[7] c – e = 5
[8] a != 2

30. Example Reduction Path Condition: [1] a + b < 10 [2] b > 6
Start with fuzzing the last constraint.
Select all constraints which contain variables in that constraint.
Path Condition:
[1] a + b < 10
[2] b > 6
[3] c < 15
[4] a < 3
[5] c + d > 7
[6] e != 1
[7] c – e = 5
[8] a != 2

31. Example Reduction Path Condition: [1] a + b < 10 [2] b > 6
Start with fuzzing the last constraint.
Select all constraints which contain variables in that constraint.
If one of the constraints contains multiple variables, select all constraints which contain those variables.
Path Condition:
[1] a + b < 10
[2] b > 6
[3] c < 15
[4] a < 3
[5] c + d > 7
[6] e != 1
[7] c – e = 5
[8] a != 2

32. Example Reduction Path Condition: [1] a + b < 10 [2] b > 6
Start with fuzzing the last constraint.
Select all constraints which contain variables in that constraint.
If one of the constraints contains multiple variables, select all constraints which contain those variables.
All other constraints can be removed.
Path Condition:
[1] a + b < 10
[2] b > 6
[4] a < 3
[8] a != 2

33. Example Reduction Path Condition: [1] a + b < 10 [2] b > 6
Start with fuzzing the last constraint.
Select all constraints which contain variables in that constraint.
If one of the constraints contains multiple variables, select all constraints which contain those variables.
All other constraints can be removed.
Variables not in the new PC are left unchanged.
Path Condition:
[1] a + b < 10
[2] b > 6
[3] a < 3
[4] a != 2

34. Reducing the Path Condition
Reductions are performed for every constraint that is negated.
jFuzz uses a UnionFind data structure to find which variables are connected to each other.
In our case study, the average reduction was from around 250 constraints to about 5 constraints.

35. Case Study Subject: Sat4J Goals: SAT solver written in Java.
Takes inputs in dimacs files.
~10 kloc.
Goals:
Create inputs that crash Sat4J.
Create a set of good inputs.
test1.dimacs
c test 3 single clauses and 2
c binary clauses
p cnf 4 5
1 0
2 0
3 0
-2 4 0
-3 4 0

36. Results After 30 minutes of execution:
12,000 input files were created.
70 crashes where found.
The crashes are actually normal for SAT4J
38 Invalid DIMACS files.
27 Contradictions.
4 Assertion Errors.
A Java compiler would be more compelling.
Any crash is due to a bug in the compiler.
Much larger program.

37. Performance Sat4J was run 100 times in each VM.
The times are average runtime.
Simplifying the Path Condition reduces the solving time by 30%.

38. Conclusions
jFuzz is the first concolic tester for Java which will work for any bytecode.
This opens the door for more advanced fuzzing techniques, such as grammar based fuzzing.

40. Fuzzing the Input a b c d e Path Condition: [1] a + b < 10
[7] c – e = 5
[8] a != 2 a b c d e

41. Fuzzing the Input
Path Condition:
[1] a + b < 10
[2] b > 6
[3] a < 3
[4] a != 2 a b c d e
The reduced PC does not effect all parts of the input.
The solver will only return values for variables in the PC.

42. Fuzzing the Input The new values are written over the original input.
Path Condition:
[1] a + b < 10
[2] b > 6
[3] a < 3
[4] a != 2 a' b' c d e
The new values are written over the original input.
The new input only differs from the original in the values in the PC.