1. VUzzer: Application-aware Evolutionary Fuzzing
Sanjay Rawat, Vivek Jain, Ashish Kumar, Lucian Cojocar, Cristiano Giuffrida, and Herbert Bos
Victor van der Veen
32 year old from Amsterdam
3rd year PhD student with Herbert Bos
ETA: summer 2018

2. VUzzer Smart fuzzing without symbolic execution
Your takeaway message of today
Smart fuzzing without symbolic execution
Extract application features for meaningful mutation
VUzzer 30K inputs: 403 crashes
AFL K inputs: 238 crashes
NDSS – March 1, 2017

3. What are we trying to solve anyway?
PROBLEM STATEMENT
What are we trying to solve anyway?
NDSS – March 1, 2017

4. AFL AFL will run for hours on this ... read(fd, buf, size);
if (buf[5] == 0xD8 && buf[4] == 0xFF)
// interesting code here
else pr_exit(“Invalid file”);
AFL will run for hours on this
NDSS – March 1, 2017

5. AFL AFL will run for hours on this ... read(fd, buf, size);
if (buf[5] == 0xD8 && buf[4] == 0xFF)
// interesting code here
else pr_exit(“Invalid file”);
AFL will run for hours on this
Has to figure out that offset 4 and 5 are of interest (where)
NDSS – March 1, 2017

6. AFL AFL will run for hours on this ... read(fd, buf, size);
if (buf[5] == 0xD8 && buf[4] == 0xFF)
// interesting code here
else pr_exit(“Invalid file”);
AFL will run for hours on this
Has to figure out that offset 4 and 5 are of interest (where)
Needs to guess 0xFFD8 (what)
NDSS – March 1, 2017

7. More Problems Handling ‘Complex’ Code Structures
Magic bytes Certain values shall be placed at pre-determined offsets
Deeper execution Many inputs will end up in less interesting error-handling code
(multibyte) Markers Not at fixed offsets: if (strstr(&buf, “MAZE”)) ...
NDSS – March 1, 2017

8. Our mutation-, coverage-based, greybox fuzzer
VUZZER
Our mutation-, coverage-based, greybox fuzzer
NDSS – March 1, 2017

9. VUzzer Where to mutate, what to insert Avoid non-scalable techniques
Evolutionary fuzzing Mutate/select most promising paths
Magic byte detection Find possible magic byte values to reach deeper into the binary
(limited) Input type detection Aid mutation by detecting input bytes of certain types (integers)
Avoid non-scalable techniques
No symbolic execution
Limited use of Dynamic Taint Analysis
NDSS – March 1, 2017

10. Feature Extraction Data-flow features Control-flow features
NDSS – March 1, 2017

11. Feature Extraction Data-flow features Control-flow features
Information about relationship between input data and program computations
Extracted using static analysis / dynamic taint analysis
Example: cmp instructions on x86
Offsets: which input bytes are compared against? (taint analysis)
Magic values: immediate operands for cmp (static analysis)
Example: lea instructions
Integer types: is index operand tainted?
Control-flow features
NDSS – March 1, 2017

12. Feature Extraction Data-flow features Control-flow features
Information about importance of certain execution paths
Identify error-handling blocks (heuristics based)
Rank basic blocks to prioritize hard-to-reach code
Each basic block gets a weight depending on how deep it is nested
Error-handling blocks get a negative weight
NDSS – March 1, 2017

13. NDSS – March 1, 2017

14. CMP Immediates
Rank Basic Blocks
NDSS – March 1, 2017

15. Seed Inputs (known valid)

16. Dynamic Taint Analysis
Seed Inputs (known valid)

17. Dynamic Taint Analysis
+ Error-handling code
+ Magic bytes
+ LEA offsets
NDSS – March 1, 2017

18. Fitness(executed code)
High scores for inputs that execute highly ranked basic blocks

19. Mutate and loop Dynamic Taint Analysis only when new code is covered
NDSS – March 1, 2017

20. VUZZER
NDSS – March 1, 2017

21. AFL: 22k inputs VUzzer: 400 inputs Evaluation
Darpa Cyber Grand Challenge
VUzzer / AFL
13 binaries, 6 hours per binary
Far fewer VUzzer inputs
VUzzer: 400 inputs
NDSS – March 1, 2017

22. Evaluation LAVA-M Dataset
LAVA: inject hard-to-reach faults to (e.g.,) evaluate fuzzers
VUzzer hits significantly more bugs than
FUZZER: coverage-based
SES: symbolic execution / SAT-based
Program
Total Bugs
FUZZER
SES
VUzzer
uniq 28 7 27
base64 44 9 17
md5sum 57 2 1
who
2136 18 50
NDSS – March 1, 2017

23. Evaluation Various Applications
Comparison against AFL on vanilla Ubuntu 14.04
More bugs with fewer inputs
Program
Crashes (AFL)
Crashes (VUzzer)
Inputs (AFL)
Inputs (VUzzer)
mpg321 19
337
883K
24K
gif2png 7
127
1.84M
43K
pdf2svg 13
923K 5K
tcpdump 3
2.89M
78K
tcptrace
238
403
3.29M
30K
Djpeg 1
35.9M
90K
NDSS – March 1, 2017

24. Evaluation Various Applications
Comparison against AFL on vanilla Ubuntu 14.04
More bugs with fewer inputs
Program
Crashes (AFL)
Crashes (VUzzer)
Inputs (AFL)
Inputs (VUzzer)
mpg321 19
337
883K
24K
gif2png 7
127
1.84M
43K
pdf2svg 13
923K 5K
tcpdump 3
2.89M
78K
tcptrace
238
403
3.29M
30K
Djpeg 1
35.9M
90K
NDSS – March 1, 2017

25. Evaluation Various Applications
Comparison against AFL on vanilla Ubuntu 14.04
Crash faster
Consistent progress
NDSS – March 1, 2017

26. Conclusion
VUzzer
Novel fuzzing technique based on evolutionary approach
Application-aware fuzzer by exploiting data-flow and control-flow features
Prioritize hard-to-reach code paths
Deprioritize error-handling code
Significantly more bugs with orders of magnitude fewer inputs in less time
NDSS – March 1, 2017

27. https://github.com/vusec/vuzzer
Final Remarks
Open Source
VUSec Project Page