1. Fuzzing for logic and state issues
Gödel's Gourd
Fuzzing for logic and state issues

2. Introductions Michael Eddington
CTO and Principal Déjà vu Security
12+ years in security consulting
Senior developer/architect in prior life
Author of Peach, an open source fuzzer
Device, Kernel, User, Web, Network

3. DARPA Cyber Fast track
Thanks Mudge!

4. Defining the Problem
Fuzzers are good at findings implementation issues
…that crash the target
…that are generically detectable (sqli, xss)
Not good at finding design, logic, and state issues
…that do not crash the target
…that are not generically detectable

5. Examples Buffer Overflows Memory Corruption Resource Usage
Easy
Hard
Buffer Overflows
Memory Corruption
Resource Usage
SQL Injection
Missing authentication
State corruption
Incorrect logic

6. Authentication Examples
Out of 100 admin pages, 5 are missing authentication
Microsoft SSPI skip a step auth bypass
OpenBSD IPSEC incorrect if/then logic

7. Authentication – Detect
Web – Missing Auth
Trigger
Request page w/o logging in
MS SSPI/OBSD IPSEC
Skip a step
Status Code (200/403)
What pages require auth
Result (Pass)
Did we complete all steps

8. Logic Example Windows 95 SMB Flaw
Logic error in password checking code
Length of loop determined by client input
Modified SMB client, ~32 attemps always wins
We never throw an exception or crash
Typical generic fuzzer will never find this

9. Logic – Win95 SMB
bool CheckPw( int userdata_len, char* userdata, int sysdata_len, char* sysdata ) { for(int i=0; i<userdata_len; i++) if(userdata[i] != sysdata[i]) return false; return true; }

10. Logic – Detect Win 95 SMB Trigger Result Does password match
Try all chars
Remove NULL
Result
Does password match

11. State Example Device (phone/tablet/laptop) with theft system
Agent “heartbeats” to server
Server can trigger “stolen” mode in laptop
Laptop will trigger if unable to “heartbeat”
Timer/counter runs down

12. State – Detect System Server Trigger Can we perform state flow?
Cause exception
Flow locked
Unable to heartbeat
Can we perform state flow?
Check result of each step

13. How to detect? Goal – Modify existing fuzzer to detect these issues
We already produce triggers
How do we add detection?

14. How to detect? What do we need to detect these issues?
Provide system constraints
If not authenticated result is 402
If steps 1, 2, and 3 not performed step 4 is error
Result is never 500
Verify we are still working
Perform state flow w/o mutations

15. Proposed Solution Gödel's Gourd Re-use Peach fuzzing engine
Mutation engine
Fault detection/reporting
Constraint language
Control iterations (non mutation iterations)
Mutate state model (skip, order, etc.)

16. Control Iterations Goal: Verify target is working correctly
No mutations
Constraints pass
State model is followed
Matches recorded control iteration

17. How it works R – Record iteration 1 – Fuzzing iteration
C – Control iteration
2 – Fuzzing Iteration
3 – Fuzzing iteration …
Remember all states/actions from record iteration
Verify on control iterations
Control iterations every N fuzzing iterations

18. Outcome If control does not match record – throw fault
Identify conditions that stop normal operation

19. Constraints Verify logic via simple constraint expressions
Apply constraints to state model
State
Action
Does not modify fuzzer state

20. Language Options Existing Traditional Languages Pro Cons Well known
JavaScript
Python
Ruby
etc.
Pro
Well known
Available via .NET scripting interface
Cons
Allows modification of fuzzer state.

21. Other Options Domain Specific Language (DSL) Use existing
Create our own
Pros
Meet all requirements
Cons
Must implement
Not well known

22. DSL Selection Object Constraint Language (OCL)
Specification language, no side effects
Developed as part of new UML standards
Familiar syntax
Relatively easy to implement

23. Object Constraint Language (OCL)
Expression types
Invariant (inv)
Always true
Pre (pre)
Evaluated before [ something ]
Post (post)
Evaluated after [ something ]
Can access state from Pre.

24. OCL Examples
“Car owner must be at least 18 years old” context Car inv: self.owner.age >= 18 “If passwords match result is true” context Login post: result = true implies pass1 = pass2

25. OCL Context Groups sets of constraints
Constraints for a context are run together
Association based on context

26. Normal Fuzzing Iteration
Enter State Model
State 1
Action 1.1
Send Data
Action 1.2
Receive Data
State N …

27. Fuzzing Iteration With Constraints
Enter State Model
State 1
Action 1.1
Send Data
Action 1.2
Receive Data
State N …
Inv(pre)
Pre
EVENT
Inv(post)
Post

28. Applying (Authentication)
Web Authentication
# Verify authentication occurred
post:
(reply = 200 &&
url.indexOf(‘/admin’) > -1)
implies
auth.reply = 200

29. Applying (Authentication)
Windows SSPI
# Verify all steps completed
post: reply = true implies (
auth.step1.reply = true &&
auth.step2.reply = true &&
auth.step3.reply = true)

30. Applying (Logic) Windows 95 Bug
post: reply = true implies userpw = ‘password’

31. Applying (State)
Antitheft System
Perform control iteration

32. Implementation

33. Technologies Used Microsoft .NET Framework – C# Peach Fuzzer 3
Cross platform using Mono
OS X
Linux

34. Implementation Diagram

35. OCL Implementation Irony .NET Language Toolkit LINQ Expressions
Many differences from traditional
Grammar is code
Easy AST hookups
LINQ Expressions
From IronPython work
Last mile is already done

36. LINQ Expressions Exposes language constructs for use in AST classes.
Does all the heavy lifting.
return Expression.Condition(
(Expression)ifNode.Evaluate(thread),
(Expression)thenNode.Evaluate(thread),
(Expression)elseNode.Evaluate(thread));

37. Gödel Usage
All the things that do the stuff

38. Peach Pit vs. Gödel Gourd
Data Model
State Model
Agents
Test
Data Model
OCL Definitions
State Model
OCL Associations
Agents
Test

39. Gödel: Define Constraints
<Ocl>
<![CDATA[
context StatusCodeOk
post: context.test.publishers[self.publisher].Result = 'OK'
]]>
</Ocl>

40. Gödel: Associate Constraints
<Action type="call" method="Logout">
<Ocl context="StatusCodeOk" />
</Action>
Constraints will now run with this Action.

41. Gödel: Control Iterations
<Test name=“Default” controlIteration=“1”> <Agent … /> <StateModel … /> <Publisher … /> <Logger … /> </Test> Define how often control iterations occur.

42. Usage Feasibility
Time and Cost

43. Adding Gödel Process: Not recreating full application logic
Existing Peach PIT
Add OCL Constraints
Test and Verify Definition
Not recreating full application logic
Just our “view of the world”

44. Time per Protocol Based on current experience of limited protocol set
Decent in 1 – 2 days
Complete in 1 week or less

45. Performance What performance impact does Gödel incur?
Constraint evaluation
Control iterations
No performance optimizations…yet

46. Performance of Constraints

47. Performance Control Iterations
Depends on how often, worst case half speed
Never longer than mutation iterations

48. Performance Conclusions
Performance impact dependent on speed of fuzzing
Ability to scale fuzzing lowers impact
For fast fuzzers, acceptable impact
For slower fuzzers, adjust control iterations to occur less often

49. Conclusions Pentesting/Quick fuzzing General definition building
Reasonable for “basics” (verify state’s work, critical logic flows)
General definition building
Reasonable to implement decent coverage
1-2 days “good enough”

50. Wrapping it up…

51. Lessons Learned Constraints applied only to control iterations
Writing good constraints that apply to all mutation cases is challenging
A few constraints can go along ways
Performance overhead needs to be lowered when many constraints used.
Optimize access to most used variables/objects

52. Looking towards next rev…
Can we “learn” basic constraints?
Performance optimizations
Shorted “name” of common objects
context.test.publishers[self.publisher].Result
self.Result

53. Thanks for all the fish! Michael Eddington mike@dejavusecurity.com