1. Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011

4. Todo Define gray-box testing Why black-box is insufficient What we built Examples Haters club

5. Definitions Black-box testing System-level tests No assumptions about implementation

6. Definitions White-box testing Examine implementation Test components in isolation

7. Definitions Gray-box testing System-level tests (like black-box) Examine implementation (like white-box)

8. The Software Security Game Objective Rules vs. Strategy Playing Field

9. OBJECTIVE: Protect everything OBJECTIVE: Exploit one vulnerability

10. Rules for the Defender 1.Don’t attack the attacker

11. Rules vs. Strategy Rules Don’t attack the attacker Strategy Emulate attacker’s techniques

12. Who wins? Technology Expertise

13. Who wins? Time Technology Expertise

14. Who wins? Technology Expertise Time

15. Changing the odds

16. The Defender’s Advantage Time Inside Access Technology Expertise

17. Prior Art 2005: Concolic testing: Sen, University of Illinois 2008: Microsoft SAGE: Godefroid, MSR 2008: Test Gen for Web Apps: Shay et al, U. Washington 2008: Accunetix: Accusensor

18. Access to the Software Allows for ‘Hybrid’ analysis Black-box ApproachWhite-box Approach

19. ‘Hybrid’ Analysis Mostly Broken Correlation Engine

21. The ‘Real-Time Hybrid’ Approach Good Results Correlation Engine

22. Evolving to Integrated Analysis Application Real-time link Find More Fix Faster

23. Find More Reduce false negatives Automatic attack surface identification Understand effects of attacks Detect new types of vulnerabilities Privacy violation, Log Forging

24. Attack surface identification /login.jsp /pages/account.jsp /pages/balance.jsp /admin/admin.jsp File system Configuration-driven Programmatic

25. Understand effects of attacks /admin/admin.jsp ✗ Command Injection sysadmin$./sh ✔

26. Fix Faster Reduce False Positives Confirm vulnerabilities Provide Actionable Details Stack trace Line of code Collapse Duplicate Issues Tie to root cause

27. Reduce False Positives /admin/admin.jsp SQLi? ✔

28. Actionable Details /login.jsp

29. Collapse Duplicate Issues /login.jsp /pages/account.jsp /pages/balance.jsp 1Cross-Site Scripting23 1

30. JavaBB – Case Study Open Source Bulletin Board Additional Vulnerabilities Finds18 SQL Injection results Root cause analysis 18 SQL injection results have 1 root cause

31. Vulnerability Diagnosis Confirmed SQL Injection

32. Actionable Details Line of Code Parameters Stack Trace

33. Yazd – Case Study Open Source Forum Additional Attack Surface Discovers hidden ‘admin’ area 3 Additional Cross-Site Scripting results Root cause analysis Collapses 34 XSS into 24 root-cause vulnerabilities

34. Attack surface identification Hidden ‘admin’ area

35. Collapse Duplicate Issues

36. One More Case Study

37. Future Automated anti-anti automation

38. The Case Against “Hybrid” Hard to find attack surface with static analysis Static/dynamic correlation doesn’t work Doesn’t help with false positives / false negatives Nobody will run a software monitor (cheating!)

39. The Case for Gray-Box Testing Black-box is a losing game Find more Attack surface Vulnerability diagnosis Fix faster Root cause analysis Collapse duplicates

40. Gray, the New Black Gray-Box Web Vulnerability Testing Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011