1. Web Application Security Testing Automation.

2. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.1 What types of automated testing are there? What does web application security assessment comprise? How much can tools help? Where is it best to use these tools? Agenda

3. What types of automated testing are there? 2

4. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.3 When can you test? User Acceptan ce Testing Project Based Development Project Based Development Functiona l Testing Functiona l Testing Non- Function al Testing Pilot Pre Production Production Thank God its gone live party. Performa nce & Volume Testing Feature requests TES T HER E? TES T HER E? TES T HER E? TES T HER E? TES T HER E? TES T HER E? TES T HER E? TES T HER E? TES T HER E? TES T HER E? BAU development BAU testing

5. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.4 Raw Source Code Review Get the code and use software configured with rules to find exceptions and investigate them What types of automated testing are there? Source Code Rules Analysis Raw Results (means something to a developer) Human review Findings (means something to a project manager) Source Secure Programming with Static Analysis Chess & West

6. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.5 Integration into development environment Static analysis as you go Write some code, push to webserver, do some “black box testing” Hmm what’s the first thing the developer will skip when he is under pressure to ship code What types of automated testing are there?

7. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.6 Integration into test software Get some test data Capture some UAT test scripts Run those UAT scripts Use those test scripts to do some “black box” testing Try and persuade a developer that the defect is a defect Try and find some project managers to agree who is to pay to fix the defect Don’t expect your UAT test team to do security testing, they are usually lovely people, as they deal with the business What types of automated testing are there?

8. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.7 Assessment from network It’s ready to go, let’s do a final check.. With some test data walk the application logic, ALL of the application logic  Scan away Try and read the report before the project goes live Try and find a developer to educate? What types of automated testing are there?

9. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.8 Fuzzing Aka I’ve run out of ideas, lets just bash away until something weird happens with input validation or business logic What types of automated testing are there?

10. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.9 What types of automated testing are there? MethodProsCons Source Code Review Can be done at any time during/after development Access to source code required, think contractuals  Development Environment Integrated Can be leveraged by the developer to help educate them. Can only be done during development Test Environment Integrated Testing is when most test data is hopefully available Can only be done during testing If you find a major input validation problem during test you will have to repeat UAT testing! From Network Can be done at any time Can cause a Denial of Service to the application Fuzzing Application has to be operational Can be slow over the internet

11. How much can tools help? 10

12. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.11 What does the testing comprise? Web Application Security Assessment Typical Breakdown of Effort There is a lot of manual testing involved in web application security testing The majority of findings are related to poor implementation of role based access controls and “business logic flows”. Hence most effort is directed towards business logic testing.

13. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.12 What are we looking for? Incidence of 10 common web application vulnerabilities in applications recently tested by Deloitte in the UK. (Vulnerability classifications defined by the Open Web Application Security Project – www.owasp.org) % of tested Web Applications susceptible to vulnerability

14. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.13 How web application scanners work Thankfully stolen from http://www.blackhat.com/presentations/win- usa-04/bh-win-04-grossman/bh-win-04-grossman-up.pdf http://www.blackhat.com/presentations/win- usa-04/bh-win-04-grossman/bh-win-04-grossman-up.pdf

15. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.14 Requirement for test data due to multi-page sequences Dynamically produced content Single Sign On/Identity Management/NTLM/Kerberos wackiness Client side code (bad architect, bad architect!) Non standard error messages (good developer!) Denial of Service to application, email system, network monitoring etc. Anti-automation Challenges of automated scanning

16. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.15 How much can tools help? Reduction in Effort Some aspects of testing can be automated and reduce effort Other aspects of testing from automation are improved by reduction in human errors

17. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.16 How good are they at finding defects? A1 - XSS A2 - Injection Flaws A3 - Malicious File Execution A4 - Insecure Direct Object Reference A5 - CSRF

18. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.17 How good are they at finding defects? A6 - Informati on leakage and improper error handling A7 - Broken Authenti cation and Session Manage ment A8 - Insecure cryptogr aphic storage A9 - Insecure communi cations A10 - Failure to restrict URL access Stolen with thanks from http://jeremiahgrossman.blogspot.com/2007/05/web- application-scan-o-meter.html http://jeremiahgrossman.blogspot.com/2007/05/web- application-scan-o-meter.html

19. Where is it best to use these tools? 18

20. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.19 Where to use? MethodA Good situation to use? Source Code Review Outsourced project development BAU development Development Environment Integrated Education of BAU developers? Test Environment Integrated Hmmm? From Network Scanning masses of brochure-ware sites for poor input validation and problems like XSS and SQL injection. Fuzzing Vulnerability research

21. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.20 Manual and Automated Testing complement each other MethodProsCons Manual Picks up business logic flaws. Flexible in the face of an unfinished/unreli able application or test environment Sample based approach may miss instances of “low hanging fruit”. Automa ted Checks for boring vulnerabilities so you don’t have to (e.g. information disclosure, backups of files, XSS) can be done more efficiently and comprehensively Doesn’t pick up the really important business logic flaws Inflexible if the application is not completed.

22. Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved.21 Of the automated tools, source code review tools are most flexible as they can be used at any point in the development cycle Manual testing and automated testing complement each other Conclusion