1. 1 Idea: Using System Level Testing for Revealing SQL Injection-Related Error Message Information Leaks Ben Smith Laurie Williams Andrew Austin North Carolina State University

2. 2 Motivation In the CWE/SANS Top 25 Most Dangerous Programming Errors: SQL injection vulnerabilities Error message information leak vulnerabilities These vulnerabilities are related –Ensuring that input falls within desired range –Handling the case when input is out of range

3. 3 Why? How?

4. 4 Objective The goal of this research is to assess the relative effectiveness of system and unit level testing of web applications to reveal both SQL injection vulnerabilities and error message information leakage vulnerabilities when used with an iterative test automation practice by a feature development team.

5. Agenda Motivation and Objective Background –What is feature development? –What’s a hotspot? Case Study on Four Java Web Apps Discussion 5

6. Feature Development 6 DatabaseApplication Logic User Interface Write Tests Edit Patient Information Database Application Logic User Interface Write Tests View Operational Profile Database Application Logic User Interface Write Tests

7. 7 HOTSPOT

8. Agenda Motivation and Objective Background –What is feature development? –What’s a hotspot? Case Study on Four Java Web Apps Discussion 8

9. 9 Case Study Four open source Java web applications (SourceForge.net): 1.How many malicious tests are included in the unit tests distributed with the applications? 2.What percentage of hotspots are executed by the unit tests distributed with the applications? 3.How does this number compare to the unit tests set’s statement coverage?

10. Subject Web Apps ProjectiTrustHispactaLogicServicesTuduLists Version4.00.0.31.82.2 Lines of Code7707199150116178 Production Classes 14342155132 Database Classes20415 10

11. Results ProjectiTrustHispactaLogicServicesTuduLists Hotspots92234813 Covered by Intrinsic Tests 8920473 Coverage97%87%98%23% Statement Coverage 84%49%53%40% Intrinsic Tests with Malicious Input 0000 11

12. Case Study, cont’d 4.Write system level test cases that execute every hotspot with input that is typical (safe) and then that is malicious. Do these tests expose: –SQL Injection Vulnerabilities? –Error Message Information Leakage Vulnerabilities? 12

13. Results, cont’d ProjectiTrustHispactaLogicServicesTuduLists System Level Tests EMIL Vulnerabilities 2294 SQLI Vulnerabilities 0000 13

14. Case Study, cont’d 5.Augment the unit tests in the same fashion that execute every hotspot with input that is typical (safe) and then that is malicious. Do these tests expose Error Message Information Leakage Vulnerabilities? 14

15. Results, cont’d ProjectiTrustHispactaLogicServicesTuduLists System Level Tests EMIL Vulnerabilities 2294 SQLI Vulnerabilities 0000 Unit Level Tests EMIL Vulnerabilities 0000 15

16. Agenda Motivation and Objective Background –What is feature development? –What’s a hotspot? Case Study on Four Java Web Apps Discussion 16

17. 17 Summary Unit and system testing every hotspot did not expose any SQL injection vulnerabilities System level testing revealed 17 error message information leak vulnerabilities Attempted to expose the error message vulnerabilities at the unit level and we could not

18. 18

19. 19 Easy Solution

20. 20 Conclusions Prepared statements (when used correctly) effectively protect against SQL injection attacks System level testing must be used to expose error message information leakage vulnerabilities when used with an iterative test automation practice by a feature development team.

21. 21 Questions?

22. 22 $username = $_POST[‘username’]; $password = $_POST[‘password’]; $result = mysql_query( “select * from users where username = ‘’ OR 1=1 ---’ AND password = ‘$password’”); $firstresult = mysql_fetch_array($result); $role = $firstresult[‘role’]; $_COOKIE[‘userrole’] = $role SQL Injection Attacks ‘ OR 1=1 --