Why Security Testing Is Hard by Herbert H. Thompson presented by Carlos Hernandez

Overview  Introduction  Side-effect behavior  The state of security testing  The need for techniques  The need for tools  Conclusion  Q & A

Introduction  Software testing has become pretty good at verifying requirements  Many types of bugs escape testing  Testers make test cases for correctness, not absence of additional behavior

Side-effect behavior  Typical functional test –Apply input A –Look for presence of result B  What if the application also performs action C?  Example: RDISK utility in Windows NT 4.0

Side-effect behavior cont.

The state of security testing  Security testing traditionally referred to executing a suite of scripted tests that represent known exploits  Problem = finds old vulnerabilities, not new ones  This technique actually works because developers make the same mistakes  Recently there has been an increasing level of security awareness

The need for techniques  Key to success is extracting techniques to find bugs instead of translating them into scripted test cases  Study conducted by Thompson and Whittaker –What fault would have caused this vulnerability? –What were the failure symptoms that should have alerted a tester to the vulnerability’s presence?

Techniques cont. –What testing technique would find this vulnerability?  4 general classes of testing techniques: 1.Dependencies 2.Unanticipated user input 3.Techniques to expose design vulnerabilities 4.Techniques to expose implementation vulnerabilities

Dependency failures  Software operates in a highly codependent environment  2 security issues are of concern: 1.Application might inherit insecurities 2.External resource that provides some security service to an application might become unavailable or fail

Unanticipated user input  Some inputs can cause undesirable side effects and require special testing attention  Most notorious side effect: buffer overflow  Applications might not consider characters and character combinations that the application could interpret as commands

Design insecurities  Many security vulnerabilities are designed into an application –i.e. test instrumentation added for testing purposes  Many applications are released with these instrumentations  These interfaces can bypass security controls to allow easy testing

Implementation insecurities  Imperfect implementation can make even the most perfect designs insecure  Specifications can outline security meticulously and yet be implemented in a way that causes insecurity  i.e. man-in-the-middle attack

The need for tools  The software community desperately needs tools that address the peculiarities of security vulnerabilities and bring their symptoms into plain view during development and testing  Able to not only monitor for side effects and environmental interactions but manipulate them as well

Conclusion  Security testing must change  We must apply new methods into practice if we ever hope to ship secure code with confidence

Q & A  If you have any questions just pretend you’re me and answer yourself.  Just remember, if there aren’t any questions we can go home faster!