1. Copyright © 2004 - The OWASP Foundation Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License. The OWASP Foundation OWASP AppSec June 2004 NYC http://www.owasp.org OWASP Testing Framework Mark Curphey, OWASP Founder Director of Software Security Foundstone mark.curphey@foundstone.com

2. OWASP AppSec 2004 2 Testing Project Background  Need  Incremental Rise of Application Security Vulnerability Reports  Increased Awareness (Dept. Homeland Security Initiative)  Lack of ‘Good’ Information and Knowledge  Marketing FUD  Part 1 – “Why, What, Where, When”  Why should I tackle the problem strategically?  What do I need to consider as “in scope”?  Where should testing take place?  When should I test; after the application is built?  Part 2 – “How”  How to Find Vulnerabilities  Source Code Analysis  Manual Inspection  Black Box Testing

3. OWASP AppSec 2004 3 Testing Project Background IdealizedReal World

4. OWASP AppSec 2004 4 The Business Landscape 50% of Capital Expenditure is Spent on IT Technology Returns Technology Is the Business Differentiator Business Management ROI IT $

5. OWASP AppSec 2004 5 Economics of Insecure Software of Software and Applications Tested Have Serious Design and Implementation Flaws - Foundstone Survey in Cost to USA Economy from Poor Software Quality –US Dept of Commerce Cost of Insecure Software to the Financial Services Industry – NIST Survey 2002 100 Times More Expensive to Fix Security Bug at Production Than Design – IBM Systems Sciences Institute 76 % $ 60 B $3 B 100x

6. OWASP AppSec 2004 6 Economics of Insecure Software

7. OWASP AppSec 2004 7 Economics of Insecure Software  Sailing with the Wind  Time is the Enemy  Growth at All Costs  Revolutionary Offers  Horizontal for Breadth  Geographical Coverage  Catching the Next Wave  Sailing into the wind  Waste is the Enemy  Cash Flow at All Costs  Evolutionary Offers  Vertical for Depth  Domain Expertise  Fixing the Leaky Pipe LostCustomerLoyalty Order Cancellation Lost Availability Late to Market Lost Productivity DelayedResponses Scrappedwork Repeated Service Calls Excessive Capacity SuboptimalDesigns Margin In Margin Out

8. OWASP AppSec 2004 8 Common Misconceptions When Building a Testing Program  “…We use penetration testing and automated scanners so we have it covered”.  If you fail a penetration test you know you have a really, really bad problem. If you pass a penetration test you do not know that you don’t have a really bad problem.  Best application scanner finds < 20% of web application security holes  “…We have an application firewall so we don’t need to test for those sorts of holes”.  Don’t understand the business logic  “…We test everything before it goes live”.  Dramatic cost implications  Usually implies black-box testing  Key Message: In order to build better software, you have to build a better software development process.

9. OWASP AppSec 2004 9 Principles of Testing  There is no silver bullet  Think strategically, not tactically  The SDLC is King  Test early and often  Understand the Scope  Mindset  Know Thy Target  Use the Right Tool for the Right Job  Devil is in the Details  Use the Source Code Where Possible  Develop Metrics for Measurement and Continuous Improvement

10. OWASP AppSec 2004 10 Scope of Testing General Guide Human Inspections and Manual Review – 50% Code Review – 35% Penetration Testing – 15% Threat Modeling – technique that can help narrow the scope of testing (and develop / ensure effective countermeasures) UML – visual modeling technique that can help remove ambiguity

11. OWASP AppSec 2004 11 UML Model Examples Use Case Diagrams help understand and document the functionality

12. OWASP AppSec 2004 12 UML Model Examples Sequence Diagrams help explore the actual workings of specific functionality

13. OWASP AppSec 2004 13 Techniques and Approaches  Human Inspections and Manual Review  Documentation  Policy  Procedures  Standards  Interviews  SDLC  Design and Architecture Reviews  Operational Management  Threat Modeling  Technological  Code Review  Walkthroughs  Audit  Penetration Testing  Run-Time Analysis  Configuration Reviews

14. OWASP AppSec 2004 14 OWASP Testing Framework Explained  Phased Approach  Not tied to a specific software development methodology  Task orientated  Phase 1 – Before development begins  Phase 2 – During definition and design  Phase 3 – During development  Phase 4 – During deployment  Phase 5 – During maintenance

15. OWASP AppSec 2004 15 Phase 1- Before Development Begins  A) SDLC Process Review  People  Process  Technology  B) Policy and Standards Review  C) Develop Measurement and Metrics Criteria

16. OWASP AppSec 2004 16 Phase 2 – During Definition and Design  A) Security Requirements Reviews  B) Security Architecture Review  C) Create / Review UML models  D) Create Review Threat Models

17. OWASP AppSec 2004 17 Phase 3 – During Development  A) Code Walkthroughs  B) Code Reviews

18. OWASP AppSec 2004 18 Phase 4 – During Deployment  A) Application Penetration Testing  B) Configuration Management Reviews  Web Servers  Application Servers  Application Config (web.config)

19. OWASP AppSec 2004 19 Phase 5- Maintenance and Operations  A) Operational Management Reviews  B) Health Checks  C) Change Validation

20. OWASP AppSec 2004 20 Summary  Software (In)Security is an expensive problem that is growing in complexity and prevalence  There are no silver bullets  To improve the quality of your software you have to improve the quality of your software development process  To improve the security quality of your software you have to improve the security quality of your software development process  Building a testing program using the OWASP Testing Framework as a base, will help organizations ensure they address the right issues at the right time.

21. OWASP AppSec 2004 21 Conference Observations  Cultural and Mind Set Shift Towards Building Secure Software  Genuine Interest and Enthusiasm  New Ideas and Collaboration