1. Copyright © 2007 - The OWASP Foundation Permission is granted to copy, distribute and/or modify this document under the terms of the Creative Commons Attribution-ShareAlike 2.5 License. To view this license, visit http://creativecommons.org/licenses/by-sa/2.5/ The OWASP Foundation OWASP & WASC AppSec 2007 Conference San Jose – Nov 2007 http://www.owasp.org/ http://www.webappsec.org/ Lessons Learned from an Application Security Program Jim Routh CISO, The Depository Trust & Clearance Corporation Jrouth@dtcc.com (212) 855-8842

2. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 2 Background  DTCC, through its subsidiaries, provides clearance, settlement and information services for equities, corporate and municipal bonds, government and mortgage-backed securities, money market instruments and over-the-counter derivatives.  The views expressed in this presentation do not necessarily reflect the views of DTCC  The lessons learned are the result of 3 years worth of experience with an application security program

3. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 3 The Challenge in 2005 The Depository Trust & Clearing Corp (DTCC) had 450 application developers on shore and over 100 offshore creating product for their brokers, bank, mutual fund and insurance carrier customers. DTCC needed to implement improved security practices as part of the application development process. The goal was to create more secure applications to handle clearance and settlement of more than $1.6 Quadrillion worth of securities transactions each year  Context: CMMI Level 3 Certified development organization  Dilemma: What is the best approach to improving the quality of software developed, enhanced and maintained?

4. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 4 The Approach  The primary focus of the ADS Program is to teach developers how to develop secure code  Enhanced SDLC requiring security deliverables and controls at every phase of the lifecycle  Designed a curriculum for a core team of highly skilled developers to teach them about security and then tested them  18 selected for the program, 16 passed the test  Selected vulnerability scanning tools (static code analysis, black box testing, integrated vulnerability reporting, etc.)  Added “gatekeeper” types of controls in the SDLC workflow  Changed the model for CIS support

5. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 5 Lessons Learned 1.A comprehensive program requires more than tools 2.Education of application developers is essential 3.The work effort supporting the implementation of controls is more like a behavioral change project than a systems integration project 4.Linking vulnerability results with an accountability model that is visible drives changes in behaviors 5.There is a compelling economic incentive for controls 6.Teaching developers how to “break” applications is hard

6. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 6 A Comprehensive program requires more than tools Product Deployment ADS Program SDLC Integration Security Vulnerability Management Program [Reporting] Static Code Scanning -Fortify Data Base Scanning & Config. – AppDetective/AppRadar Web Integration Testing - AppScanHigh Risk End-to-End Pen Test - Primeon SILC/Clasp Methodology Integration Standard Application Security Logging CIS Project Level Support & Code Reviews ASAR Reengineering [Design] Code complexity scan & review CAST COTS Code Assessment - Veracode Website Vulnerability Scanning- WhiteHat = In implementation

7. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 7 Four Primary Areas of Focus Policy Process TrainingAutomation App Sec Policy Development App Sec Control Standards Secure Coding Guidelines Security Requirements Threat Modeling Test Planning Stage Gate, PSA, CIS support, Work flow Deep Source Analysis Penetration Testing Vulnerability Assessment Metrics / Trending Reporting Security Awareness Remediation for Developers Role Based Security Process Tool integration

8. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 SDLC Enhancements

9. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 9 Education is Essential Mavens Java C++ Mainframe All Developers Proj. Mgrs. Portfolio Leaders CIO StakeholderEducation Content Secure Programming Techniques & Tools Java static code analysis C++ static code analysis Mainframe security techniques OWASP Top 10, Tool training Managing defect removal and remediation within budget Techniques for reducing the vulnerabilities per line of code Industry practices in Security

10. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 10 Soft Skills in Implementation  Stakeholder analysis  Transparency of vulnerability information PhasePhase Business Unit Design Development Requirements Testing Production 123456 S S S SRR R RR R=Resister S= Supporter N=Neutral S S S NN N S S S S S S S N N NS N N N R

11. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 11 Accountability Model KPIs CIO Portfolio Leaders Project Team

12. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 KPIs Lifecycle Phase Description Comments or Formula Month Initiation Respond to project in 3 business days Date out - Date in 93.33% 14 out of 15 PSA are responded in 3 days Initiation KPI reflects number of projects in which CIS Consulting team advised to change the response due to knowledge of systems. # PSA corrected for responses/Total #PSA Requires process change and changes to Portal 5.88% One PSA was corrected Build KPI reflects the percentage of Code scanned via Vulnerability scanner # LOC scanned/ # LOC 50%

13. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 KPI’s (Continued) KPISuccess CriteriaFrequencyPhase % of Projects with security exceptions in production Green <3% = Green Yellow 3-5%= Yellow >5%= Red MonthlyIntegrated Testing Security Maven Monthly Meetings attendance Green >85% = Green Yellow 60-85%= Yellow <60%= Red MonthlyAll % of Projects completing static code scan checkpoint Green >90% = Green Yellow 75-90%= Yellow <75%= Red MonthlyBuild

14. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 14 Economics of Defect Management  Root cause of security challenges:  Gartner- 75% of breaches due to security flaws in software  NIST- 92% of vulnerabilities are in software  The cost of fixing a bug in the field is approximately $30,000 vs. $5,000 during coding (NIST, “The Economic Impacts of Inadequate Infrastructure for Software Testing 2002” )  “Software development organizations that perform security code reviews will experience a 60% decrease in critical vulnerabilities found in production environments” Gartner, April 2006

15. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 “Fuzzing” Is More Difficult than it Sounds  Fuzz testing has emerged as a highly useful testing technique to add to the SDLC  Black box tools are useful but Fuzz testing typically addresses application design issues in a more holistic approach  Gaming skills may be transferable to fuzz testing  Using external expertise makes sense  Organically growing this skill is difficult

16. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 16 A Comprehensive program requires more than tools Product Deployment ADS Program SDLC Integration Security Vulnerability Management Program [Reporting] Static Code Scanning -Fortify Data Base Scanning & Config. – AppDetective/AppRadar Web Integration Testing - AppScanHigh Risk End-to-End Pen Test - Primeon SILC/Clasp Methodology Integration Standard Application Security Logging CIS Project Level Support & Code Reviews ASAR Reengineering [Design] Code complexity scan & review CAST COTS Code Assessment - Veracode Website Vulnerability Scanning- WhiteHat = In implementation

17. OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 Resources OWASP Top 10 Web Application Vulnerabilities http://www.owasp.org/index.php/Top_10_2007 http://www.sans-ssi.org/ Sources Services http://www.webappsec.org/ http://www.aspectsecurity.com/ http://www.cigital.com/