Copyright © The OWASP Foundation Permission is granted to copy, distribute and/or modify this document under the terms of the Creative Commons Attribution-ShareAlike 2.5 License. To view this license, visit The OWASP Foundation OWASP & WASC AppSec 2007 Conference San Jose – Nov Lessons Learned from an Application Security Program Jim Routh CISO, The Depository Trust & Clearance Corporation (212)

OWASP & WASC AppSec 2007 Conference – San Jose – Nov Background  DTCC, through its subsidiaries, provides clearance, settlement and information services for equities, corporate and municipal bonds, government and mortgage-backed securities, money market instruments and over-the-counter derivatives.  The views expressed in this presentation do not necessarily reflect the views of DTCC  The lessons learned are the result of 3 years worth of experience with an application security program

OWASP & WASC AppSec 2007 Conference – San Jose – Nov The Challenge in 2005 The Depository Trust & Clearing Corp (DTCC) had 450 application developers on shore and over 100 offshore creating product for their brokers, bank, mutual fund and insurance carrier customers. DTCC needed to implement improved security practices as part of the application development process. The goal was to create more secure applications to handle clearance and settlement of more than $1.6 Quadrillion worth of securities transactions each year  Context: CMMI Level 3 Certified development organization  Dilemma: What is the best approach to improving the quality of software developed, enhanced and maintained?

OWASP & WASC AppSec 2007 Conference – San Jose – Nov The Approach  The primary focus of the ADS Program is to teach developers how to develop secure code  Enhanced SDLC requiring security deliverables and controls at every phase of the lifecycle  Designed a curriculum for a core team of highly skilled developers to teach them about security and then tested them  18 selected for the program, 16 passed the test  Selected vulnerability scanning tools (static code analysis, black box testing, integrated vulnerability reporting, etc.)  Added “gatekeeper” types of controls in the SDLC workflow  Changed the model for CIS support

OWASP & WASC AppSec 2007 Conference – San Jose – Nov Lessons Learned 1.A comprehensive program requires more than tools 2.Education of application developers is essential 3.The work effort supporting the implementation of controls is more like a behavioral change project than a systems integration project 4.Linking vulnerability results with an accountability model that is visible drives changes in behaviors 5.There is a compelling economic incentive for controls 6.Teaching developers how to “break” applications is hard

OWASP & WASC AppSec 2007 Conference – San Jose – Nov A Comprehensive program requires more than tools Product Deployment ADS Program SDLC Integration Security Vulnerability Management Program [Reporting] Static Code Scanning -Fortify Data Base Scanning & Config. – AppDetective/AppRadar Web Integration Testing - AppScanHigh Risk End-to-End Pen Test - Primeon SILC/Clasp Methodology Integration Standard Application Security Logging CIS Project Level Support & Code Reviews ASAR Reengineering [Design] Code complexity scan & review CAST COTS Code Assessment - Veracode Website Vulnerability Scanning- WhiteHat = In implementation

OWASP & WASC AppSec 2007 Conference – San Jose – Nov Four Primary Areas of Focus Policy Process TrainingAutomation App Sec Policy Development App Sec Control Standards Secure Coding Guidelines Security Requirements Threat Modeling Test Planning Stage Gate, PSA, CIS support, Work flow Deep Source Analysis Penetration Testing Vulnerability Assessment Metrics / Trending Reporting Security Awareness Remediation for Developers Role Based Security Process Tool integration

OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 SDLC Enhancements

OWASP & WASC AppSec 2007 Conference – San Jose – Nov Education is Essential Mavens Java C++ Mainframe All Developers Proj. Mgrs. Portfolio Leaders CIO StakeholderEducation Content Secure Programming Techniques & Tools Java static code analysis C++ static code analysis Mainframe security techniques OWASP Top 10, Tool training Managing defect removal and remediation within budget Techniques for reducing the vulnerabilities per line of code Industry practices in Security

OWASP & WASC AppSec 2007 Conference – San Jose – Nov Soft Skills in Implementation  Stakeholder analysis  Transparency of vulnerability information PhasePhase Business Unit Design Development Requirements Testing Production S S S SRR R RR R=Resister S= Supporter N=Neutral S S S NN N S S S S S S S N N NS N N N R

OWASP & WASC AppSec 2007 Conference – San Jose – Nov Accountability Model KPIs CIO Portfolio Leaders Project Team

OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 KPIs Lifecycle Phase Description Comments or Formula Month Initiation Respond to project in 3 business days Date out - Date in 93.33% 14 out of 15 PSA are responded in 3 days Initiation KPI reflects number of projects in which CIS Consulting team advised to change the response due to knowledge of systems. # PSA corrected for responses/Total #PSA Requires process change and changes to Portal 5.88% One PSA was corrected Build KPI reflects the percentage of Code scanned via Vulnerability scanner # LOC scanned/ # LOC 50%

OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 KPI’s (Continued) KPISuccess CriteriaFrequencyPhase % of Projects with security exceptions in production Green <3% = Green Yellow 3-5%= Yellow >5%= Red MonthlyIntegrated Testing Security Maven Monthly Meetings attendance Green >85% = Green Yellow 60-85%= Yellow <60%= Red MonthlyAll % of Projects completing static code scan checkpoint Green >90% = Green Yellow 75-90%= Yellow <75%= Red MonthlyBuild

OWASP & WASC AppSec 2007 Conference – San Jose – Nov Economics of Defect Management  Root cause of security challenges:  Gartner- 75% of breaches due to security flaws in software  NIST- 92% of vulnerabilities are in software  The cost of fixing a bug in the field is approximately $30,000 vs. $5,000 during coding (NIST, “The Economic Impacts of Inadequate Infrastructure for Software Testing 2002” )  “Software development organizations that perform security code reviews will experience a 60% decrease in critical vulnerabilities found in production environments” Gartner, April 2006

OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 “Fuzzing” Is More Difficult than it Sounds  Fuzz testing has emerged as a highly useful testing technique to add to the SDLC  Black box tools are useful but Fuzz testing typically addresses application design issues in a more holistic approach  Gaming skills may be transferable to fuzz testing  Using external expertise makes sense  Organically growing this skill is difficult

OWASP & WASC AppSec 2007 Conference – San Jose – Nov A Comprehensive program requires more than tools Product Deployment ADS Program SDLC Integration Security Vulnerability Management Program [Reporting] Static Code Scanning -Fortify Data Base Scanning & Config. – AppDetective/AppRadar Web Integration Testing - AppScanHigh Risk End-to-End Pen Test - Primeon SILC/Clasp Methodology Integration Standard Application Security Logging CIS Project Level Support & Code Reviews ASAR Reengineering [Design] Code complexity scan & review CAST COTS Code Assessment - Veracode Website Vulnerability Scanning- WhiteHat = In implementation

OWASP & WASC AppSec 2007 Conference – San Jose – Nov 2007 Resources OWASP Top 10 Web Application Vulnerabilities Sources Services