1. Blending Automated and Manual Testing Making Application Vulnerability Management Pay Dividends

2. My Background Dan Cornell, founder and CTO of Denim Group Software developer by background (Java,.NET, etc) OWASP San Antonio @danielcornell

3. My Background Steve Springett, Application Security Architect for Axway Software developer by background Leader of OWASP Dependency-Track Contributor to OWASP Dependency-Check @stevespringett

4. Goal: Continuous Security Prerequisites – Standardization – Continuous Integration – Continuous Delivery Compliments – Continuous Acceptance

5. Standardization All projects use same build system All projects built the same way Automated onboarding for new projects Per-project build expertise not required

6. Metrics Artifacts Continuous Integration Continuous Integration Factory Source Code (SCM)

7. Deliverables Continuous Delivery Continuous Delivery Factory Artifacts

8. Security Metrics Continuous Security Continuous Security Factory Source Code (SCM) Deliverables

9. Automated Security Metrics Static Analysis Findings Dynamic Analysis Findings Component Analysis Findings Attack Surface Analysis Findings

10. Continuous Security Pipe Jenkins CI ThreadFix Defect Tracker SCM False Positive

11. Target Application

12. 12 ThreadFix Accelerate Software Remediation ThreadFix is a software vulnerability aggregation and management system that helps organizations aggregate vulnerability data, generate virtual patches, and interact with software defect tracking systems.

13. ThreadFix Open Source (MPL) application vulnerability management platform Create a consolidated view of your applications and vulnerabilities Prioritize application risk decisions based on data Translate vulnerabilities to developers in the tools they are already using

14. ThreadFix Community Edition Main ThreadFix website: www.threadfix.orgwww.threadfix.org – General information, downloads ThreadFix GitHub site: www.github.com/denimgroup/threadfixwww.github.com/denimgroup/threadfix – Code, issue tracking ThreadFix GitHub wiki: https://github.com/denimgroup/threadfix/wiki https://github.com/denimgroup/threadfix/wiki – Project documentation ThreadFix Google Group: https://groups.google.com/forum/?fromgroups#!forum/threadfix https://groups.google.com/forum/?fromgroups#!forum/threadfix – Community support, general discussion

15. Vulnerability Aggregation Automated Manual

16. Access to Vulnerability Data Tradeoffs – The more places the vulnerability data lives, the more likely a compromise – Withholding information from people who need it makes remediation more challenging

17. Managing All Vulnerability Data Manual activities – Penetration Testing – Code Reviews 3 rd Party Data Sources – Customer-performed Testing – External auditor-performed Results

18. SSVL and Manual Results SSVL Data Format: – https://github.com/owasp/ssvl https://github.com/owasp/ssvl SSVL Conversion Tool: – https://github.com/denimgroup/threadfix/wiki/SSVL-Converter https://github.com/denimgroup/threadfix/wiki/SSVL-Converter

19. RESTful API to Vulnerability Data Custom R&D Monitoring Dashboard Custom Dashboards

20. Key Performance Indicators Don’t go overboard – Use only what is needed Progress and velocity Per team comparison Min/max/avg time to close per severity By CWE

21. Lessons Learned Always automate static analysis Always automate attack surface analysis Always automate component analysis Always automate dynamic analysis Always perform manual dynamic analysis Use native tools & workflow for static analysis

22. Lessons Learned Provide as much visibility as possible – Varying degrees of detail – Multiple delivery vehicles Set clear pass/fail criteria for Security Bars – Provide custom dashboard to provide status and advanced warning

23. Additional Advice Automation is not better than manual – It’s faster and more efficient – Both are necessary Don’t forget manual assessments – Threat Modeling – Secure Design/Architecture and Code Review – Penetration Testing

24. Finally Vulnerabilities in CI / CD / CS Infrastructure – Threat Model – Secure Architecture Review – Patch Management – Configuration Management – Key Management – Always use TLS

25. Q & A