Agile Security Testing German OWASP Day 2016 Matthias Rohr

Introduction Matthias Rohr Active in AppSec > 12 years Professional Focuses: Building Secure Web-based (Java) Applications Secure SDLC Security Test Automation

How App Dev & Delivery Changed… 1. Waterfall Design e.g. 1-2 releases a year Code Test Deploy Go Live 2. Agile (e.g. SCRUM) without Continuous Deployment 2-4 weeks per Sprint e.g. 2-4 releases each YEAR Design Code Test Code Test Code Test Deploy … Go Live 3. Agile with Continuous Deployment (DevOps) … Design 2-4 weeks per Sprint e.g. 1-2 releases each DAY Go Live Go Live Go Live Go Live Go Live Go Live Go Live Based on a figure of Dephix

How Test Artefacts Changed… Monolithic Architecture Microservice Architecture Source: continoudautomation.com

Agile Security Touch Points Team Responsibilities & Agile Practices => Goals: (1) Make teams responsible for (their) security (2) Implement security activities that work in agile development Secure Foundation => Goals: (1) Prevent vulnerabilities (e.g. architecture, secure defaults) (2) Continuous improvement of product security / hardening Security Test Automation => Goal : Integrate security checks into build & delivery pipeline to allow contiuous security testing

Secure Foundation vs. Security Testing Strengthen the Security Foundation of a product can result in concideratable less need for security testing! Need for Security Tests Secure Foundation (Defaults, Architecture, etc.)

AppSec Test Layers Vulnerabilities in Application Layer Session Mgmt / CSRF, Insecure client-side, Business Logic Errors, Missing Anti Autom., … XSS, SQL Injection, Datavalidation Authentication, Access Controls, Cryptography, Inform. Leakage, Error Handling, Configuration, … Race Conditions Buffer Overflows Backdoors, Insecure APIs, Backend, … Vulnerabilities in Code Layer

AppSec Test Layers Application or Code Layer is not enough, Vulnerabilities in Application Layer Application or Code Layer is not enough, we need to test BOTH! Session Mgmt / CSRF, Insecure client-side, Business Logic Errors, Missing Anti Autom., … XSS, SQL Injection, Datavalidation Authentication, Access Controls, Cryptography, Inform. Leakage, Error Handling, Configuration, … Race Conditions Buffer Overflows Backdoors, Insecure APIs, Backend, … Vulnerabilities in Code Layer

Shifting Left ... of Security (Secure SDLC) of Operational Concerns (DevOps) of Operational Security Concerns (part of SecDevOps) DevOps teams need to run security tests on system / network layer as well

(OpenSource) AppSec Tool Chain I Network Layer Security Scans (for DevOps) Docker Bench/CIS, Nmap, sslyze, bats, etc. Application Layer Security Scans Web UI Non-Functional: DAST Scanner: Arachni, OWASP ZAP, w3af, Dirbuster, gamr, wpscan, Gauntlt/Mittn …. Functional: Security Integration Tests with HTTPClient, Selenium/WebDriver, BDD-Security Rest Services (Web APIs) Functional: soapUI TestRunner, HTTPClient, etc. Non-Functional: ??? (Radamsa)

(OpenSource) AppSec Tool Chain II 3. Code Layer Security Scans Java: Custom Code (SAST): Findbugs Sec Plugin 3rd Party Code: OWASP Dependency Check Functional: JUnit JavaScript Custom Code (SAST): ScanJS 3rd Party Code: Retire JS Functional: Unit JS, QUnit Ruby on Rails Custom Code (SAST) Brakeman ….

Tool Conciderations You use a lot of languages = You need a lot of (OSS) tools!* You use a lot of tools = You get a lot of findings You need to do a lot of (constant) customizations! * Commercial SAST solutions often provide support multiple languages

Example 1: Integrated AppSec Scans in CI Pulls Code Pushes New Code Git Repo Buid Job for App X Deploy OWASP ZAP Plugin Developer Scans Test App Server OWASP Dependency Checker Plugin Analyzes Results Problems: Sec. tool config overhead Management of Findings Increased Build Time!!! Project CI

Example 2: Dedicated AppSec Pipelines Git / SVN Repo Local Scans Pushes new Code Developer Pulls Code Security Projekt-Specific Sec Checks (every build) Scans Project Security Pipeline (every 24h) Results Test App Server Vulnerability Repository (e.g. Management of False Positives) Jenkins CI Problems: Sec. tool config overhead Management of Findings

Screenshot

Code Firewalls Implement simple sec rules via Git Post Receive Hooks: $ git push origin master treating file insecure.js [master 530596e] adding insecure.js … Writing objects: 100% (2/2), 237 bytes | 0 bytes/s, done. Total 2 (delta 1), reused 0 (delta 0) remote: [SEC] Scanning File insecure.js remote: [SEC] Scanning with JavaScript rules remote: [SEC] Insecure eval() function found in file insecure.js

Example 3: AppSec Scan Factory Local Scans Code Firewall Git / SVN Repo Developer Pulls Code Projekt-Specific Sec Checks (every build) Results P: Git / SVN Uri P: Build Flags Java SAST Pipeline Project Security Pipeline (every 24h) DAST Pipeline P:URL Security Project CI Security CI Scans Test App Server Problems: Not easy to set-up

IAST (Commercial) IAST = Dynamic Code Scanning at Runtime Combination of DAST and SAST approach Some products analyze code completely passively during normal testing (= no extra security testing required) Testserver with IAST Agents IAST Management Server Developer / Tester Developer / Acceptance Testing Findings Review Findings

Agile Security Organization Agile Security is at first an organizational problem Agile Teams must be made responsible for security too! Agile Teams InfoSec Function AppSec Team ? ? ? Test Factory Security Champion ?

Examples for Security Test Execution By Teams: E.g. tests of changes by Security Champion By internal AppSec Team / Test Factory: Often best solution for limited security tests External: E.g. Periodic full site assessments outside of Sprints

Pentest and Agile Security Create Security (JIRA) Artefacts for all Security Test Activities Create separate stories & tasks for tests activities Cover insecure business logic with Evil User Stories and test them Split large test activities into smaller ones that fit into one Sprint (e.g. „Pentest of Functionality X“)

Security & Sprint Planning PM needs to be aware of security relevant changes that require a security activity (e.g. a Pentest) => Define Criterias Tip: Bundle security activities in Security Sprints! Daily (24h) Potential Shippable Product Increment Product Backlog Sprint Backlog Sprint (2-4 weeks) DoD Optional Pentests, etc. Continuously Improve Product Attack Resistance Identify Risks Sprints Activities Automated Security Scans

Agile Security Gates Definition of Done (DoD) Define required security activities in a Sprint for applications (e.g. review scan results) and integrate them in Definition of Done (DoD) => implicit security gate Peer Reviews Implement implicit peer reviews (e.g. with Gerrit or Crucible) for changes of critical code such as auth controls => implicit security gate Sign-Offs Implement explicit security sign-offs for critical changes or infrequent releases

Recommendation I Don‘t just integrate various scanning tools Instead, focus on: What needs to be tested? Can we implement controls that make these tests redundant? Can we automate it? If so, Where? (e.g. within delivery pipeline) What are suitable tools for that? How often should be tested? (e.g. with every build) How are results analyzed and who is doing this? Start with baseline scans and extend it step by step

Recommendation II Don‘t rely on (non-functional) security scanners! Instead: Teach your agile teams how prevent vulnerabilities & make it their responsibility too Implement a strong security foundation and continuously improve it Provide sound security guidance & requirements Focus on precise & fast security tests to test those

Matthias Rohr Secodis GmbH m.rohr@secodis.com Thank you! Questions? Matthias Rohr Secodis GmbH m.rohr@secodis.com