1. Agile Security Testing
German OWASP Day 2016
Matthias Rohr

2. Introduction Matthias Rohr Active in AppSec > 12 years
Professional Focuses:
Building Secure Web-based (Java) Applications
Secure SDLC
Security Test Automation

3. How App Dev & Delivery Changed…
1. Waterfall
Design
e.g. 1-2 releases a year
Code
Test
Deploy
Go Live
2. Agile (e.g. SCRUM) without Continuous Deployment
2-4 weeks per Sprint e.g. 2-4 releases each YEAR
Design
Code
Test
Code
Test
Code
Test
Deploy …
Go Live
3. Agile with Continuous Deployment (DevOps) …
Design
2-4 weeks per Sprint e.g. 1-2 releases each DAY
Go Live
Go Live
Go Live
Go Live
Go Live
Go Live
Go Live
Based on a figure of Dephix

4. How Test Artefacts Changed…
Monolithic Architecture Microservice Architecture
Source: continoudautomation.com

5. Agile Security Touch Points
Team Responsibilities & Agile Practices => Goals: (1) Make teams responsible for (their) security (2) Implement security activities that work in agile development
Secure Foundation => Goals: (1) Prevent vulnerabilities (e.g. architecture, secure defaults) (2) Continuous improvement of product security / hardening
Security Test Automation => Goal : Integrate security checks into build & delivery pipeline to allow contiuous security testing

6. Secure Foundation vs. Security Testing
Strengthen the Security Foundation of a product can result in concideratable less need for security testing!
Need for Security Tests
Secure Foundation (Defaults, Architecture, etc.)

7. AppSec Test Layers Vulnerabilities in Application Layer
Session Mgmt / CSRF, Insecure client-side, Business Logic Errors,
Missing Anti Autom., …
XSS,
SQL Injection,
Datavalidation
Authentication,
Access Controls,
Cryptography, Inform. Leakage,
Error Handling,
Configuration, …
Race Conditions
Buffer Overflows Backdoors,
Insecure APIs,
Backend, …
Vulnerabilities in Code Layer

8. AppSec Test Layers Application or Code Layer is not enough,
Vulnerabilities in Application Layer
Application or Code Layer is not enough,
we need to test BOTH!
Session Mgmt / CSRF, Insecure client-side, Business Logic Errors,
Missing Anti Autom., …
XSS,
SQL Injection,
Datavalidation
Authentication,
Access Controls,
Cryptography, Inform. Leakage,
Error Handling,
Configuration, …
Race Conditions
Buffer Overflows Backdoors,
Insecure APIs,
Backend, …
Vulnerabilities in Code Layer

9. Shifting Left ... of Security (Secure SDLC)
of Operational Concerns (DevOps)
of Operational Security Concerns (part of SecDevOps)
DevOps teams need to run security tests on system / network layer as well

10. (OpenSource) AppSec Tool Chain I
Network Layer Security Scans (for DevOps)
Docker Bench/CIS, Nmap, sslyze, bats, etc.
Application Layer Security Scans
Web UI
Non-Functional: DAST Scanner: Arachni, OWASP ZAP, w3af, Dirbuster, gamr, wpscan, Gauntlt/Mittn ….
Functional: Security Integration Tests with HTTPClient, Selenium/WebDriver, BDD-Security
Rest Services (Web APIs)
Functional: soapUI TestRunner, HTTPClient, etc.
Non-Functional: ??? (Radamsa)

11. (OpenSource) AppSec Tool Chain II
3. Code Layer Security Scans
Java:
Custom Code (SAST): Findbugs Sec Plugin
3rd Party Code: OWASP Dependency Check
Functional: JUnit
JavaScript
Custom Code (SAST): ScanJS
3rd Party Code: Retire JS
Functional: Unit JS, QUnit
Ruby on Rails
Custom Code (SAST) Brakeman ….

12. Tool Conciderations
You use a lot of languages = You need a lot of (OSS) tools!*
You use a lot of tools =
You get a lot of findings
You need to do a lot of (constant) customizations!
* Commercial SAST solutions often provide support multiple languages

13. Example 1: Integrated AppSec Scans in CI
Pulls Code
Pushes New Code
Git Repo
Buid Job for App X
Deploy
OWASP ZAP Plugin
Developer
Scans
Test App Server
OWASP Dependency Checker Plugin
Analyzes Results
Problems:
Sec. tool config overhead
Management of Findings
Increased Build Time!!!
Project CI

14. Example 2: Dedicated AppSec Pipelines
Git / SVN Repo
Local Scans
Pushes new Code
Developer
Pulls Code
Security
Projekt-Specific Sec Checks (every build)
Scans
Project Security Pipeline (every 24h)
Results
Test App Server
Vulnerability Repository
(e.g. Management of False Positives)
Jenkins CI
Problems:
Sec. tool config overhead
Management of Findings

15. Screenshot

16. Code Firewalls Implement simple sec rules via Git Post Receive Hooks:
$ git push origin master
treating file insecure.js
[master e] adding insecure.js …
Writing objects: 100% (2/2), 237 bytes | 0 bytes/s, done.
Total 2 (delta 1), reused 0 (delta 0)
remote: [SEC] Scanning File insecure.js
remote: [SEC] Scanning with JavaScript rules
remote: [SEC] Insecure eval() function found in file insecure.js

17. Example 3: AppSec Scan Factory
Local Scans
Code Firewall
Git / SVN Repo
Developer
Pulls Code
Projekt-Specific Sec Checks (every build)
Results
P: Git / SVN Uri P: Build Flags
Java SAST Pipeline
Project Security Pipeline (every 24h)
DAST Pipeline
P:URL
Security
Project CI
Security CI
Scans
Test App Server
Problems:
Not easy to set-up

18. IAST (Commercial) IAST = Dynamic Code Scanning at Runtime
Combination of DAST and SAST approach
Some products analyze code completely passively during normal testing (= no extra security testing required)
Testserver with IAST Agents
IAST Management Server
Developer / Tester
Developer / Acceptance Testing
Findings
Review Findings

19. Agile Security Organization
Agile Security is at first an organizational problem
Agile Teams must be made responsible for security too!
Agile Teams
InfoSec Function
AppSec Team ? ? ?
Test Factory
Security Champion ?

20. Examples for Security Test Execution
By Teams: E.g. tests of changes by Security Champion
By internal AppSec Team / Test Factory: Often best solution for limited security tests
External: E.g. Periodic full site assessments outside of Sprints

21. Pentest and Agile Security
Create Security (JIRA) Artefacts for all Security Test Activities
Create separate stories & tasks for tests activities
Cover insecure business logic with Evil User Stories and test them
Split large test activities into smaller ones that fit into one Sprint (e.g. „Pentest of Functionality X“)

22. Security & Sprint Planning
PM needs to be aware of security relevant changes that require a security activity (e.g. a Pentest) => Define Criterias
Tip: Bundle security activities in Security Sprints!
Daily (24h)
Potential Shippable Product Increment
Product Backlog
Sprint Backlog
Sprint (2-4 weeks)
DoD
Optional Pentests, etc.
Continuously Improve Product Attack Resistance
Identify Risks
Sprints
Activities
Automated
Security Scans

23. Agile Security Gates
Definition of Done (DoD) Define required security activities in a Sprint for applications (e.g. review scan results) and integrate them in Definition of Done (DoD) => implicit security gate
Peer Reviews Implement implicit peer reviews (e.g. with Gerrit or Crucible) for changes of critical code such as auth controls => implicit security gate
Sign-Offs Implement explicit security sign-offs for critical changes or infrequent releases

24. Recommendation I Don‘t just integrate various scanning tools
Instead, focus on:
What needs to be tested?
Can we implement controls that make these tests redundant?
Can we automate it? If so, Where? (e.g. within delivery pipeline)
What are suitable tools for that?
How often should be tested? (e.g. with every build)
How are results analyzed and who is doing this?
Start with baseline scans and extend it step by step

25. Recommendation II Don‘t rely on (non-functional) security scanners!
Instead:
Teach your agile teams how prevent vulnerabilities & make it their responsibility too
Implement a strong security foundation and continuously improve it
Provide sound security guidance & requirements
Focus on precise & fast security tests to test those

26. Matthias Rohr Secodis GmbH m.rohr@secodis.com
Thank you! Questions?
Matthias Rohr Secodis GmbH