1. OWASP ASVS Project Discussion & Reflection 1

2. The case study This is a real open source CMS system, albeit an older release from 2012 The only intentional defect in the code: re-inclusion of one vulnerability from the previous release So none of the code was not intentionally bad... [16] 2

3. The process Dividing the work by requirements / the code / tool? Or “agile” [5]? And/or by expertise or interest of the group members? Tools to support group process – gitlab, OWAAT..? Tools to support inspecting the code – editors & IDEs, eg Doxygen, PHPDoc, IntelliJ, PHPStorm...? Looking at running wesite as well as code? Ideally you’d like to check all this while developing the code, but then there may still have to be independent security evaluation afterwards 3

4. Attacker model One group considered diffferent attacker models – A: insider attacks – B: outsider attacks Shouldn’t attacker model get some attention in ASVS process? 4

5. test vs code review 5

6. Static Code Analysis Tools RATS, RIPS, Fortify, Checkmarx Usefulness? only “basic and blatant flaws”? (m)any true positives? unworkably many false positives? comparison [4] covering only small subset of possibel problems? [17] Improvements? more intelligence in the tool beyond just syntax (eg for “exec”) but: limits in understanding meaning/semantics by any tool 6

7. Dynamic analysis tools? OWASP ZAP Wapiti DirBuster sqlmap Skipfish Nessus OpenVAS... 7

8. code vs deployment/configuration/server source code of the application could be fine, but configuration could still screw things up in defence of ASVS: not meant for evaluation using just source code review Other ‘scoping’ issues: – do we have to look at libraries, frameworks,...? – what about dead code/unused part? (eg Curl) 8

9. ASVS problems missing requirements? unclear requirements/formulations? [16] – more explanation? – more context needed about the application policies & templates and business rules requirements with very wide scope? different leves of importance/impact? not giving enough hints on how to organize things – how rather than what (Some requirements, eg V1, only existed in earlier ASVS releases & since removed) 9

10. ASVS improvements it should also describe how to verify things – how rather than what distinguish – different levels of certainty – different levels of importance/relevance/impact Missing requirements, eg “SQL truncation & crypto setting”? [1] Should the ASVS specify when specific protection mechanism against say clickjacking (V11.8) is good enough? [14] Should the ASVS make explicit link with OWASP Top 10? [6] As side-product of doing a security assessment, produce a checklist with dependencies on libraries, interpreters, runtime, etc, so that anyone deploying the system knows that for these they should keep of security issues, updates, etc 10

11. NA Different meanings of Not Applicable (NA) and Not Relevant (NR) are used NA, so trivially passed NA, hence clearly failed we don’t really know – because the requirement is not clear, or – because the code is not clear we can’t really know, as this depends – on the configuration or the web server – on the context: policies and business rules 11

12. RSS issue with an empty line [2] Whose fault is this? 12

13. TestCMS 13

14. PHP 14

15. security assessment & assurance Different ways to do security assessment & get some assurance: 1.pen-test 2.doing OWASP ASVS review like you did 3.simply running Fortify & Checkmarx Would these draw similar conclusions about the overall security? finding the same/similar security flaws? draw same/similar conclusions & recommendations? 15

16. Anything else? about tools process ASVS TestCMS... 16

17. This group project itself did you learn anything/enough? is this a effective way to learn anything? 17