1. False Positives in Static Analysis * The Good, the Bad, and the Ugly Andy Earle Hewlett-Packard Enterprise Security Solutions Architect

2. Who am I? Andy Earle CISSP, CSSLP, CEH HPE Fortify Application Security Solutions Architect / Presales Engineer Sell, deliver solutions to commercial and US Federal accounts Demos, Proofs-of-Concept, Technical Discussions, speaking engagements Previous Product Manager for High Assurance computer system at BAE Systems Mobile and App Security, multiple jobs Software Engineer and Consultant, multiple jobs

3. Agenda Background Static Analysis Dataflow Analysis False Positives What are they? Why do we encounter them? Specific examples What to do about them ???

4. BACKGROUND * STATIC ANALYSIS & DATAFLOW ANALYSIS

5. 5 Source Code Analyzers Automated scanning tools A computer inspects the application for security vulnerabilities Static analysis targets source code and/or compiled code The Machine is Faster than the Man (or Woman) Automated static analysis provides… Thorough coverage of the application Relatively ease of deployment and use Greatly reduced human error …But, does not fully remove the human

6. Discovering Attacks Static analysis incorporates different types of analyses. To name a few Dataflow Control Flow Semantic Structural More… Dataflow: a particularly crucial analysis technique, as it finds various injection attacks, which are mostly very nasty vulnerabilities. Via Dataflow Analysis

7. Injection Attack Injection flaws occur when an application sends untrusted data to an interpreter Attack Vector Attacker sends simple text-based attacks that exploit the syntax of the targeted interpreter. Almost any source of data can be an injection vector, including internal sources. Technical Impacts Injection can result in data loss or corruption, lack of accountability, or denial of access. Injection can sometimes lead to complete host takeover. Defined (source: OWASP.org)

8. Dataflow Analysis What Find vulnerabilities where non-trusted input can potentially control application operation. Vulnerabilities to injection attacks How? Analyzer uses global taint propagation to trace the flow of non-trusted data Taint: Non-trusted (user controlled) input Source: Location of entry of non-trusted input (taint) Sink: Potentially dangerous function call or operation (location in app where the damage is done) Static Analysis Method to Find Injection Vulnerabilities

9. Data Flow Analysis userName = request.getParameter(“user_id");... query = "select uname, passwd from users where uname like "+userName+"%";... stmnt = conn.createStatement ();... rs = stmnt.executeQuery (query); Tracking a SQL Injection Vulnerability

10. Data Flow Analysis userName = request.getParameter(“user_id");... query = "select uname, passwd from users where uname like "+userName+"%";... stmnt = conn.createStatement ();... rs = stmnt.executeQuery (query); Tracking a SQL Injection Vulnerability Source

11. Data Flow Analysis userName = request.getParameter(“user_id");... query = "select uname, passwd from users where uname like "+userName+"%";... stmnt = conn.createStatement ();... rs = stmnt.executeQuery (query); Tracking a SQL Injection Vulnerability userName is tainted

12. Data Flow Analysis userName = request.getParameter(“user_id");... query = "select uname, passwd from users where uname like "+userName+"%";... stmnt = conn.createStatement ();... rs = stmnt.executeQuery (query); Tracking a SQL Injection Vulnerability Taint is propogated to query userName is tainted

13. Data Flow Analysis userName = request.getParameter(“user_id");... query = "select uname, passwd from users where uname like "+userName+"%";... stmnt = conn.createStatement ();... rs = stmnt.executeQuery (query); Tracking a SQL Injection Vulnerability Finding! Taint hits the Sink

14. FALSE POSITIVES * WHAT AND WHY?

15. False Positive A static code analysis tool will often produce false positive results where the tool reports a possible vulnerability that in fact is not. This often occurs because the tool cannot be sure of the integrity and security of data as it flows through the application from input to output. False positive results might be reported when analyzing an application that interacts with closed source components or external systems because without the source code it is impossible to trace the flow of data in the external system and hence ensure the integrity and security of the data. Source: https://www.owasp.org/index.php/Static_Code_Analysishttps://www.owasp.org/index.php/Static_Code_Analysis Defined

16. False Positive …Possible vulnerability that in fact is not… Why is it not? If not, why did the analyzer report it? If we label it a False Positive, is there any value in the finding? What can we do to deal with lots of false positives? Things to Ponder

17. Vulnerability Analysis Philosophy When searching for vulnerabilities… Avoiding False Negatives is more important than reporting False Positives Find and report everything, even low likelihood suspicions Examine all I/O, including non-user input The context of the application is critical to interpreting vulnerability findings A single vulnerability may be all it takes… …or a few seemingly low impact findings… Basic Tenants of finding security flaws with a machine

18. False Positive? userName = “chewbacca”;... query = "select uname, passwd from users where uname = ‘“ + userName + “’"; stmnt = conn.createStatement (); rs = stmnt.executeQuery (query); Finding: SQL InjectionTrue Finding?False Positive Why? Bug in analyzer Sample #1

19. False Positive? rs = statement.executeQuery(query); ssn = rs.getString(zipcode); // write to log Logger.log(SOME_MESSAGE + ssn); Finding: Privacy ViolationTrue Finding?False Positive Why? Lack of context (social security # logged) Sample #2

20. False Positive? userName = request.getParameter(“userName"); addr = missingobject.doSomething(userName); // write to webpage print(addr); Finding: XSS ReflectedTrue Finding?Possible false positive Why? Analyzer unsure (Hint: missingobject is AWOL) Sample #3

21. Can I trust val? False Positives Explained // calls into unknown object missing.doIt(cleanValue); missing.doIt(taintedValue); // read from unknown object // can I trust this? val = missing.getVal(); Missing File/obj/code cleantainted val Assume tainted The Effect of Missing Pieces / Lost Sinks

22. False Positive? sql = file.readFromFile(“sql_stmts.xml”, UPDATE_PO_SQL); po = request.getParameter(“po_num”); stmt = conn.prepareStatement(sql); stmt.setString(1, po); results = stmt.execute(); Finding: SQL InjectionTrue Finding?Likely “False Positive” Why? Trusted input(?) (sql originates from file) Sample #4

23. False Positive Some (not all) False Positive “explanations”… Bug in the analyzer or rule set Lack of context: an incorrect or irrelevant best guess Missing component(s) of application in scan (black holes) Source of the “untrusted” input is trustworthy Review

24. FALSE POSITIVES * DEALING WITH THEM…

25. Dealing with False Positives Bug in the product Mark as “Not an Issue” and include comments Suppress/filter/deprioritize the issue Report the bug to the vendor Missing Components Get them! And rescan Refine your process, so you get everything every time Integrate with development infrastructure (CI, VCS, IDEs, etc.) Get development and security to work together We understand a bit more, but So What?!

26. String pwd; // (password) keep private String pass_key; // (lookup key for password) not private String pwd_encr; // (encrypted password) not private? Dealing with False Positives Broad category…not a one size fits all answer Coding standards, naming conventions Threat modeling Consider Application Type (web, console, batch, client/server, internet, firewalled, etc.) In many cases, these simply require manual review Coding Standards (examples) Lack of Context: analyzer takes a (conservative) guess

27. Dealing with False Positives Others in this category… It is the way the application is designed to work (i.e. mysql client SQLi vulnerability) User is trusted, or has the privilege to execute the “attack” anyway (i.e. admin, DBA) No impact if vulnerability is exploited (judgment call) These can be dealt with in bulk Use metadata of the issues Threat modeling to understand the attack surface and risks Use filters, groupings and overlays in the results Establish and model policy that weeds out issues unimportant to you Custom rules/checks? (model missing libraries; unrecognized sources, sinks, cleansing) Source of Input is Trustworthy

28. Use your Metadata Vulnerability Category – do I care? Source – Where did the data originate? (user, DB, registry, etc.) – Do I trust that source? Where in the app is the issue? – Would that part of the app be susceptible to the attack? What do the scoring or metrics tell you? – Example: likelihood Are there external controls that defend against it?

29. Dealing with False Positives SQL Injection Policy Where (injection) source is not end user input, set to Low priority [use the data source] If Administration section, set to Medium priority [I am less concerned, since Administrators have heightened trust] Where (injection) source is file system, set as “false positive” [I trust what is on my server; proper controls are in place] Examples of Proactively using Meta-data for cleaner results

30. Static Analysis Flow Audit and Triage Scan for Real Tweak Scan Settings Assess Scan Success Baseline Scan Initial scan Goal is to get through a complete scan Addressing False Positives Proactively

31. Static Analysis Flow Audit and Triage Scan for Real Tweak Scan Settings Assess Scan Success View logs Review results Determine integrity and accuracy of scan Baseline Scan Initial scan Goal is to get through a complete scan Addressing False Positives Proactively

32. Static Analysis Flow Audit and Triage Scan for Real Tweak Scan Settings Set up filters Grab missing pieces of code and libraries Custom rules? Assess Scan Success View logs Review results Determine integrity and accuracy of scan Baseline Scan Initial scan Goal is to get through a complete scan Addressing False Positives Proactively

33. Static Analysis Flow Audit and Triage Scan for Real Goal is a complete and accurate scan Tweak Scan Settings Set up filters Grab missing pieces of code and libraries Custom rules? Assess Scan Success View logs Review results Determine integrity and accuracy of scan Baseline Scan Initial scan Goal is to get through a complete scan Addressing False Positives Proactively

34. Static Analysis Flow Audit and Triage Filter further as needed Build reusable filters and project templates FIX issues! Scan for Real Goal is a complete and accurate scan Tweak Scan Settings Set up filters Grab missing pieces of code and libraries Custom rules? Assess Scan Success View logs Review results Determine integrity and accuracy of scan Baseline Scan Initial scan Goal is to get through a complete scan Addressing False Positives Proactively

35. Dealing with False Positives Establish a software security policy Understand your threats, attack surface, risks, etc. Understand your app: context Model your policies and compliance obligations into your technologies Filters and Groupings, based on meta-data Custom rules/checks Start slow and build Systemic Strategy

36. Resources http://www.owasp.org http://www.opensamm.org/ http://bsimm.com/ https://buildsecurityin.us-cert.gov/ …Many, many others…

37. Thank You! Andy Earle Security Solutions Architect andy.earle@hpe.com