How to Operationalize Big Data Security Analytics Roy Wilds Field Data Scientist Interset.AI

We uncover the threats that matter. About Interset 75 employees & growing 450% ARR growth Data science & analytics focused on cybersecurity 100 person-years of Anomaly Detection R&D Offices in Ottawa, Canada & Newport Beach, California Welcome About Me Data miner scientist since 2006 4+ years building machine learning systems for threat hunting 8 years experience using Hadoop for large scale advanced analytics Field Data Scientist Identify valuable data feeds Optimize system for use cases Partners We uncover the threats that matter.

What is AI-Based Security Analytics About? Advanced analytics to help you catch the bad guys

Increasing Threat Hunting Efficiency Low Success Rate SOC Cycle Generate Highly Anomalous Threat Leads z z

Increasing Visibility by Augmenting Existing Tools SIEM SECURITY ANALYTICS SIEM IAM ENDPOINT NETWORK DLP IAM ENDPOINT BUSINESS APPLICATIONS CUSTOM DATA NETWORK DLP

Case Study #1: Every SOC Data, Data, Data! Users, machines, files, projects, servers, sharing behavior, resource, websites, IP Addresses, and more 5,210,465,083 Billions of events analyzed with machine learning Anomalies discovered by data science High-quality “most wanted” list

Lesson #1: Fewer Alerts, Not More z Solution should help you deal with fewer alerts, not more alerts Solution should leverage sound statistical methods to reduce false positives and noise Should allow you to do more with the limited resources you have Recommendations Measure and quantify the amount of work effort involved with and without the security analytics system

Field Examples Telecom Healthcare Defense Potential Data Staging/Theft Account Compromise Lateral Movement Indicators Healthcare Data Theft Defense Incident Response

Case Study #2: Large Telco The Situation Highly secure & diverse environment – protected by multiple security products The Challenge Large rule/policy set developed Too many indicators to optimize threat leads Inefficient SOC cycle USB Sudden increase in file copy volumes The Solution Surface mathematically valid leads – “legit anomalies” Unique normal baselines – removes threshold/rule limitations Google Drive Permissive controls Personal/external sharing Authentication Sudden change in workstation access Odd working hours

Lesson #2: The Math Matters – Test It USB Sudden increase in file copy volumes z Data Theft Data Staging Google Drive Permissive controls Personal/external sharing Lateral Movement Account Compromise Authentication Sudden change in workstation access Odd working hours Recommendations Agree on the use cases in advance Use a proof-of-concept with historical/existing data to test the SA’s math Engage red team or pen testing if available Evaluate the results: Do they support the use cases?

Case Study #3: Healthcare Records & Payments Profile: 6.5 billion transactions annually, 750+ customers, 500+ employees Team of 7: CISO, 1 security architect, 3 security analysts, 2 network security Analytics surfaced (for example) an employee who attempted to move “sensitive data” from endpoint to personal Dropbox Employee was arrested and prosecuted using incident data Focus and prioritized incident responses Incident alert accuracy increased from 28% to 92% Incident mitigation coverage doubled from 70 per week to 140

Lesson #3: Meaningful Metrics Hawthorne Effect: Whatever gets measured, gets optimized Recommendations Define meaningful operational metrics (not just “false positives”) Build a process for measuring and quantifying over time, not just during a pilot Ensure the security analytics system supports a feedback process to adjust the analytics to support your target metrics

What Have We Learned? Lessons Learned Recommendations The Math Matters – Test It Fewer Alerts, Not More Automated, Measured Responses Meaningful Metrics Recommendations Agree on the use cases in advance Evaluate results with and without security analytics system Assess risk level, not binary alert Ensure integrated feedback and automated response

Roy Wilds – Field Data Scientist @roywilds QUESTIONS? Learn more at Interset.AI Roy Wilds – Field Data Scientist @roywilds