1. How to Operationalize Big Data Security Analytics
Roy Wilds
Field Data Scientist
Interset.AI

2. We uncover the threats that matter.
About Interset
75 employees & growing
450% ARR growth
Data science & analytics focused on cybersecurity
100 person-years of Anomaly Detection R&D
Offices in Ottawa, Canada & Newport Beach, California
Welcome
About Me
Data miner scientist since 2006
4+ years building machine learning systems for threat hunting
8 years experience using Hadoop for large scale advanced analytics
Field Data Scientist
Identify valuable data feeds
Optimize system for use cases
Partners
We uncover the threats that matter.

3. What is AI-Based Security Analytics About?
Advanced analytics to help you catch the bad guys

4. Increasing Threat Hunting Efficiency
Low Success Rate SOC Cycle
Generate Highly Anomalous Threat Leads z z

5. Increasing Visibility by Augmenting Existing Tools
SIEM
SECURITY ANALYTICS
SIEM
IAM
ENDPOINT
NETWORK
DLP
IAM
ENDPOINT
BUSINESS APPLICATIONS
CUSTOM DATA
NETWORK
DLP

6. Case Study #1: Every SOC Data, Data, Data!
Users, machines, files, projects, servers, sharing behavior, resource, websites, IP Addresses, and more
5,210,465,083
Billions of events analyzed with machine learning
Anomalies discovered by data science
High-quality “most wanted” list

7. Lesson #1: Fewer Alerts, Not Morez
Solution should help you deal with fewer alerts, not more alerts
Solution should leverage sound statistical methods to reduce false positives and noise
Should allow you to do more with the limited resources you have
Recommendations
Measure and quantify the amount of work effort involved with and without the security analytics system

8. Field Examples Telecom Healthcare Defense Potential Data Staging/Theft
Account Compromise
Lateral Movement Indicators
Healthcare
Data Theft
Defense
Incident Response

9. Case Study #2: Large Telco
The Situation
Highly secure & diverse environment – protected by multiple security products
The Challenge
Large rule/policy set developed
Too many indicators to optimize threat leads
Inefficient SOC cycle
USB
Sudden increase in file copy volumes
The Solution
Surface mathematically valid leads – “legit anomalies”
Unique normal baselines – removes threshold/rule limitations
Google Drive
Permissive controls
Personal/external sharing
Authentication
Sudden change in workstation access
Odd working hours

10. Lesson #2: The Math Matters – Test It
USB
Sudden increase in file copy volumes z
Data Theft
Data Staging
Google Drive
Permissive controls
Personal/external sharing
Lateral Movement
Account Compromise
Authentication
Sudden change in workstation access
Odd working hours
Recommendations
Agree on the use cases in advance
Use a proof-of-concept with historical/existing data to test the SA’s math
Engage red team or pen testing if available
Evaluate the results: Do they support the use cases?

11. Case Study #3: Healthcare Records & Payments
Profile: 6.5 billion transactions annually, 750+ customers, 500+ employees
Team of 7: CISO, 1 security architect, 3 security analysts, 2 network security
Analytics surfaced (for example) an employee who attempted to move “sensitive data” from endpoint to personal Dropbox
Employee was arrested and prosecuted using incident data
Focus and prioritized incident responses
Incident alert accuracy increased from 28% to 92%
Incident mitigation coverage doubled from 70 per week to 140

12. Lesson #3: Meaningful Metrics
Hawthorne Effect: Whatever gets measured, gets optimized
Recommendations
Define meaningful operational metrics (not just “false positives”)
Build a process for measuring and quantifying over time, not just during a pilot
Ensure the security analytics system supports a feedback process to adjust the analytics to support your target metrics

13. What Have We Learned? Lessons Learned Recommendations
The Math Matters – Test It
Fewer Alerts, Not More
Automated, Measured Responses
Meaningful Metrics
Recommendations
Agree on the use cases in advance
Evaluate results with and without security analytics system
Assess risk level, not binary alert
Ensure integrated feedback and automated response

14. Roy Wilds – Field Data Scientist @roywilds
QUESTIONS?
Learn more at Interset.AI
Roy Wilds – Field Data Scientist
@roywilds