Carleton University School of Computer Science Detecting Intra-enterprise Scanning Worms based on Address Resolution David Whyte, Paul van Oorschot, Evangelos Kranakis

Carleton University School of Computer Science Outline ● Internet Environment ● Scanning Worm Propagation Characteristics ● Address Resolution (ARP) Approach ● Results ● Limitations ● Conclusions

Carleton University School of Computer Science Dsheild Report –- December 6, 2005 Top Attacker: Most Attacked Port: 445

Carleton University School of Computer Science Worm Outbreaks ● Sapphire/Slammer worm – Jan 25, 2003 – Fastest spreading worm yet ● 90% compromised in first 10 minutes ● August 2003 the “Month of Worms” – SoBig.F: #1 “mass mailing” virus of all time – Blaster/LovSan/Welchia/Nachi ● Witty worm – March 2004 – Buffer overflow in a suite of security products ● Use of a hit-list?

Carleton University School of Computer Science Countermeasure Challenges ● Propagation speed renders human-based defensive strategies non-effective ● Security patches – frequent, large and sometimes broken – Slammer fix SQL Server 2000 SP2: three parts 26MB to 506MB – sql2kdeskfullsp2.exe 390 MB

Carleton University School of Computer Science Countermeasure Challenges ● Active response (i.e. containment and suppression) – risky (self imposed DoS) ● The IDS that cried “Worm!!”… – (Snort) alert UDP any any -> any 1434 (msg:"SQL Slammer Worm"; rev:1; content:"|726e51686f 756e b |";) – (Dragon) NAME=SMB:DCOM-OVERFLOW SIGNATURE=T D A B SMB:DCOM- OVERFLOW /5c/00/43/00/24/00/5c/00*/00*/00*/00*/00*/00*/00 */00*/00*/00*/00*/00*/00*/00*/00, /01/10/08/00/cc/c c/cc/cc

Carleton University School of Computer Science Solution/Problem ● Automated countermeasures are required for worm containment and suppression ● Worm propagation detection methods need to: – Detect propagation quickly – Detect propagation accurately – Detect propagation of varying rates – Detect zero-day worms

Carleton University School of Computer Science Scanning Worm Characteristics ● Scanning worms can employ a variety of strategies to infect systems – Topological scanning – Slow scanning – Fast scanning ● Topological scanning – Use of local information to find victims – Potential to be very fast

Carleton University School of Computer Science Enterprise Network

Carleton University School of Computer Science ARP-based Scanning Worm Detection Approach ● Focus on protecting the internal network – Network 'hard crunchy' exterior 'soft gooey' middle – Limit damage within network cell ● Behavioral '“”signature' – Based on anomalous behavior – ARP request is a 'scan' – Premise: measurable changes in amount/pattern ARP activity ● 3-factor anomaly score

Carleton University School of Computer Science 3-Factor Anomaly Score ● Peer-list: previous connection activity ● ARP activity: 'expected' amount of ARP activity vs. observed ARP activity (mean + sd) ● Internal network dark space: ARP requests to non-existent systems ● Factor weighting is configurable ● When aggregate score from three factors exceeds threshold in specified time window: ALERT!!

Carleton University School of Computer Science Set-up and Training Period ● Divide network into cells (broadcast domains) ● Training period – Gather ARP broadcast requests – Construct peer-list – Calculate expected ARP activity for each system – Identify internal network address darkspace

Carleton University School of Computer Science Software Developed Prototype uses Perl modules and libpcap High-level design –Packet Processing Engine (PPE) Extract features from ARP broadcast traffic –ARP Correlation Engine (ACE) Create individual system ARP statistics Create peer-list Observe ARP request activity to generate 3-factor anomaly score Generate alerts

Carleton University School of Computer Science Testing Environment Two-week training period Two-week test run Lab/production network –One quarter Class C network –DNS/mail/web –Small user population –Linux OS –Closed network

Carleton University School of Computer Science ARP Statistics (Training Period)

Carleton University School of Computer Science Alert Thresholds ● Common threshold ● Function-specific threshold – Server/workstation ● Threshold as well as anomaly factor weighting is configurable ● Prototype can give default threshold based on training period – (r=3): 3 scans within a 3 minute window

Carleton University School of Computer Science Testing Results (Alerts) 2-week Period

Carleton University School of Computer Science False Positive Analysis ● Increasing scan threshold reduces false positives (our network r = 3 only 5 false positives in a two week period) ● Function-specific vs. common threshold approach reduced false positive rate ● All false positives were caused by a“ 'burst' in server activity

Carleton University School of Computer Science Worm Simulation Results ● Not practical to infect network ● Nmap scanning – Sequential scanning – Random scanning ● Detected all scan activity scenarios within 3 scans – Internal network darkspace a factor

Carleton University School of Computer Science Limitations ● Test network size ● Simulation vs. actual infection ● Exclusive observation of broadcast traffic may lead to underestimation of dark space ● DHCP ● P2P, highly distributed network: 'your mileage may vary'

Carleton University School of Computer Science Conclusions ● An approach to protect the internal network (topological worms) ● Analysis provides evidence that the approach has merit ● Full prototype developed ● Anomaly-based – ability to detect emerging worms

Carleton University School of Computer Science Questions?