A Dos Resilient Flow-level Intrusion Detection Approach for High-speed Networks Yan Gao, Zhichun Li, Yan Chen Department of EECS, Northwestern University Presented By Sudarsan Vinay Maddi Christopher Brandon Barkley

Outline  Motivation  Background on Sketches  Design of the HiFIND system  Evaluation  Conclusion

The Problem  The increasing frequency, severity, and sophistication of viruses makes it critical to detect outbursts at routers and gateways instead of end hosts.

Current Intrusion Detection Systems  Signature-based Detection  Anomaly-based Detection

Signature-based Intrustion Detection  Examples: BRO, Snort  Perform pattern-matching and report situations that match known attack types.  Advantage: Accurately detects known attack types.  Disadvantage: Attackers can modify or create attacks that avoid detection until a software update.

Anomaly-based Intrusion Detection  Example: Manhunt  Build a model of acceptable behavior and flag exceptions using heuristics.  Advantage: Model is built according to actual use and can detect previously unknown attacks.  Disadvantage: Heuristic model can lead to false positives, system is inaccurate in the beginning (when it has little information).

Existing Network IDSes Insufficient  Signature based IDS cannot recognize unknown or polymorphic intrusions  Statistical IDSes for rescue, but Flow-level detection: unscalable  Vulnerable to DoS attacks e.g. TRW [IEEE SSP 04], TRW-AC [ USENIX Security Symposium 04], Superspreader [NDSS 05] for port scan detection Symposium 04], Superspreader [NDSS 05] for port scan detection Overall traffic based detection: inaccurate, high false positives e.g. Change Point Monitoring for flooding attack e.g. Change Point Monitoring for flooding attack detection [IEEE Trans. on DSC 04] detection [IEEE Trans. on DSC 04]

Existing Network IDSes Insufficient  Key features missing Distinguish SYN flooding and various port scans for effective mitigation Aggregated detection over multiple vantage points

Other Limitations  Another limitation of existing IDSes is that they are implemented in software.  Software-based data recording have trouble keeping up with link speeds of high-speed routers.  To solve this data recording must be hardware implementable.

HiFIND System The main goal is to develop an accurate High- speed Flow-level Intrusion Detection (HiFIND) system  Leverage the data streaming techniques: reversible sketches  Select an optimal small set of metrics from TCP/IP headers for monitoring and detection  Aggregate compact sketches from multiple routers for distributed detection

Goals of HiFIND  Scalable to flow-level detection on high speed networks  DoS resilient  Distinguish SYN flooding from port scans  Enable aggregate detection over multiple gateways.  Seperate anomalies to limit false positives.

Deployment of HiFIND  Attached to a router/switch as a black box  Edge network detection particularly powerful Original configuration Monitor each port separately Monitor aggregated traffic from all ports Router LA N Inter net Switch LA N (a) Router LAN Inter net LA N (b) HiFIND system scan port Splitter Router LA N Inter net LA N (c) Splitter HiFIND system Switch HiFIND system HiFIND system

Outline  Motivation  Background on Sketches  Design of the HiFIND system  Evaluation  Conclusion

Reversible Sketches  Traditional sketches do not store key information making it hard to infer a culprit flow.  Reversible sketches use a reversible hashing function to infer keys of culprits without storing explicit key information.  More info: Reversible Sketches for Efficient and Accurate Change Detection over Network Data Streams by Schweller, Gupta, Parsons, and Chen of Northwestern University.

Two Dimensional k-ary Sketch  Instead of using one-dimensional hash table, use a 2D hash table matrix.  Allows to distinguish between types of attacks by keeping track of more information.  Ex. Columns are a hash of {SIP,DIP}, rows are a hash of Dport.

Outline  Motivation  Background on Sketches  Design of the HiFIND system Architecture Sketch-based intrusion detection Intrusion classification with 2D sketches Feature analysis  Evaluation  Conclusion

Architecture of the HiFIND system

 Threat model TCP SYN flooding (DoS attack) Port scan  Horizontal scan  Vertical scan  Block scan  Forecast methods EWMA

Sketch-based Detection Algorithm 2NoYes {Dport} 2YesNoYes{DIP} 2.5Yes non-spoofed{SIP} 1.5YesNonon-spoofed{SIP, DIP} 1No Yes{DIP, Dport} 1.5NoYesnon-spoofed{SIP, Dport} ScoreVscanHscanSYN floodingKeys

Sketch-based Detection Algorithm RS({DIP, Dport}, #SYN - #SYN/ACK)  Detect SYN flooding attacks RS({SIP, DIP}, #SYN - #SYN/ACK)  Detect any intruder trying to attack a particular IP address RS({SIP, Dport}, #SYN - #SYN/ACK)  Detect any source IP which causes a large number of uncompleted connections to a particular destination port

 Major challenge Can not completely differentiate different types of attacks E.g., if destination port distribution unknown, it is hard to distinguish non-Spoofing SYN flooding attacks from vertical scans by RS({SIP, DIP}, #SYN - #SYN/ACK) Intrusion Classification

 Bi-modal distribution SYN floodings Vertical scans

Two-dimensional (2D) Sketch For example: differentiate vertical scan from SYN flooding attack  The two-dimensional k-ary sketches  An example of UPDATE operation

DoS Resilience Analysis HiFIND system is resilient to various DoS attacks as follows  Send source spoofed SYN packets to a fixed destination Detected as SYN flooding attack  Send source spoofed packet to random destinations Evenly distributed in the buckets of each hash table, no false positives  Reverse-engineer the hash functions to create collisions Difficult to reverse engineering of hash functions  Unknown hash output of each hash function  Multiple hash tables and different hash functions  Even know the hash functions of sketches Very hard to find collisions through exhaustive search

Distributed Intrusion Detection Naive solution: Transport all the packet traces or connection states to the central site HiFIND: Summarize the traffic with compact sketches at each edge router, and deliver them to the central site SYN1 SYN/ACK1 SYN2 SYN/ACK2

Outline  Motivation  Background on Sketches  Design of the HiFIND system  Evaluation  Conclusion

Evaluation Methodology  Router traffic traces Lawrence Berkeley National Laboratory  One-day trace with ~900M netflow records Northwestern University  One day experiment in May 2005 with 239M netflow records, 1.8TB traffic and 1:1 packet samples  Evaluation metrics Detection accuracy Online performance:  Speed  Memory consumption  Memory access per packet

Highly Accurate

Detection Validation  SYN flooding Backscatter Hscans and Vscans The knowledge of port number e.g. 5 major scenarios of the top 10 Hscans Rahack worm MySQL Bot scans Scan SSH SQLSnake scan SQLSnake scan Cause# DIPDportAnonymized SIP

Sasser worm Nachi or MSBlast worm NetBIOS scan Sasser and Korgo worm Nachi or MSBlast worm Cause# DIPDportAnonymized SIP e.g. 5 major scenarios of the bottom 10 Hscans Detection Validation

Online performance evaluation  Small memory access per packet 16 memory accesses per packet with parallel recording  Small memory consumption

Online performance evaluation  Recording speed Worst case: recording 239M items in 20.6 seconds i.e., 11M insertions/sec  Detection speed Detection on 1430 minute intervals  Average detection time: 0.34 seconds  Maximum detection time: seconds Stress experiments in each hour interval  Detecting top 100 anomalies with average seconds and maximum seconds

Outline  Motivation  Background on Sketches  Design of the HiFIND system  Evaluation  Conclusion

Conclusion - Advantages  Achieves proposed goals including scalability and distinguishing attack types.  Highly accurate on test data.  Reduction in False Positives  Very low memory usage (13.2 MB)

Conclusion - Disadvantages  HiFIND did not detect some small horizontal port scans that TRW detected.  Authors said these were a combination of multiple small scans too stealthy for their thresholds  Future work to further investigate this and find a way to account for it.

Conclusion – Paper Disadvantages  Authors vague on implementation, only mentioning it used a single FPGA board.  Authors not explicitly define terms (e.g. Sketches).  Authors do not explain or cite heuristics used to reduce false positives.

Thank You ! Questions?