1. A Dos Resilient Flow-level Intrusion Detection Approach for High-speed Networks Yan Gao, Zhichun Li, Yan Chen Department of EECS, Northwestern University Presented By Sudarsan Vinay Maddi Christopher Brandon Barkley

2. Outline  Motivation  Background on Sketches  Design of the HiFIND system  Evaluation  Conclusion

3. The Problem  The increasing frequency, severity, and sophistication of viruses makes it critical to detect outbursts at routers and gateways instead of end hosts.

4. Current Intrusion Detection Systems  Signature-based Detection  Anomaly-based Detection

5. Signature-based Intrustion Detection  Examples: BRO, Snort  Perform pattern-matching and report situations that match known attack types.  Advantage: Accurately detects known attack types.  Disadvantage: Attackers can modify or create attacks that avoid detection until a software update.

6. Anomaly-based Intrusion Detection  Example: Manhunt  Build a model of acceptable behavior and flag exceptions using heuristics.  Advantage: Model is built according to actual use and can detect previously unknown attacks.  Disadvantage: Heuristic model can lead to false positives, system is inaccurate in the beginning (when it has little information).

7. Existing Network IDSes Insufficient  Signature based IDS cannot recognize unknown or polymorphic intrusions  Statistical IDSes for rescue, but Flow-level detection: unscalable  Vulnerable to DoS attacks e.g. TRW [IEEE SSP 04], TRW-AC [ USENIX Security Symposium 04], Superspreader [NDSS 05] for port scan detection Symposium 04], Superspreader [NDSS 05] for port scan detection Overall traffic based detection: inaccurate, high false positives e.g. Change Point Monitoring for flooding attack e.g. Change Point Monitoring for flooding attack detection [IEEE Trans. on DSC 04] detection [IEEE Trans. on DSC 04]

8. Existing Network IDSes Insufficient  Key features missing Distinguish SYN flooding and various port scans for effective mitigation Aggregated detection over multiple vantage points

9. Other Limitations  Another limitation of existing IDSes is that they are implemented in software.  Software-based data recording have trouble keeping up with link speeds of high-speed routers.  To solve this data recording must be hardware implementable.

10. HiFIND System The main goal is to develop an accurate High- speed Flow-level Intrusion Detection (HiFIND) system  Leverage the data streaming techniques: reversible sketches  Select an optimal small set of metrics from TCP/IP headers for monitoring and detection  Aggregate compact sketches from multiple routers for distributed detection

11. Goals of HiFIND  Scalable to flow-level detection on high speed networks  DoS resilient  Distinguish SYN flooding from port scans  Enable aggregate detection over multiple gateways.  Seperate anomalies to limit false positives.

12. Deployment of HiFIND  Attached to a router/switch as a black box  Edge network detection particularly powerful Original configuration Monitor each port separately Monitor aggregated traffic from all ports Router LA N Inter net Switch LA N (a) Router LAN Inter net LA N (b) HiFIND system scan port Splitter Router LA N Inter net LA N (c) Splitter HiFIND system Switch HiFIND system HiFIND system

13. Outline  Motivation  Background on Sketches  Design of the HiFIND system  Evaluation  Conclusion

14. Reversible Sketches  Traditional sketches do not store key information making it hard to infer a culprit flow.  Reversible sketches use a reversible hashing function to infer keys of culprits without storing explicit key information.  More info: Reversible Sketches for Efficient and Accurate Change Detection over Network Data Streams by Schweller, Gupta, Parsons, and Chen of Northwestern University.

15. Two Dimensional k-ary Sketch  Instead of using one-dimensional hash table, use a 2D hash table matrix.  Allows to distinguish between types of attacks by keeping track of more information.  Ex. Columns are a hash of {SIP,DIP}, rows are a hash of Dport.

16. Outline  Motivation  Background on Sketches  Design of the HiFIND system Architecture Sketch-based intrusion detection Intrusion classification with 2D sketches Feature analysis  Evaluation  Conclusion

17. Architecture of the HiFIND system

18.  Threat model TCP SYN flooding (DoS attack) Port scan  Horizontal scan  Vertical scan  Block scan  Forecast methods EWMA

19. Sketch-based Detection Algorithm 2NoYes {Dport} 2YesNoYes{DIP} 2.5Yes non-spoofed{SIP} 1.5YesNonon-spoofed{SIP, DIP} 1No Yes{DIP, Dport} 1.5NoYesnon-spoofed{SIP, Dport} ScoreVscanHscanSYN floodingKeys

20. Sketch-based Detection Algorithm RS({DIP, Dport}, #SYN - #SYN/ACK)  Detect SYN flooding attacks RS({SIP, DIP}, #SYN - #SYN/ACK)  Detect any intruder trying to attack a particular IP address RS({SIP, Dport}, #SYN - #SYN/ACK)  Detect any source IP which causes a large number of uncompleted connections to a particular destination port

21.  Major challenge Can not completely differentiate different types of attacks E.g., if destination port distribution unknown, it is hard to distinguish non-Spoofing SYN flooding attacks from vertical scans by RS({SIP, DIP}, #SYN - #SYN/ACK) Intrusion Classification

22.  Bi-modal distribution SYN floodings Vertical scans

23. Two-dimensional (2D) Sketch For example: differentiate vertical scan from SYN flooding attack  The two-dimensional k-ary sketches  An example of UPDATE operation

24. DoS Resilience Analysis HiFIND system is resilient to various DoS attacks as follows  Send source spoofed SYN packets to a fixed destination Detected as SYN flooding attack  Send source spoofed packet to random destinations Evenly distributed in the buckets of each hash table, no false positives  Reverse-engineer the hash functions to create collisions Difficult to reverse engineering of hash functions  Unknown hash output of each hash function  Multiple hash tables and different hash functions  Even know the hash functions of sketches Very hard to find collisions through exhaustive search

25. Distributed Intrusion Detection Naive solution: Transport all the packet traces or connection states to the central site HiFIND: Summarize the traffic with compact sketches at each edge router, and deliver them to the central site SYN1 SYN/ACK1 SYN2 SYN/ACK2

26. Outline  Motivation  Background on Sketches  Design of the HiFIND system  Evaluation  Conclusion

27. Evaluation Methodology  Router traffic traces Lawrence Berkeley National Laboratory  One-day trace with ~900M netflow records Northwestern University  One day experiment in May 2005 with 239M netflow records, 1.8TB traffic and 1:1 packet samples  Evaluation metrics Detection accuracy Online performance:  Speed  Memory consumption  Memory access per packet

28. Highly Accurate

30. Detection Validation  SYN flooding Backscatter Hscans and Vscans The knowledge of port number e.g. 5 major scenarios of the top 10 Hscans Rahack worm23687489915.192.50.153 MySQL Bot scans25964330695.30.62.202 Scan SSH4501422109.132.101.1 99 SQLSnake scan5478814335.4.247.103 SQLSnake scan562751433204.10.110.38 Cause# DIPDportAnonymized SIP

31. Sasser worm625554165.5.42.10 Nachi or MSBlast worm6413598.198.0.101 NetBIOS scan641392.0.28.90 Sasser and Korgo worm644453.66.52.227 Nachi or MSBlast worm6413598.198.251.16 8 Cause# DIPDportAnonymized SIP e.g. 5 major scenarios of the bottom 10 Hscans Detection Validation

32. Online performance evaluation  Small memory access per packet 16 memory accesses per packet with parallel recording  Small memory consumption

33. Online performance evaluation  Recording speed Worst case: recording 239M items in 20.6 seconds i.e., 11M insertions/sec  Detection speed Detection on 1430 minute intervals  Average detection time: 0.34 seconds  Maximum detection time: 12.91 seconds Stress experiments in each hour interval  Detecting top 100 anomalies with average 35.61 seconds and maximum 46.90 seconds

34. Outline  Motivation  Background on Sketches  Design of the HiFIND system  Evaluation  Conclusion

35. Conclusion - Advantages  Achieves proposed goals including scalability and distinguishing attack types.  Highly accurate on test data.  Reduction in False Positives  Very low memory usage (13.2 MB)

36. Conclusion - Disadvantages  HiFIND did not detect some small horizontal port scans that TRW detected.  Authors said these were a combination of multiple small scans too stealthy for their thresholds  Future work to further investigate this and find a way to account for it.

37. Conclusion – Paper Disadvantages  Authors vague on implementation, only mentioning it used a single FPGA board.  Authors not explicitly define terms (e.g. Sketches).  Authors do not explain or cite heuristics used to reduce false positives.

38. Thank You ! Questions?