1. Network-based Intrusion Detection, Prevention and Forensics System 1 Yan Chen Department of Electrical Engineering and Computer Science Northwestern University Lab for Internet & Security Technology (LIST) http://list.cs.northwestern.edu

2. 2 The Spread of Sapphire/Slammer Worms

3. 3 Current Intrusion Detection Systems (IDS) Mostly host-based and not scalable to high- speed networks –Slammer worm infected 75,000 machines in <10 mins –Host-based schemes inefficient and user dependent Have to install IDS on all user machines ! Mostly simple signature-based –Cannot recognize unknown anomalies/intrusions –New viruses/worms, polymorphism

4. 4 Current Intrusion Detection Systems (II) Cannot provide quality info for forensics or situational-aware analysis –Hard to differentiate malicious events with unintentional anomalies Anomalies can be caused by network element faults, e.g., router misconfiguration, link failures, etc., or application (such as P2P) misconfiguration –Cannot tell the situational-aware info: attack scope/target/strategy, attacker (botnet) size, etc.

5. 5 Network-based Intrusion Detection, Prevention, and Forensics System Online traffic recording [SIGCOMM IMC 2004, INFOCOM 2006, ToN 2007] [INFOCOM 2008] –Reversible sketch for data streaming computation –Record millions of flows (GB traffic) in a few hundred KB –Small # of memory access per packet –Scalable to large key space size (2 32 or 2 64 ) Online sketch-based flow-level anomaly detection [IEEE ICDCS 2006] [IEEE CG&A, Security Visualization 2006] –Adaptively learn the traffic pattern changes –As a first step, detect TCP SYN flooding, horizontal and vertical scans even when mixed Online stealthy spreader (botnet scan) detection [IEEE IWQoS 2007]

6. 6 Network-based Intrusion Detection, Prevention, and Forensics System (II) Polymorphic worm signature generation & detection [IEEE Symposium on Security and Privacy 2006] [IEEE ICNP 2007] Accurate network diagnostics [SIGCOMM IMC 2003, SIGCOMM 2004, ToN 2007] [SIGCOMM 2006] [INFOCOM 2007 (2)] Scalable distributed intrusion alert fusion w/ DHT [SIGCOMM Workshop on Large Scale Attack Defense 2006] Large-scale botnet and P2P misconfiguration event forensics [work in progress]

7. 7 System Deployment Attached to a router/switch as a black box Edge network detection particularly powerful Original configuration Monitor each port separately Monitor aggregated traffic from all ports Router LAN Inter net Switch LAN (a) Router LAN Inter net LAN (b) RAND system scan port Splitter Router LAN Inter net LAN (c) Splitter RAND system Switch HPNAIDM system RAND system

8. P2P Doctor: Measurement and Diagnosis of Misconfigured Peer- to-Peer Traffic Zhichun Li, Anup Goyal, Yan Chen and Aleksandar Kuzmanovic Lab for Internet and Security Technology (LIST) Northwestern Univ.

9. What is P2P Misconfiguration  P2P file sharing accounted for > 60% of traffic in USA and > 80% in Asia  Thousands of peers send P2P file downloading requests to a “random” target on the Internet possibly triggered by bugs or by malicious reasons generates large amount of unwanted traffic Influence the performance of peers  It contributes on an average of about 37% of the “Internet background radiation” in 2007

10. Motivations  P2P software DC++ has already been exploited by attackers for DoS  direct gigabit “junk” data per second to a victim host from more than 150,000 peers  Currently, little is known about the characteristics or root causes of P2P misconfiguration events

11. Outline Motivation Passive measurement results P2P Doctor system design Root cause diagnosis and analysis Conclusion

12. Peer Classification All the peers Not in the P2P Network In the P2P Network Bogus Peers Anti-P2P Peers Normal Peers Unintentionally Misconfigured peers Poisoned Peers (Intentional)

13. Passive Measurement Honeynet/honeyfarm datasets Events: # of unique sources > 100 in 6 hours –After filtering scan traffic Event characteristics: –Mostly target a single IP –Duration: A few hours to up to a month LBLNUGQ Sensor5 /2410 /244 /16 Traces883GB287GB49GB Duration37 months 7 months 26 days LBLNU eMule106 BitTorrent24290 Gnutella11 Soribada40 Xunlei180 VAgaa10

14. Popularity Growth Trend: IP space: observed in three sensors in five different /8 IP prefixes Significant problem: –Amount of traffic only from 15 /24 networks. The real traffic can be 1M times more. The average total connections of P2P misconfiguration events per month. 37%!

15. Further Diagnosis Problems with passive measurement on archived data –Events have gone –Hard to backtrack the propagation –Root cause? Need a real-time backtracking and diagnosis system!

16. Outline Motivation Passive measurement results P2P Doctor system design Root cause diagnosis and analysis Conclusion

17. Design of P2P Doctor System Root cause inference Backtracking system P2P-enabled Honeynet P2P payload signature based responder Event identification 10100101011101 infohash; ‘abc.avi’ Protocol parsing for metadata

18. Design of P2P Doctor System Root cause inference Backtracking system P2P-enabled Honeynet Index Server (tracker) Crawling BT: top 100, eMule: 185 Peer Exchange Protocol Crawling DHT Crawling

19. Design of P2P Doctor System Root cause inference Backtracking system P2P-enabled Honeynet What is the root cause? Which peers spread misconfigurtion? How is misconfiguration disseminated? What is the percentage of bogus peers in the misconfigured P2P networks?

20. Deployment and Data Collection Deployed the P2P doctor system on NU honeynet (10 /24 networks in three /8) Real-time events –Previous passive measurement data referred as historical events BitTorrenteMule # of events2042 Duration23 days 08/23/2007 to 09/15/2007

21. Outline Motivation Passive measurement results P2P Doctor system design Root cause diagnosis and analysis Conclusion

22. Root Cause Analysis Methodology –Track how honeynet IPs propagated in P2P systems –Use unroutable IP space as a big honeynet (66.8% of IPv4 Space) –Hypothesis formulation and testing Classification of measured peers –Misconfigured peers: Passively observed from honeynet –Backtracked peers: actively observed through backtracking –Reverse honeynet peers: the IP obtained by reversing the target IP from the honeynets Results –Data plane traffic radiation –Detailed results focus on eMule and BitTorrent

23. Data Plane Traffic Radiation DHT Peer Exchange Index Server Who has Beowulf.avi? 1.2.3.4 Resource mapping

24. eMule – Root Cause Byte ordering is the problem! 1.2.3.4 4.3.2.1 1.2.3.4

25. eMule – Root Cause Byte ordering is the problem! –Hypothesis from the historical data In 80% of events, the reverse target IPs are alive –Verified with real-time events 61% of the reverse honeynet peers indeed running eMule with the port number reported For the backtracked peers which is in the unroutable IP space, 69.6% of them having reverse IPs run eMule

26. eMule – Peers & Dissemination Which peers spread misconfiguration? –99.24% of misconfigured peers are normal peers How is the misconfiguration disseminated? –Index Server? No –Peer exchange? Yes Percentage of bogus peers in eMule network? –[12.7%, 25.0%] w/ a total of 37,079 backtracked peers

27. BitTorrent – Responsible Peers  Both anti-P2P and normal peers are responsible  Events classified to two types with diagonally different sets of characteristics  For anti-P2P peers events  All the sources are from the IP range owned by anti-p2p companies like Media Defender, Media Sentry, Net Sentry etc.  Seen 6 out of 7 major anti-P2P companies sources in our honeynet. Anti-P2P peersNormal peers Number of Events127 (39%)205 (61%) Client Software100% - Azureus 90% - UTorrent (NU) 88% - BitComet+BitSpirit (LBL) Avg. number of Connections / src 40025 Arrival & DepartureAll togetherPoisson Avg. Duration4.5 hours106.1 hours

28. BitTorrent – Root Cause  Refuted Byte Ordering Hypothesis –For 20 real-time events, no reverse honeynet peers runs BitTorrent  For normal peer events, culprit is Peer Exchange (PEX) protocol implemented by uTorrent-compatible clients  For anti-P2P peer events –Possibly related to Azureus system –Still an open question (No real- time events)

29. BitTorrent – Root Cause II  How is the misconfigured peers influenced? –On average [0.053%,32%]  Where is the origin of bogus peers? –Nine hosts are the major players – each has >50% of peers in their buddy list as bogus. –Eight of them are from a small IP range belong to an Anti- P2P company. –Only bogus and other Anti-P2P peers are in the buddy list of those eight peers. –Support uTorrent Peer Exchange protocol and respond regardless of the infohash in request.

30. Conclusions The first study to measure and diagnose large- scale P2P misconfiguration events Found 30% Internet background radiation is caused by P2P misconfiguration –Popular in various P2P systems, exponential growth trend, and scattered in the IPv4 space For eMule, we found it is caused by network byte order problem For BitTorrent, classified to anti-P2P peer events and normal peer events with diagonally different sets of characteristics –Found the uTorrent PEX causes the problem in normal peer events

32. Backup Slides

33. Motivation  Given unprecedented amount of traffic, even a slight mis- configuration of the P2P system can result in a DDoS kind of situation  Prevalence in time, space, and across a number of distinct P2P systems with a temporal increasing trend is alarming.  P2P miscongurations can cause innocent people to get involved in the above “war” between P2P and anti-P2P systems.  Presently, nothing is known about the causes or overall effects of P2P mis-configurations  Our goal is to determine the root cause(s) of each type of mis-configuration

34. Related Work Misconguration is widely spread across different networked and distributed systems like BGP [Labovitz et al. ] and firewalls [Cuppens et al. ]. Measurement studies of normal P2P traffic [ACM SOSP (2003), MCN (2002)], while we measure the abnormal P2P traffic observed in honeynets. In [INFOCOM (2005)], Content pollution including intentional and unintentional pollution is widespread for popular titles. P2P systems like Fasttrack and Overnet are vulnerable to the index poisoning attack [INFOCOM (2006)] All of the above studies focus on the content pollution or index poisoning while our focus is the index misconfiguration. First large-scale measurement study on the root causes for both intentional/unintentional index misconfiguration.

35. What is P2P Misconfiguration  More than 50% of the traffic in the Internet today is P2P traffic By Symantec Corporation ’ s recent report  P2P file sharing accounted for > 60% of traffic in USA and > 80% in Asia

36. eMule – Misconfigured peers study Examine the misconfigured peers –9.3% of such peers has unroutable peers in their buddy list (discovered by peer exchange). –Those 9.3% of peers are high affected by unroutable peers –None of them in the Anti-P2P blacklist –6 out of the top 10 peers in terms of number of connection they issued to the honeynet are running Linux. (99% as whole running Windows) More than 67% of peers has more than 80% of unroutable peers

37. BitTorrent – Dissemination  How is misconfiguration disseminated? –Index server? - No –Peer exchange? - Yes  Percentage of bogus peers in BitTorrent network ?  Out of a total of 9,000 backtracked peers, only 13 IPs are unroutable and 3,150 IPs gave connection timeout  0.14% < bogus Peers < 35%