An Introduction of Botnet Detection – Part 2 Guofei Gu, Wenke Lee (Georiga Tech)

2009/5/26 Speaker: Li-Ming Chen 2 Reference Guofei Gu, Wenke Lee, et al.  BotHunter: Detecting Malware Infection through IDS-driven Dialog Correlation USENIX Security 2007  BotSniffer: Detecting Botnet Command and Control Channels in Network Traffic ACM NDSS 2008  BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-independent Botnet Detection USENIX Security 2008 Moheeb Abu Rajab, et al.  A Multifaceted Approach to Understanding the Botnet Phenomenon ACM IMC 2006

2009/5/26 Speaker: Li-Ming Chen 3 Lifecycle of a Typical Botnet Infection Why Botnet is hard to detect? involving multiple steps flexible design of C&C channels 6. Malicious activities (e.g., DDoS) (borrow infection strategies from traditional malicious attacks) (optional) authentication

2009/5/26 Speaker: Li-Ming Chen 4 C&C (Command and Control) Channels Centralized C&C channel P2P C&C channel Message Response Crowd Activity Response Crowd

2009/5/26 Speaker: Li-Ming Chen 5 Comparison of the 3 Approaches BotHunterBotSnifferBotMiner Detection Target BotBotnet DescriptionDetect the lifecycle of a bot, including infection and command execution Detect group of hosts with spatial-temporal similarity in C&C communication BotSniffer extension. Support various C&C comm. framework. AssumptionsPredefine bot infection lifecycle Focus on centralized C&C communication Bots will perform tasks and response InsightVertical correlation of IDS alerts Horizontal correlation of similar behaviors Cluster hosts with similar traffic patterns Approach detect individual events identify parts of the lifecycle group hosts connect to the same C&C server detect similar activity or message response behaviors cluster similar C&C comm. cluster similar malicious traffic. cross clustering

2009/5/26 Speaker: Li-Ming Chen 6 BotHunter Utilize Snort to detect sign of local infection Signs match the predefined evidences (dialog transitions) A Bot could be: E2 AND E3-E5 At least two distinct signs of E3-E5 Predefined Lifecycle

2009/5/26 Speaker: Li-Ming Chen 7 BotHunter (cont ’ d) Current bots are multi-vector Design two modules (inbound/outbound) for scan detection Assign high weight to ports often used by malware (predefined) Observe outbound scan rate, outbound connection failure rate, and address dispersion Anomaly-based payload exploit detection Learn normal profile (using 2-gram PAYL) Check deviation distance of a test payload from the normal profile Use bot-specific heuristics to build signatures (rules)

2009/5/26 Speaker: Li-Ming Chen 8 BotHunter: Evaluation Results (1/2) Experiments in a virtual network  To test FN rate (by examining 10 different bots) # of generated dialog warnings # involving the victim

2009/5/26 Speaker: Li-Ming Chen 9 BotHunter: Evaluation Results (2/2) Honeynet-based experiments  Use SRI honeynet to capture real-world bot infection  Use BotHunter to analysis these traces  95.1% TP rate (1920/2019 in 3 weeks)  FN is due to: Infection failure, honeynet setup and policy failure, data corruption failure. Experiments in a campus network  98 profiles were generated in 4 months (no FP) Experiments in SRI laboratory network  Generate 1 bot profile and it is FP (a 1.6 GB multifile FTP transfer matchs “E2 & E3”)

2009/5/26 Speaker: Li-Ming Chen 10 BotHunter: Pros and Cons Pros:  Real-time detection of bot infections  Evidence trail gathering for investigation of putative infections Cons:  Use heuristic (2 conditions) to decide a bot infection  Less flexible

2009/5/26 Speaker: Li-Ming Chen 11 BotSniffer Response crowd: Density check Homogeneity check (data reduction) Port-independent, payload inspection

2009/5/26 Speaker: Li-Ming Chen 12 BotSniffer: Evaluation Methodology Use normal traffic traces to test the FP rate and use botnet traces (mix normal traffic) to test the detection performance Normal traces:  Capture 8 IRC traces (port 6667) and 5 complete traces from campus network Botnet traces:  Collect 3 real-world IRC-based botnet traces  Generate 3 botnet traffic by modifying source codes of 3 common botnets  Implement 2 http-based botnet

2009/5/26 Speaker: Li-Ming Chen 13 BotSniffer: Evaluation Results (1/2) All FP are generated due to single client incoming message response analysis. (Apply both activity response and message response group analysis)

2009/5/26 Speaker: Li-Ming Chen 14 BotSniffer: Evaluation Results (2/2) honeynet IRC logs (both message and activity) (periodically connect to server) (random delay) (the randomization of connection periods did not cause a problem, because there were still several clients performing activity responses at the time window)

2009/5/26 Speaker: Li-Ming Chen 15 BotSniffer: Pros and Cons Pros  Successfully detect all botnets (low FP rate)  Efficient alert reduction  More robust than other botnet detection system Cons  Focus on centralized C&C communication  Configure time window for group analysis  Possible evasions (e.g., misusing whitelist, encryption, protocol matcher, long response delay, obfuscation)

2009/5/26 Speaker: Li-Ming Chen 16 BotMiner (similar to BotSniffer) Focus on flow statistics, not message response! log Combine results and make final decision (more straightforward) (more complex)

2009/5/26 Speaker: Li-Ming Chen 17 BotMiner: Evaluation Methodology (same) use normal traffic traces to test the FP rate and use botnet traces (mix normal traffic) to test the detection performance Normal traces:  Capture 10 days traffic record at the campus network Botnet traces:  4 IRC, 2 HTTP and 2 P2P botnets 2 IRC and 2 HTTP are also used for BotSniffer P2P: 2 real-world traces (Nugache and Storm) TCP, encryptedUDP

2009/5/26 Speaker: Li-Ming Chen 18 BotMiner: Evaluation Results (1/3) (C-plan data reduction) Most useful, Only record internal to external flows. Remove helf-open TCP flows Whitelist

2009/5/26 Speaker: Li-Ming Chen 19 BotMiner: Evaluation Results (2/3) 4 features: temporal – fph, bps spatial – ppf, bpp Cluster by using the mean and variance of the features Further cluster by separating each feature as a vector of 13 elements according to their distribution Ignore clusters only contain 1 host Most FP clusters contain only 2 hosts

2009/5/26 Speaker: Li-Ming Chen 20 BotMiner: Evaluation Results (3/3) FN

2009/5/26 Speaker: Li-Ming Chen 21 BotMiner: Pros and Cons Pros:  Anomaly-based botnet detection system (independent of the protocol and structure used by botnets)  Low FN and FP rate Cons:  Stealthy: botmaster can commond the bots to perform extremely delayed task (evade cross clustering)

2009/5/26 Speaker: Li-Ming Chen 22 Summary Bothunter:  Vertical Correlation  Correlation on the behaviors of single host Botsniffer:  Horizontal Correlation  Focus on centralized C&C botnets Botminer:  Extension on Botsniffer  No limitations on the C&C types.