1 Integrating BotMiner & SNARE into SMITE Nick Feamster and Wenke Lee Georgia Tech Students: Shuang Hao, Junjie Zhang.

Slides:



Advertisements
Similar presentations
1 UNIT I (Contd..) High-Speed LANs. 2 Introduction Fast Ethernet and Gigabit Ethernet Fast Ethernet and Gigabit Ethernet Fibre Channel Fibre Channel High-speed.
Advertisements

eSafe Implementation Topologies
Building Fast, Flexible Virtual Networks on Commodity Hardware Nick Feamster Georgia Tech Trellis: A Platform for Building Flexible, Fast Virtual Networks.
Nick Feamster Georgia Tech
Wenke Lee and Nick Feamster Georgia Tech Botnet and Spam Detection in High-Speed Networks.
11/20/09 ONR MURI Project Kick-Off 1 Network-Level Monitoring for Tracking Botnets Nick Feamster School of Computer Science Georgia Institute of Technology.
Wenke Lee and Nick Feamster Georgia Tech Botnet and Spam Detection in High-Speed Networks.
BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection Guofei Gu1,2, Roberto Perdisci3, Junjie Zhang1,
Challenges in Making Tomography Practical
Integrating BotMiner and SNARE into SMITE Nick Feamster and Wenke Lee Students: Shuang Hao and Junjie Zhang Georgia Tech.
Network-Based Spam Filtering Anirudh Ramachandran Nick Feamster Georgia Tech.
Network-Based Spam Filtering Nick Feamster Georgia Tech Joint work with Anirudh Ramachandran and Santosh Vempala.
1 Building a Fast, Virtualized Data Plane with Programmable Hardware Bilal Anwer Nick Feamster.
Network Security Highlights Nick Feamster Georgia Tech.
1 Resonance: Dynamic Access Control in Enterprise Networks Ankur Nayak, Alex Reimers, Nick Feamster, Russ Clark School of Computer Science Georgia Institute.
1 Network-Level Spam Detection Nick Feamster Georgia Tech.
Network Operations Research Nick Feamster
Network Security Highlights Nick Feamster Georgia Tech.
1 Resonance: Dynamic Access Control in Enterprise Networks Ankur Nayak, Alex Reimers, Nick Feamster, Russ Clark School of Computer Science Georgia Institute.
Jeopardy Q 1 Q 6 Q 11 Q 16 Q 21 Q 2 Q 7 Q 12 Q 17 Q 22 Q 3 Q 8 Q 13
0 - 0.
Addition Facts
Communicating over the Network
Video Services over Software-Defined Networks
1 IP - The Internet Protocol Relates to Lab 2. A module on the Internet Protocol.
SHARKFEST '09 | Stanford University | June 15–18, 2009 The Reality of 10G Analysis Presented by: Network Critical Wednesday, June 17 th, :30 pm –
Break Time Remaining 10:00.
Zhiyun Qian, Z. Morley Mao (University of Michigan)
Local Area Networks - Internetworking
Application Layer: functionality and Protocols
1 Welcome Overview of DOCSIS. 2 Data Over Cable Service Interface Specification.
Christophe Jelger – CS221 Network and Security - Universität Basel Christophe Jelger Post-doctoral researcher IP Multicasting.
MOBILE DATA CHARGING: NEW ATTACKS AND COUNTERMEASURES Chunyi Peng,
Johan Garcia Karlstads Universitet Datavetenskap 1 Datakommunikation II Signaling/Voice over IP / SIP Based on material from Henning Schulzrinne, Columbia.
Virtual Switching Without a Hypervisor for a More Secure Cloud Xin Jin Princeton University Joint work with Eric Keller(UPenn) and Jennifer Rexford(Princeton)
Configuring and Troubleshooting ACLs
1 Network Address Translation (NAT) Relates to Lab 7. Module about private networks and NAT.
ICmyNet.Flow Network Traffic Analysis System If You Want to See Your Net
IPv6 Routing.
1 © 2004, Cisco Systems, Inc. All rights reserved. CCNA 1 v3.1 Module 10 Routing Fundamentals and Subnets.
© 2007 Cisco Systems, Inc. All rights reserved.Cisco Public 1 ETHERNET Derived From CCNA Network Fundamentals – Chapter 9 EN0129 PC AND NETWORK TECHNOLOGY.
Page 1 / 18 Internet Traffic Monitor IM Page 2 / 18 Outline Product Overview Product Features Product Application Web UI.
Addition 1’s to 20.
25 seconds left…...
Detecting Spam Zombies by Monitoring Outgoing Messages Zhenhai Duan Department of Computer Science Florida State University.
Week 1.
We will resume in: 25 Minutes.
Clock will move after 1 minute
1 Unit 1 Kinematics Chapter 1 Day
Select a time to count down from the clock above
Link State Routing Jean-Yves Le Boudec Fall
New Opportunities for Load Balancing in Network-Wide Intrusion Detection Systems Victor Heorhiadi, Michael K. Reiter, Vyas Sekar UNC Chapel Hill UNC Chapel.
An Introduction of Botnet Detection – Part 2 Guofei Gu, Wenke Lee (Georiga Tech)
1 A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection 許富皓 資訊工程學系 中央大學 1.
BotMiner Guofei Gu, Roberto Perdisci, Junjie Zhang, and Wenke Lee College of Computing, Georgia Institute of Technology.
Network Security: Spam Nick Feamster Georgia Tech CS 6250 Joint work with Anirudh Ramachanrdan, Shuang Hao, Santosh Vempala, Alex Gray.
Botnet Dection system. Introduction  Botnet problem  Challenges for botnet detection.
Detecting Spammers with SNARE: Spatio-temporal Network-level Automatic Reputation Engine Shuang Hao, Nadeem Ahmed Syed, Nick Feamster, Alexander G. Gray,
Guofei Gu, Roberto Perdisci, Junjie Zhang, and Wenke Lee College of Computing, Georgia Institute of Technology USENIX Security '08 Presented by Lei Wu.
Amir Houmansadr CS660: Advanced Information Assurance Spring 2015
BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection Guofei Gu, Roberto Perdisci, Junjie Zhang, and.
Wide-scale Botnet Detection and Characterization Anestis Karasaridis, Brian Rexroad, David Hoeflin In First Workshop on Hot Topics in Understanding Botnets,
Understanding the Network-Level Behavior of Spammers Author: Anirudh Ramachandran, Nick Feamster SIGCOMM ’ 06, September 11-16, 2006, Pisa, Italy Presenter:
Botnets Usman Jafarey Including slides from The Zombie Roundup by Cooke, Jahanian, McPherson of the University of Michigan.
BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection Presented by D Callahan.
2009/6/221 BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure- Independent Botnet Detection Reporter : Fong-Ruei, Li Machine.
1 Detecting Spammers with SNARE: Spatio-temporal Network-level Automatic Reputation Engine Speaker: Jun-Yi Zheng 2010/01/18.
Guofei Gu, Roberto Perdisci, Junjie Zhang, and Wenke Lee
Presented by Aaron Ballew
Presentation transcript:

1 Integrating BotMiner & SNARE into SMITE Nick Feamster and Wenke Lee Georgia Tech Students: Shuang Hao, Junjie Zhang

2 Status Report Summary of BotMiner and SNARE Integration on GaTech campus network Preliminary evaluation results Next steps

3 SMITE Integration

4 BotMiner: Structure and Protocol Independent Botnets can change their C&C content (encryption, etc.), protocols (IRC, HTTP, etc.), structures (P2P, etc.), C&C servers, infection models …

5 Definition of a Botnet A coordinated group of malware instances that are controlled by a botmaster via some C&C channel –Hosts that have similar C&C-like traffic and similar malicious activities We need to monitor two planes –C-plane (C&C communication plane): who is talking to whom –A-plane (malicious activity plane): who is doing what

6 BotMiner Architecture Sensors Algorithms Correlation

7 SNARE: Network-Level Spam Filter Single-Packet –AS of senders IP –Distance to k nearest senders –Status of service ports –Geodesic distance –Time of day Single-Message –Number of recipients –Length of message Aggregate (Multiple Message/Recipient)

8 Test Environment Port mirrored from College of Computing network switch –About 300 Mbps

9 Current Status Real-time test on college network Summary of results –Pipeline runs in real-time (200 to 300 Mbps) –BotMiner & SNARE run in batch mode, detecting bots/spammers based on data of one day –Results from 4 days of testing: September 21-24, 2009

10 Metrics Volume –N1: raw by pipeline. –N2: raw flows recorded. –N3-B: C-flows. (BotMiner) –N4-S: SMTP flows (SNARE) Time –T1: Dumping raw flows –T2-B: Aggregating raw flows to c-flows –T3-B: Clustering and correlation. –T4-S: Feature extraction (single-packet based) –T5-S: Building classifier (based on sampled flows) –T6-S: Detection

11 Detection Metrics BotMiner –TP: Detection Rate (6 botnets including HTTP-, IRC-, P2P-based botnets). –FP: False positive rate SNARE –TP: (Ground truth from DNSBL) –FP: False positive rate

12 Reducing Flow Volume N2 (# of flows recorded) < N1 (# of raw flows) Policies for reducing volume –Keep the only flows whose SrcIP is from internal networks and DstIP is to external networks For TCP flows, to eliminate flows for scanning, we only record flows in database which have at least 2 packets in outgoing or incoming direction. –BotMiner detects scanning/spamming behaviors on raw flows (rather than flow recorded in database) –SNARE works on SMTP flows Discard the flows whose IP appear on the whitelist (e.g., internal major HTTP/DNS)

13 Pipeline Configuration Device Info –Box Intel(R) Xeon(TM) CPU 3.00GHz 2G Memory Debian Linux –NIC information Link encap:Ethernet HWaddr 00:15:c5:e6:72:96 inet6 addr: 2610:148:1f02:8f00:215:c5ff:fee6:7296/64 Scope:Global inet6 addr: fe80::215:c5ff:fee6:7296/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 Pipeline Configuration - pcaplive device=eth1 -addressanalysis -flow_analyzer dump_period=600 (10 minutes)

14 Volume: Number of Flows DateN1 (# of raw flows) N2 (# flows in DB) N3-B (# of C-flows) N4-S (# SMTP flows) ,636,396936,39721,204307, ,589,989936,96229,912287, ,575,773869,81115,796404, ,792,651967,94513,070404,426

15 BotMiner Evaluation: Time All times in minutes DateT1 (dumping raw flows) T2-B (flow aggregation) T3-B (clustering and correlation)

16 BotMiner Evaluation: Detection The number of the hosts we used to evaluate the false positives is the number of internal hosts in the recorded flows. DateB- HTTP-I B-HTTP- II B-IRCB- spybot B-sdbotWaldec (p2p) False positives /42/44/43/4 3/311/ /42/44/43/4 3/310/ /42/44/42/4 3/39/ /4 3/311/801

17 SNARE Evaluation Single packet/header features (for initial testing): –AS number –Geodesic distance between the sender and the recipient –Message size (bytes sent) –Local hour when the was sent

18 Evaluation of SNARE SNARE trains on sampled SMTP flows (in T5-S) All times in seconds DateT4-S (feature extraction) T5-S (model, samples) T5-S (model, samples) T5-S (model, samples) T6-S (detection) ) Time for training 50,000 samples (in T5-S) is high, probably because it reaches the physical memory limitations. 1) The detection time (T6-S) is relatively small (note: all SMTP flows)

19 Next Steps Optimize the flow dumping process to improve efficiency. In the case of SNARE, evaluate with more features.