Usenix Security 2004 Autograph Toward Automated, Distributed Worm Signature Detection Hyang-Ah KimBrad Karp Carnegie Mellon UniversityIntel Research &

Slides:



Advertisements
Similar presentations
Greg Williams CS691 Summer Honeycomb  Introduction  Preceding Work  Important Points  Analysis  Future Work.
Advertisements

Polygraph: Automatically Generating Signatures for Polymorphic Worms James Newsome *, Brad Karp *†, and Dawn Song * † Intel Research Pittsburgh * Carnegie.
Worm Origin Identification Using Random Moonwalks Yinglian Xie, V. Sekar, D. A. Maltz, M. K. Reiter, Hui Zhang 2005 IEEE Symposium on Security and Privacy.
Automated Worm Fingerprinting [Singh, Estan et al] Internet Quarantine: Requirements for Self- Propagating Code [Moore, Shannon et al] David W. Hill CSCI.
 Looked at some research approaches to: o Evaluate defense effectiveness o Stop worm from spreading from a given host o Defend a circle of friends against.
 Population: N=100,000  Scan rate  = 4000/sec, Initially infected: I 0 =10  Monitored IP space 2 20, Monitoring interval:  = 1 second Infected hosts.
 Well-publicized worms  Worm propagation curve  Scanning strategies (uniform, permutation, hitlist, subnet) 1.
Modeling the spread of active worms Zesheng Chen, Lixin Gao, and Kevin Kwiat bearhsu - INFOCOM 2003.
Internet Quarantine: Requirements for Containing Self- Propagating Code David Moore, Colleen Shannon, Geoffrey M. Voelker, Stefan Savage.
Internet Intrusions: Global Characteristics and Prevalence Presented By: Elliot Parsons Using slides from Vinod Yegneswaran’s presentation at SIGMETRICS.
5/1/2006Sireesha/IDS1 Intrusion Detection Systems (A preliminary study) Sireesha Dasaraju CS526 - Advanced Internet Systems UCCS.
Usenix Security 2004 Autograph Toward Automated, Distributed Worm Signature Detection Hyang-Ah KimBrad Karp Carnegie Mellon UniversityIntel Research &
Distributed Intrusion Detection Systems (dIDS) 2/10 CIS 610.
Worms: Taxonomy and Detection Mark Shaneck 2/6/2004.
© 2006 Cisco Systems, Inc. All rights reserved. Implementing Secure Converged Wide Area Networks (ISCW) Module 6: Cisco IOS Threat Defense Features.
Automated Worm Fingerprinting Sumeet Singh, Cristian Estan, George Varghese, and Stefan Savage Manan Sanghi.
Worm Defense. Outline  Internet Quarantine: Requirements for Containing Self-Propagating Code  Netbait: a Distributed Worm Detection Service  Midgard.
Autograph Toward Automated, Distributed Worm Signature Detection (Hyang-Ah Kim, Brad Karp) Yunhai & Justin.
Analysis of Anomalous Payload-based Worm Detection and Signature Generation by Ke Wang, Gabriela Cretu, Salvatore J.Stolfo Columbia University.
How to Own the Internet in your spare time Ashish Gupta Network Security April 2004.
Internet Quarantine: Requirements for Containing Self-Propagating Code David Moore et. al. University of California, San Diego.
A Study of Mass- mailing Worms By Cynthia Wong, Stan Bielski, Jonathan M. McCune, and Chenxi Wang, Carnegie Mellon University, 2004 Presented by Allen.
Network Defenses Brad Karp UCL Computer Science CS GZ03 / th December, 2007.
Network-based and Attack-resilient Length Signature Generation for Zero-day Polymorphic Worms Zhichun Li 1, Lanjia Wang 2, Yan Chen 1 and Judy Fu 3 1 Lab.
Network-based and Attack-resilient Length Signature Generation for Zero-day Polymorphic Worms Zhichun Li 1, Lanjia Wang 2, Yan Chen 1 and Judy Fu 3 1 Lab.
Lucent Technologies – Proprietary Use pursuant to company instruction Learning Sequential Models for Detecting Anomalous Protocol Usage (work in progress)
Presentation by Kathleen Stoeckle All Your iFRAMEs Point to Us 17th USENIX Security Symposium (Security'08), San Jose, CA, 2008 Google Technical Report.
A Statistical Anomaly Detection Technique based on Three Different Network Features Yuji Waizumi Tohoku Univ.
BY ANDREA ALMEIDA T.E COMP DON BOSCO COLLEGE OF ENGINEERING.
Fast Portscan Detection Using Sequential Hypothesis Testing Authors: Jaeyeon Jung, Vern Paxson, Arthur W. Berger, and Hari Balakrishnan Publication: IEEE.
Carleton University School of Computer Science Detecting Intra-enterprise Scanning Worms based on Address Resolution David Whyte, Paul van Oorschot, Evangelos.
1 How to 0wn the Internet in Your Spare Time Authors: Stuart Staniford, Vern Paxson, Nicholas Weaver Publication: Usenix Security Symposium, 2002 Presenter:
CSCI 530 Lab Intrusion Detection Systems IDS. A collection of techniques and methodologies used to monitor suspicious activities both at the network and.
1 Figure 4-16: Malicious Software (Malware) Malware: Malicious software Essentially an automated attack robot capable of doing much damage Usually target-of-opportunity.
Click to add Text Automated Worm Fingerprinting Sumeet Singh, Cristian Estan, George Varghese and Stefan Savage Department of Computer Science and Engineering.
DoWitcher: Effective Worm Detection and Containment in the Internet Core S. Ranjan et. al in INFOCOM 2007 Presented by: Sailesh Kumar.
Hamsa: Fast Signature Generation for Zero-day Polymorphic Worms with Provable Attack Resilience Zhichun Li, Manan Sanghi, Yan Chen, Ming-Yang Kao and Brian.
Detection Unknown Worms Using Randomness Check Computer and Communication Security Lab. Dept. of Computer Science and Engineering KOREA University Hyundo.
MonNet – a project for network and traffic monitoring Detection of malicious Traffic on Backbone Links via Packet Header Analysis Wolfgang John and Tomas.
Modeling Worms: Two papers at Infocom 2003 Worms Programs that self propagate across the internet by exploiting the security flaws in widely used services.
Learning Rules for Anomaly Detection of Hostile Network Traffic Matthew V. Mahoney and Philip K. Chan Florida Institute of Technology.
Automatically Generating Models for Botnet Detection Presenter: 葉倚任 Authors: Peter Wurzinger, Leyla Bilge, Thorsten Holz, Jan Goebel, Christopher Kruegel,
IEEE Communications Surveys & Tutorials 1st Quarter 2008.
Wide-scale Botnet Detection and Characterization Anestis Karasaridis, Brian Rexroad, David Hoeflin In First Workshop on Hot Topics in Understanding Botnets,
1 Modeling, Early Detection, and Mitigation of Internet Worm Attacks Cliff C. Zou Assistant professor School of Computer Science University of Central.
Defending Against Internet Worms: A Signature-Based Approach Aurthors: Yong Tang, and Shigang Chen Publication: IEEE INFOCOM'05 Presenter : Richard Bares.
1 HoneyNets. 2 Introduction Definition of a Honeynet Concept of Data Capture and Data Control Generation I vs. Generation II Honeynets Description of.
Internet Quarantine: Requirements for Containing Self-Propagating Code David Moore, Colleen Shannon, Geoffrey M.Voelker, Stefan Savage University of California,
1 On the Performance of Internet Worm Scanning Strategies Authors: Cliff C. Zou, Don Towsley, Weibo Gong Publication: Journal of Performance Evaluation,
Polygraph: Automatically Generating Signatures for Polymorphic Worms James Newsome, Brad Karp, and Dawn Song Carnegie Mellon University Presented by Ryan.
Worm Defense Alexander Chang CS239 – Network Security 05/01/2006.
HoneyComb HoneyComb Automated IDS Signature Generation using Honeypots Prepare by LIW JIA SENG Supervisor : AP. Dr. Mohamed Othman.
Polygraph: Automatically Generating Signatures for Polymorphic Worms Presented by: Devendra Salvi Paper by : James Newsome, Brad Karp, Dawn Song.
DoS/DDoS attack and defense
1 On the Performance of Internet Worm Scanning Strategies Cliff C. Zou, Don Towsley, Weibo Gong Univ. Massachusetts, Amherst.
1 Modeling, Early Detection, and Mitigation of Internet Worm Attacks Cliff C. Zou Assistant professor School of Computer Science University of Central.
Automated Worm Fingerprinting Authors: Sumeet Singh, Cristian Estan, George Varghese and Stefan Savage Publish: OSDI'04. Presenter: YanYan Wang.
1 Monitoring and Early Warning for Internet Worms Authors: Cliff C. Zou, Lixin Gao, Weibo Gong, Don Towsley Univ. Massachusetts, Amherst Publish: 10th.
UDP & TCP Where would we be without them!. UDP User Datagram Protocol.
HoneyStat: Local Worm Detection Using Honeypots David Dagon, Xinzhou Qin, Guofei Gu, Wenke Lee, et al from Georgia Institute of Technology Authors: The.
Network-based and Attack-resilient Length Signature Generation for Zero-day Polymorphic Worms Zhichun Li 1, Lanjia Wang 2, Yan Chen 1 and Judy Fu 3 1 Lab.
Polygraph: Automatically Generating Signatures for Polymorphic Worms Authors: James Newsome (CMU), Brad Karp (Intel Research), Dawn Song (CMU) Presenter:
Worms 1. Viruses don’t break into your computer – they are invited by you – They cannot spread unless you run infected application or click on infected.
Internet Quarantine: Requirements for Containing Self-Propagating Code
POLYGRAPH: Automatically Generating Signatures for Polymorphic Worms
Worm Origin Identification Using Random Moonwalks
Polygraph: Automatically Generating Signatures for Polymorphic Worms
Modeling, Early Detection, and Mitigation of Internet Worm Attacks
CSE551: Introduction to Information Security
Introduction to Internet Worm
Presentation transcript:

Usenix Security 2004 Autograph Toward Automated, Distributed Worm Signature Detection Hyang-Ah KimBrad Karp Carnegie Mellon UniversityIntel Research & Carnegie Mellon University

Usenix Security Internet Worm  Large costs due to lost productivity Code Red: $2.6 billion, Slammer: $1 billion  Vulnerabilities still plentiful  Smarter, faster, and more malicious worms easily possible [Staniford et al., 2002] Internet Worm Quarantine Techniques  Destination port blocking  Infected source host IP blocking  Content-based blocking Content-based blocking [Moore et al., 2003]

Usenix Security Content-based Blocking 05:45: > :. 0:1460(1460) ack 1 win 8760 (DF) 0x dc 84af f ac4 0x0010 d14e eb80 06b e86 fe57 440b 7c3b.N.....P^..WD.|; 0x c8f f P."8l...GET./def 0x c74 2e f ault.ida?XXXXXXX 0x XXXXXXXXXXXXXXXX x00e XXXXXXXXXXXXXXXX 0x00f XXXXXXXXXXXXXXXX 0x XXXXXXXXXXXXXXXX 0x XXXXXXXXX%u9090% 0x u6858%ucbd3%u780 0x %u9090%u6858%uc 0x bd3%u7801%u9090% 0x u6858%ucbd3%u780 0x %u9090%u9090%u8 0x %u00c3%u0003% 0x u8b00%u531b%u53f 0x f%u0078%u0000%u0 0x01a0 303d f31 2e30 0d0a 436f0=a.HTTP/1.0..Co..... Signature : A Payload Content String Specific To A Worm Signature for CodeRed II

Usenix Security Content-based Blocking Our network X Traffic Filtering Internet Signature for CodeRed II  Can be used by Bro, Snort, Cisco’s NBAR,...

Usenix Security Signature derivation is too slow Current Signature Derivation Process  New worm outbreak  Report of anomalies from people via phone/ /newsgroup  Worm trace is captured  Manual analysis by security experts  Signature generation  Labor-intensive, Human-mediated

Usenix Security Goal Automatically generate signatures of previously unknown Internet worms  as quickly as possible  as accurately as possible

Usenix Security Our Work We focus on TCP worms that propagate via scanning Actually, any transport  in which spoofed sources cannot communicate successfully  in which transport framing is known to monitor Worm’s payloads share a common substring  Vulnerability exploit part is not easily mutable

Usenix Security Outline Problem and Motivation Automated Signature Detection  Desiderata  Technique  Evaluation Distributed Signature Detection  Tattler  Evaluation Related Work Conclusion

Usenix Security Desiderata Automation: Minimal manual intervention Signature quality: Sensitive & specific  Sensitive: match all worms  low false negative rate  Specific: match only worms  low false positive rate Timeliness: Early detection Application neutrality  Broad applicability

Usenix Security Automated Signature Generation Step 1: Select suspicious flows using heuristics Step 2: Generate signature using content- prevalence analysis Our network Traffic Filtering Internet Autograph Monitor Signature X

Usenix Security Heuristic: Flows from scanners are suspicious  Focus on the successful flows from IPs who made unsuccessful connections to more than s destinations for last 24hours  Suitable heuristic for TCP worm that scans network Suspicious Flow Pool  Holds reassembled, suspicious flows captured during the last time period t  Triggers signature generation if there are more than  flows S1: Suspicious Flow Selection Reduce the work by filtering out vast amount of innocuous flows Autograph (s=2) Non-existent  This flow will be selected

Usenix Security S1: Suspicious Flow Selection Heuristic: Flows from scanners are suspicious  Focus on the successful flows from IPs who made unsuccessful connections to more than s destinations for last 24hours  Suitable heuristic for TCP worm that scans network Suspicious Flow Pool  Holds reassembled, suspicious flows captured during the last time period t  Triggers signature generation if there are more than  flows Reduce the work by filtering out vast amount of innocuous flows

Usenix Security S2: Signature Generation All instances of a worm have a common byte pattern specific to the worm Rationales  Worms propagate by duplicating themselves  Worms propagate using vulnerability of a service Use the most frequent byte sequences across suspicious flows as signatures How to find the most frequent byte sequences?

Usenix Security Worm-specific Pattern Detection Use the entire payloads  Brittle to byte insertion, deletion, reordering XXXXXXXX YYYY Flow 1 Flow 2 XXXXXXX YYYYY

Usenix Security Worm-specific Pattern Detection Partition flows into non-overlapping small blocks and count the number of occurrences Fixed-length Partition  Still brittle to byte insertion, deletion, reordering XXXXXXXX YYYY Flow 1 Flow 2 XXXXXXX YYYYY

Usenix Security Worm-specific Pattern Detection Content-based Payload Partitioning (COPP)  Determine boundaries of block using LBFS style  Partition if Rabin fingerprint of a sliding window matches Breakmark  Configurable parameters: content block size (minimum, average, maximum), breakmark, sliding window  Content Blocks Breakmark = last 8bits of fingerprint ( 9025 ) XXXXXXXX YYYY Flow 1 Flow 2 XXXXXXX YYYYY

Usenix Security Why Prevalence? Worm flows dominate in the suspicious flow pool Content-blocks from worms are highly ranked Nimda CodeRed2 Nimda (16 different payloads) WebDAV exploit Innocuous, misclassified

Usenix Security Select Most Frequent Content Block ABD ABE ACE AD CF CDG B f0 f1 f2 f3 f4 f5 HIJ f6 IHJ f7 GIJ f8

Usenix Security A A A E E A F C C C D D DB B B H H G G I I I J J J Select Most Frequent Content Block D C E E A A A A D F C C DG B B B H H G I I I J J J f0 f1 f2 f3 f4 f5 f6 f7 f8 f0 C F f1 C D G f2 A B D f3 A C E f4 A B E f5 A B D f6 H I J f7 I H J f8 G I J

Usenix Security Select Most Frequent Content Block A B D A BE A C E A D C F C D G B H I J I H J GI J f0 C F f1 C D G f2 A B D f3 A C E f4 A B E f5 A B D f6 H I J f7 I H J f8 G I J p≥3p≥3 W ≥ 90% Signature:

Usenix Security Signature: A Select Most Frequent Content Block A B D A BE A C E A D C F C D G B H I J I H J GI J f0 C F f1 C D G f2 A B D f3 A C E f4 A B E f5 A B D f6 H I J f7 I H J f8 G I J p≥3p≥3 W ≥ 90%

Usenix Security Select Most Frequent Content Block B DB A A A C E E A D F C C D G B H I J I H J GI J p≥3p≥3 W ≥ 90% Signature: A f0 C F f1 C D G f2 A B D f3 A C E f4 A B E f5 A B D f6 H I J f7 I H J f8 G I J

Usenix Security Select Most Frequent Content Block F C C D G H I J I H J GI J p≥3p≥3 W ≥ 90% Signature: A f0 C F f1 C D G f2 A B D f3 A C E f4 A B E f5 A B D f6 H I J f7 I H J f8 G I J I

Usenix Security Select Most Frequent Content Block F C C DG p≥3p≥3 W ≥ 90% Signature: A f0 C F f1 C D G f2 A B D f3 A C E f4 A B E f5 A B D f6 H I J f7 I H J f8 G I J I U Signature:

Usenix Security Outline Problem and Motivation Automated Signature Detection  Desiderata  Technique  Evaluation Distributed Signature Detection  Tattler  Evaluation Related Work Conclusion

Usenix Security Behavior of Signature Generation Objectives  Effect of COPP parameters on signature quality Metrics  Sensitivity = # of true alarms / total # of worm flows  false negatives  Efficiency = # of true alarms / # of alarms  false positives Trace  Contains 24-hour http traffic  Includes 17 different types of worm payloads

Usenix Security Signature Quality Larger block sizes generate more specific signatures A range of w (90-95%, workload dependent) produces a good signature

Usenix Security Outline Problem and Motivation Automated Signature Detection  Desiderata  Technique  Evaluation Distributed Signature Detection  Tattler  Evaluation Related Work Conclusion

Usenix Security Problem: Slow Payload Accumulation Before signature generation,  Detect scanners (possibly infected hosts)  Aggressiveness (s ) of flow selection heuristics  Accumulate payloads enough for content analysis  Earliness (  ) of signature generation trigger Infected vulnerable hosts (  =5, 63 monitors) Info Sharing Autograph Monitor Aggressiveness of flow selection s = 1s = 4 None Luckiest2%60% Average25% -- Scanners, Signatures Average<1%15%

Usenix Security Faster Signature Detection Share the scanner information with others Our network Traffic Filtering Internet Autograph Monitor Network A Network C Network B tattler

Usenix Security Benefit from tattler Objective  Measure the detection speed and the signature quality Methodology  Trace generation Background noise flows from the real 24hr trace Captured worm flows from simulation (63 monitors)  Signature generation with COPP varying suspect flow pool size Metrics  Percentage of infected hosts  Number of unspecific signatures (that causes false positives)

Usenix Security Tradeoff: Speed vs. Quality Decreasing s and  parameters  faster signature generation  more false positives x x  = 15, s = 2, < 2% infected

Usenix Security Attacks Overload due to flow reassembly  Multiple instances of Autograph on separate HW (port-disjoint)  Suspicious flow sampling under heavy load Abuse Autograph for DoS: pollute suspicious flow pool  Port scan and then send innocuous traffic  Distributed verification of signatures at many monitors  Source-address-spoofed port scan  Reply with SYN/ACK on behalf of non-existent hosts/services

Usenix Security Related Work EarlyBird [Singh et al. 2003]  Pure content-based approach first, then address dispersion heuristic  Targets a single high speed link; no flow reassembly HoneyComb [Kreibich et al. 2003]  Signature detection by Honeypot & LCS algorithm  Targets host-based deployment; small number of flows Honeyd [Provos 2003]  Use distributed honeypots to gather worm payloads & infected IP addresses quickly  Focus on harvesting malicious traffic DOMINO [Yegneswaran et al. 2004]  Scanner IP information sharing among distributed nodes  Distributed monitoring assures earlier & more accurate detection

Usenix Security Future Work Online evaluation with diverse traces Deployment on distributed sites Broader set of suspicious flow selection heuristics  Non-scanning worms (ex. hit-list worms, topological worms, worms)  UDP worms Distributed agreement for signature quality testing

Usenix Security Conclusion Stopping spread of novel worms requires early generation of signatures Autograph: automated signature detection system  Suspicious flow selection → Content prevalence analysis  COPP: robustness against payload variability  Distributed monitoring: faster signature generation Autograph finds sensitive & specific signatures early in real network traces