Click to add Text Automated Worm Fingerprinting Sumeet Singh, Cristian Estan, George Varghese and Stefan Savage Department of Computer Science and Engineering.

Slides:



Advertisements
Similar presentations
Wenke Lee and Nick Feamster Georgia Tech Botnet and Spam Detection in High-Speed Networks.
Advertisements

Wenke Lee and Nick Feamster Georgia Tech Botnet and Spam Detection in High-Speed Networks.
Chapter 19: Computer and Network Security Techniques Business Data Communications, 6e.
Greg Williams CS691 Summer Honeycomb  Introduction  Preceding Work  Important Points  Analysis  Future Work.
CS 443 Advanced OS Fabián E. Bustamante, Spring 2005 Automated Worm Fingerprinting Sumeet Singh, Cristian Estan, George Varghese and Stefan Savage Presenter:
Automated Worm Fingerprinting [Singh, Estan et al] Internet Quarantine: Requirements for Self- Propagating Code [Moore, Shannon et al] David W. Hill CSCI.
 Looked at some research approaches to: o Evaluate defense effectiveness o Stop worm from spreading from a given host o Defend a circle of friends against.
 Well-publicized worms  Worm propagation curve  Scanning strategies (uniform, permutation, hitlist, subnet) 1.
Intrusion Detection Systems and Practices
5/1/2006Sireesha/IDS1 Intrusion Detection Systems (A preliminary study) Sireesha Dasaraju CS526 - Advanced Internet Systems UCCS.
Deterministic Memory- Efficient String Matching Algorithms for Intrusion Detection Nathan Tuck, Timothy Sherwood, Brad Calder, George Varghese Department.
Distributed Intrusion Detection Systems (dIDS) 2/10 CIS 610.
Worms: Taxonomy and Detection Mark Shaneck 2/6/2004.
Lesson 13-Intrusion Detection. Overview Define the types of Intrusion Detection Systems (IDS). Set up an IDS. Manage an IDS. Understand intrusion prevention.
Automated Worm Fingerprinting Sumeet Singh, Cristian Estan, George Varghese, and Stefan Savage Manan Sanghi.
Analysis of Anomalous Payload-based Worm Detection and Signature Generation by Ke Wang, Gabriela Cretu, Salvatore J.Stolfo Columbia University.
Algorithms for Network Security
Internet Quarantine: Requirements for Containing Self-Propagating Code David Moore et. al. University of California, San Diego.
Host Intrusion Prevention Systems & Beyond
Lecture 11 Intrusion Detection (cont)
Internet Relay Chat Chandrea Dungy Derek Garrett #29.
1 Chap 10 Malicious Software. 2 Viruses and ”Malicious Programs ” Computer “Viruses” and related programs have the ability to replicate themselves on.
Automated Worm Fingerprinting
Intrusion Prevention System. Module Objectives By the end of this module, participants will be able to: Use the FortiGate Intrusion Prevention System.
BY ANDREA ALMEIDA T.E COMP DON BOSCO COLLEGE OF ENGINEERING.
Firewall and Internet Access Mechanism that control (1)Internet access, (2)Handle the problem of screening a particular network or an organization from.
Principles of Computer Security: CompTIA Security + ® and Beyond, Third Edition © 2012 Principles of Computer Security: CompTIA Security+ ® and Beyond,
A virus is software that spreads from program to program, or from disk to disk, and uses each infected program or disk to make copies of itself. Basically.
1 Higher Computing Topic 8: Supporting Software Updated
1 Chap 10 Virus. 2 Viruses and ”Malicious Programs ” Computer “Viruses” and related programs have the ability to replicate themselves on an ever increasing.
Virus Detection Mechanisms Final Year Project by Chaitanya kumar CH K.S. Karthik.
CSCI 530 Lab Intrusion Detection Systems IDS. A collection of techniques and methodologies used to monitor suspicious activities both at the network and.
1 Figure 4-16: Malicious Software (Malware) Malware: Malicious software Essentially an automated attack robot capable of doing much damage Usually target-of-opportunity.
Detection Unknown Worms Using Randomness Check Computer and Communication Security Lab. Dept. of Computer Science and Engineering KOREA University Hyundo.
HIPS Host-Based Intrusion Prevention System By Ali Adlavaran & Mahdi Mohamad Pour (M.A. Team) Life’s Live in Code Life.
Automated Worm Fingerprinting Sumeet Singh, Cristian Estan, George Varghese, and Stefan Savage.
IEEE Communications Surveys & Tutorials 1st Quarter 2008.
The UCSD Network Telescope A Real-time Monitoring System for Tracking Internet Attacks Stefan Savage David Moore, Geoff Voelker, and Colleen Shannon Department.
Mapping Internet Sensors with Probe Response Attacks Authors: John Bethencourt, Jason Franklin, Mary Vernon Published At: Usenix Security Symposium, 2005.
Wide-scale Botnet Detection and Characterization Anestis Karasaridis, Brian Rexroad, David Hoeflin In First Workshop on Hot Topics in Understanding Botnets,
SNORT Biopsy: A Forensic Analysis on Intrusion Detection System By Asif Syed Chowdhury.
1 HoneyNets. 2 Introduction Definition of a Honeynet Concept of Data Capture and Data Control Generation I vs. Generation II Honeynets Description of.
Switch Features Most enterprise-capable switches have a number of features that make the switch attractive for large organizations. The following is a.
1 Very Fast containment of Scanning Worms By: Artur Zak Modified by: David Allen Nicholas Weaver Stuart Staniford Vern Paxson ICSI Nevis Netowrks ICSI.
Worm Defense Alexander Chang CS239 – Network Security 05/01/2006.
Search Worms, ACM Workshop on Recurring Malcode (WORM) 2006 N Provos, J McClain, K Wang Dhruv Sharma
D 陳怡安 R 解巽評 R 高榮泰 IEEE/ACM TRANSACTIONS ON NETWORKING OCTOBER 2006 Cristian Estan, George Varghese, Member, IEEE, and Michael Fisk.
Polygraph: Automatically Generating Signatures for Polymorphic Worms Presented by: Devendra Salvi Paper by : James Newsome, Brad Karp, Dawn Song.
CSCI1600: Embedded and Real Time Software Lecture 33: Worst Case Execution Time Steven Reiss, Fall 2015.
Intrusion Detection Systems Paper written detailing importance of audit data in detecting misuse + user behavior 1984-SRI int’l develop method of.
1 Modeling, Early Detection, and Mitigation of Internet Worm Attacks Cliff C. Zou Assistant professor School of Computer Science University of Central.
Automated Worm Fingerprinting Authors: Sumeet Singh, Cristian Estan, George Varghese and Stefan Savage Publish: OSDI'04. Presenter: YanYan Wang.
I NTRUSION P REVENTION S YSTEM (IPS). O UTLINE Introduction Objectives IPS’s Detection methods Classifications IPS vs. IDS IPS vs. Firewall.
Mapping Internet Sensor With Probe Response Attacks Authors: John Bethencourt, Jason Franklin, and Mary Vernon. University of Wisconsin, Madison. Usenix.
Polygraph: Automatically Generating Signatures for Polymorphic Worms Authors: James Newsome (CMU), Brad Karp (Intel Research), Dawn Song (CMU) Presenter:
Usenix Security 2004 Autograph Toward Automated, Distributed Worm Signature Detection Hyang-Ah KimBrad Karp Carnegie Mellon UniversityIntel Research &
Page 1 Viruses. Page 2 What Is a Virus A virus is basically a computer program that has been written to perform a specific set of tasks. Unfortunately,
IPv6 Security Issues Georgios Koutepas, NTUA IPv6 Technology and Advanced Services Oct.19, 2004.
Using Honeypots to Improve Network Security Dr. Saleh Ibrahim Almotairi Research and Development Centre National Information Centre - Ministry of Interior.
Cosc 4765 Antivirus Approaches. In a Perfect world The best solution to viruses and worms to prevent infected the system –Generally considered impossible.
POLYGRAPH: Automatically Generating Signatures for Polymorphic Worms
Click to edit Master subtitle style
Automated Worm Fingerprinting
CSCI1600: Embedded and Real Time Software
Chap 10 Malicious Software.
Chap 10 Malicious Software.
Automated Worm Fingerprinting
Introduction to Internet Worm
CSCI1600: Embedded and Real Time Software
Intrusion Detection Systems
Presentation transcript:

Click to add Text Automated Worm Fingerprinting Sumeet Singh, Cristian Estan, George Varghese and Stefan Savage Department of Computer Science and Engineering University of California, San Diego Presented at : Operating System Design & Implementation (OSDI) 2004 Ramanarayanan Ramani (Ram)

Overview Why Automated Systems Detecting Worms Characterize Worms Worm Containment Worm Behavior Identify Worm Signatures Earlybird System Design Statistics Conclusion

Why Automated Systems Identify worm – Manually characterize Signature – Update Antivirus & Network filters Code Red worm took 14 hours to infect Slammer took 10 minutes – no time to manually identify signature Need automatic worm signature identification & secure networks

Detecting Worms Network Telescopes : Monitor request to large unused, yet routable address space Can Identify random scan worms Cannot identify Hit-list or worms Cannot characterize the signature

Detecting Worms Using Honeypots Not allow any malicious incoming traffic Unwanted outgoing traffic : may be due to worm : identify malicious code performing this Use malicious code to identify signature Takes long time & requires manual signature identification

Detecting Worms Host-based behavioral detection Analyze patterns of system calls. (e.g.) Route Received packet to be sent Identify suspicious activity Expensive to manage Needs to employed in every system separately

Characterize Worm Characterization is the process of analyzing and identifying a new worm or exploit Create a priori vulnerability signatures Can only be applied to vulnerabilities that are already well-known and well- characterized manually

Characterize Worm First Automated System : Used by IBM for virus Allow to infect “Decoy” programs Identify invariant strings in Infected objects to characterize viruses Assumes the presence of a known instance of a virus and a controlled environment to monitor

Characterize Worm Honeycomb system of Kreibich and Crowcroft Host-based intrusion detection system Automatically generates signatures by looking for longest common subsequences among sets of strings found in message exchanges. Very slow

Characterize Worm Kim and Karp's Autograph system Autograph also uses network-level data to infer worm signatures Employ Rabin fingerprints to index counters of content substrings Use white-lists to set aside well known false positives Has extensive support for distributed deployments Relies on a pre filtering step that identifies flows with suspicious scanning activity

Worm Containment Mechanism used to slow or stop the spread of an active worm Host quarantine String-matching Connection throttling

Worm Behavior Behave quite differently from the popular client-server and peer-to-peer applications Have some common behavior patterns across worms – useful to identify and characterize them

Worm Behavior Content invariance Some or all of the worm program is invariant across every copy Some worms make use of limited polymorphism - encrypting each worm instance independently and/or randomizing filler text But still some portion is invariant

Worm Behavior Content prevalence Worms are designed foremost to spread - the invariant portion of a worm's content will appear frequently on the network as it spreads or attempts to spread

Worm Behavior Address dispersion Packets containing a live worm will tend to reflect a variety of different source and destination addresses This range increases when there is a major outbreak

Identify Worm Signatures ProcessTrafc(payload,srcIP,dstIP) 1 prevalence[payload]++ 2 Insert(srcIP,dispersion[payload].sources) 3 Insert(dstIP,dispersion[payload].dests) 4 if (prevalence[payload]>PrevalenceTh 5 and size(dispersion[payload].sources)>SrcDispTh 6 and size(dispersion[payload].dests)>DstDispTh) 7 if (payload in knownSignatures) 8 return 9 endif 10 Insert(payload,knownSignatures) 11 NewSignatureAlarm(payload) 12 endif

Identify Worm Signature This method is called Content Sifting Too much data to be handled in high speed networks Too many substrings need to be stored Too much time taken to process one packet

Earlybird System Design Scan network & process packets Identify repeating substrings along with list of the source & destination If repetition is over threshold, set substring to be signature & ask network security system to block packets with respective signature

Estimate Content prevalence Finding the packet payloads that appear at least x times among the N packets sent during a given interval Uses multi-stage filters with conservative update to dramatically reduce the memory footprint of the problem Append the destination port and protocol to the content before hashing Detecting repeating strings with a small fixed length B Compute a variant of Rabin fingerprints for all possible substrings of a certain length Each packet with a payload of s bytes has s - B +1 strings of length, so the memory references used per packet – very high

Estimating address dispersion Address dispersion is critical for avoiding false positives Count the distinct source IP addresses and destination IP addresses associated with each piece of content suspected of being generated by a worm Use approximate counting of distinct addresses using Bitmaps Direct Bitmaps : 32-bits. Hash Addresses to One bit and set that bit For a threshold of 30 distinct addresses – 20 bits set Ability to estimate the actual values of each counter is less

Estimating address dispersion Earlybird technique – Scaled Bitmaps Accurately estimates address dispersion using five times less memory Sub-sampling the range of the hash space (e.g.) To count up to 64 sources using 32 bits, one might hash sources into a space from 0 to 63 yet only set bits for values that hash between 0 and 31 - ignoring half of the sources We track a continuously increasing count by simply increasing this scaling factor whenever the bitmap is filled

Estimating address dispersion Once the bitmap is scaled to a new configuration, the addresses that were active throughout the previous configuration are lost and adjusting for this bias directly can lead to double counting So we use multiple bitmaps to store history – here we use 3

Estimating address dispersion

UpdateBitmap(IP) 1 code = Hash(IP) 2 level = CountLeadingZeroes(code) 3 bitcode = FirstBits(code << (level+1)) 4 if (level base and level < base+numbmps) 5 SetBit(bitcode,bitmaps[level-base]) 6 if (level == base and CountBitsSet(bitmaps[0]) == max) 7 NextConguration() 8 endif 9 endif ComputeEstimate(bitmaps,base) 1 numIPs=0 2 for i= 0 to numbmps-1 3 numIPs=numIPs+b ln(b/CountBitsNotSet(bitmaps[i])) 4 endfor 5 correction= 2(2^base - 1) / (2^numbmps - 1). b ln(b/(b - max)) 6 return numIPs 2base=(1 – (2 ^ (-numbmps)))+correction

CPU scaling Processing each packet payload as a single string is easy But when applying Rabin fingerprints, the processing of every substring of length B can overload the CPU during high traffic load – too much processing A packet with 1,000 bytes of payload and B = 40, requires processing 960 Rabin fingerprints To reduce processing time – sample the packets which are processed Randomly sampling substrings to process could cause us to miss a large fraction of the occurrences of each substring

CPU Scaling Instead use value sampling and select only those substrings for which the fingerprint matches a certain pattern – like last six bits are 0 The probability of detecting a worm with a signature of length x Probability of tracking a worm with a signature of 100 bytes is 55%, but for a worm with a signature of 200 bytes it increases to 92%, and for 400 bytes to 99.64%

Complete System

Program Loop ProcessPacket() 1 InitializeIncrementalHash(payload,payloadLength,dstPort) 2 while (currentHash=GetNextHash()) 3if (currentADEntry=ADEntryMap.Find(currentHash)) 4 UpdateADEntry(currentADEntry,srcIP,dstIP,packetTime) 5 if ( (currentADEntry.srcCount > SrcDispTh) and (currentADEntry.dstCount > DstDispTh) ) 6 ReportAnomalousADEntry(currentADEntry,packet) 7 endif 8 else 9 if ( MsfIncrement(currentHash) > PravalenceTh) 10 newADEntry=InitializeADEntry(srcIP,dstIP,packetTime) 11 ADEntryMap.Insert(currentHash,newADEntry) 12 endif 13 endif 14 endwhile

Statistics Implementation is written in C The aggregator also uses the MySql database to log all events Used popular rrd-tools library for graphical reporting PHP scripting for administrative control

Content prevalence threshold Using a 60 second measurement interval and a whole packet CRC, over 97 percent of all signatures repeat two or fewer times and 94.5 percent are only observed once Using a finer grained content hash or a longer measurement interval increases these numbers even further

Address dispersion threshold After 10 minutes there are over 1000 signatures with a low dispersion threshold of 2 Using a threshold of 30, there are only 5 or 6 prevalent strings meeting the dispersion criteria

Garbage Collection When the timeout is set to 100 seconds, then almost 60 percent of all signatures are garbage collected before a subsequent update Using a timeout of 1000 seconds, this number is reduced to roughly 20 percent of signatures

Positives Automatic Detection, Characterization & Containment Low processor time consumed Low memory consumption Identify new worms and produce signatures – even worms

Problems Can’t identify worms with very less or no invariant portion Can use compression modules like zip to confuse Earlybird Vulnerabilities in IPSec, SSL & VPN can’t be secured Attempt to evade our monitor through traditional IDS evasion techniques – like IP spoofing Stealth worm difficult to identify Purposely create worm defense to disallow some service by spreading similar packets

Suggestions Uncompress Packets & Identify original contents Need to have system as firewall for Secure protocols Use triggering data across time scales (In paper) or maintain history of slowly repeating data Check working of worm – see if it is really a worm in infected systems

Click to add Text Questions