Polymorphic Malware Detection Connor Schnaith, Taiyo Sogawa 9 April 2012.

Slides:



Advertisements
Similar presentations
Top-Down & Bottom-Up Segmentation
Advertisements

Decision Tree Approach in Data Mining
1 Advancing Supercomputer Performance Through Interconnection Topology Synthesis Yi Zhu, Michael Taylor, Scott B. Baden and Chung-Kuan Cheng Department.
Presented by: GROUP 7 Gayathri Gandhamuneni & Yumeng Wang.
1 Detection of Injected, Dynamically Generated, and Obfuscated Malicious Code (DOME) Subha Ramanathan & Arun Krishnamurthy Nov 15, 2005.
Addressing Diverse User Preferences in SQL-Query-Result Navigation SIGMOD ‘07 Zhiyuan Chen Tao Li University of Maryland, Baltimore County Florida International.
Linear Obfuscation to Combat Symbolic Execution Zhi Wang 1, Jiang Ming 2, Chunfu Jia 1 and Debin Gao 3 1 Nankai University 2 Pennsylvania State University.
Effective and Efficient Malware Detection at the End Host Clemens Kolbitsch, Paolo Milani TU Vienna Christopher UCSB Engin Kirda.
Models and Security Requirements for IDS. Overview The system and attack model Security requirements for IDS –Sensitivity –Detection Analysis methodology.
1 Complexity of Network Synchronization Raeda Naamnieh.
1 Learning to Detect Objects in Images via a Sparse, Part-Based Representation S. Agarwal, A. Awan and D. Roth IEEE Transactions on Pattern Analysis and.
Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach,
5/1/2006Sireesha/IDS1 Intrusion Detection Systems (A preliminary study) Sireesha Dasaraju CS526 - Advanced Internet Systems UCCS.
© 2005, it - instituto de telecomunicações. Todos os direitos reservados. Gerhard Maierbacher Scalable Coding Solutions for Wireless Sensor Networks IT.
Backtracking Reading Material: Chapter 13, Sections 1, 2, 4, and 5.
DIDS part II The Return of dIDS 2/12 CIS GrIDS Graph based intrusion detection system for large networks. Analyzes network activity on networks.
C++ fundamentals.
Network-based and Attack-resilient Length Signature Generation for Zero-day Polymorphic Worms Zhichun Li 1, Lanjia Wang 2, Yan Chen 1 and Judy Fu 3 1 Lab.
Automated malware classification based on network behavior
MutantX-S: Scalable Malware Clustering Based on Static Features Xin Hu, IBM T.J. Watson Research Center; Sandeep Bhatkar and Kent Griffin, Symantec Research.
EFFECTIVE AND EFFICIENT MALWARE DETECTION AT THE END HOST Presentation by Clark Wachsmuth C. Kolbitsch, P. M. Comparetti, C. Kreugel, E. Kirda, X. Zhou.
Hands-On Microsoft Windows Server 2008 Chapter 1 Introduction to Windows Server 2008.
Lucent Technologies – Proprietary Use pursuant to company instruction Learning Sequential Models for Detecting Anomalous Protocol Usage (work in progress)
272: Software Engineering Fall 2012 Instructor: Tevfik Bultan Lecture 17: Code Mining.
Panorama: Capturing System-wide Information Flow for Malware Detection and Analysis Authors: Heng Yin, Dawn Song, Manuel Egele, Christoper Kruegel, and.
Yin, H., Song, D., Egele, M., Kruegel, C., Kirda, E. In Proc. of the 14th ACM conference on Computer and communications security, October /9/31.
Computer Viruses Preetha Annamalai Niranjan Potnis.
Layered Approach using Conditional Random Fields For Intrusion Detection.
Stamping out worms and other Internet pests Miguel Castro Microsoft Research.
Introduction Overview Static analysis Memory analysis Kernel integrity checking Implementation and evaluation Limitations and future work Conclusions.
Behavior-based Spyware Detection By Engin Kirda and Christopher Kruegel Secure Systems Lab Technical University Vienna Greg Banks, Giovanni Vigna, and.
WHAT IS VIRUS? NAE GRAND CHALLENGE SECURE CYBERSPACE.
DISTRIBUTED DATABASES IN ADBMS Shilpa Seth
Finding dense components in weighted graphs Paul Horn
Inferring Specifications to Detect Errors in Code Mana Taghdiri Presented by: Robert Seater MIT Computer Science & AI Lab.
AccessMiner Using System- Centric Models for Malware Protection Andrea Lanzi, Davide Balzarotti, Christopher Kruegel, Mihai Christodorescu and Engin Kirda.
Automated Classification and Analysis of Internet Malware M. Bailey J. Oberheide J. Andersen Z. M. Mao F. Jahanian J. Nazario RAID 2007 Presented by Mike.
Roberto Paleari,Universit`a degli Studi di Milano Lorenzo Martignoni,Universit`a degli Studi di Udine Emanuele Passerini,Universit`a degli Studi di Milano.
Vigilante: End-to-End Containment of Internet Worms Authors : M. Costa, J. Crowcroft, M. Castro, A. Rowstron, L. Zhou, L. Zhang, and P. Barham In Proceedings.
Christopher Kruegel University of California Engin Kirda Institute Eurecom Clemens Kolbitsch Thorsten Holz Secure Systems Lab Vienna University of Technology.
MICHALIS POLYCHRONAKIS(COLUMBIA UNIVERSITY,USA), KOSTAS G. ANAGNOSTAKIS(NIOMETRICS, SINGAPORE), EVANGELOS P. MARKATOS(FORTH-ICS, GREECE) ACSAC,2010 Comprehensive.
Systems II San Pham CS /20/03. Topics Operating Systems Resource Management – Process Management – CPU Scheduling – Deadlock Protection/Security.
Stamping out worms and other Internet pests Miguel Castro Microsoft Research.
Chapter 3. Community Detection and Evaluation May 2013 Youn-Hee Han
Finding Diversity in Remote Code Injection Exploits Justin Ma, John Dunagan, Helen J. Wang, Stefan Savage, Geoffrey M. Voelker *University of California,
Mining Specifications of Malicious Behavior Mihai Christodorescu (work done at University of Wisconsin) Somesh Jha University of Wisconsin Christopher.
Search Worms, ACM Workshop on Recurring Malcode (WORM) 2006 N Provos, J McClain, K Wang Dhruv Sharma
Ensemble Learning for Low-level Hardware-supported Malware Detection
Security Vulnerabilities in A Virtual Environment
Lorenzo Martignoni, Elizabeth Stinson, Matt Fredrikson, Somesh Jha, John Mitchell RAID
Polygraph: Automatically Generating Signatures for Polymorphic Worms Presented by: Devendra Salvi Paper by : James Newsome, Brad Karp, Dawn Song.
Introduction to Active Directory
Role Of Network IDS in Network Perimeter Defense.
Network-based and Attack-resilient Length Signature Generation for Zero-day Polymorphic Worms Zhichun Li 1, Lanjia Wang 2, Yan Chen 1 and Judy Fu 3 1 Lab.
ICS 353: Design and Analysis of Algorithms Backtracking King Fahd University of Petroleum & Minerals Information & Computer Science Department.
The Canopies Algorithm from “Efficient Clustering of High-Dimensional Data Sets with Application to Reference Matching” Andrew McCallum, Kamal Nigam, Lyle.
Network Partition –Finding modules of the network. Graph Clustering –Partition graphs according to the connectivity. –Nodes within a cluster is highly.
Page 1 Viruses. Page 2 What Is a Virus A virus is basically a computer program that has been written to perform a specific set of tasks. Unfortunately,
Constraint Framework, page 1 Collaborative learning for security and repair in application communities MIT site visit April 10, 2007 Constraints approach.
SQL Database Management
Graph clustering to detect network modules
Harvesting Runtime Values in Android Applications That Feature Anti-Analysis Techniques Presented by Vikraman Mohan.
BotCatch: A Behavior and Signature Correlated Bot Detection Approach
De-anonymizing the Internet Using Unreliable IDs By Yinglian Xie, Fang Yu, and Martín Abadi Presented by Peng Cheng 03/22/2017.
All You Ever Wanted to Know About Dynamic Taint Analysis & Forward Symbolic Execution (but might have been afraid to ask) Edward J. Schwartz, Thanassis.
Intrusion Prevention Systems
CSC-682 Advanced Computer Security
ICS 353: Design and Analysis of Algorithms
Autonomous Network Alerting Systems and Programmable Networks
Presentation transcript:

Polymorphic Malware Detection Connor Schnaith, Taiyo Sogawa 9 April 2012

Motivation “5000 new malware samples per day” --David Perry of Trend Micro Large variance between attacks Polymorphic attacks Perform the same function Altered immediate values or addressing Added extraneous instructions Current detection methods insufficient Signature-based matching not accurate Behavioral-based detection requires human analysis and engineering

Malware Families Classified into related clusters (families) Tracking of development Correlating information Identifying new variants Based on similarity of code Koobface Bredolab PoisonIvy Conficker (7 mil. Infected) Source: Carrera, Ero, and Peter Silberman. "State of Malware: Family Ties." Media.blackhat.com Web. 7 Apr

~300 samples of malware with 60% similarity threshold

Current Research Techniques for identifying malicious behavior Mining and clustering Building behavior trees Industry ThreatFire and Sana Security developing behavioral-based malware detection

Design challenges Discerning malicious portions of code o Dynamic program slicing o accounting for control flow dependencies Reliable automation o Must be able to be reliable w/o human intervention o Minimal false positives

Holmes: Main Ideas Two major tasks o Mining significant behaviors from a set of samples o Synthesizing an optimally discriminative specification from multiple sets of samples Key distinction in approach o "positive" set - malicious o "negative" set - benign o Malware: fully described in the positive set, while not fully described in the negative set

Main Ideas: behavior mining Extracts portions of the dependence graphs of programs from the positive set that correspond to behaviors that are significant to the programs’ intent. The algorithm determines what behaviors are significant (next slide) Can be thought of as contrasting the graphs of positive programs against the graphs of negative programs, and extracting the subgraphs that provide the best contrast.

Main ideas: behavior mining A "behavior" is a data dependence graph G = (V, E, a, B) o V is the set of vertices that correspond to operations (system calls) o E is the edges of the graph and correspond to dependencies between operations o a is the labeling function that associates nodes with the operations they represent o B is the labeling function that associates the edges with the logic that represents the dependencies

Main ideas: behavior mining A program P exhibits a behavior G if it can produce an execution trace T with the following properties o Every operation in the behavior corresponds to an operation invocation and its arguments satisfy certain logical constraints o the logic formula on edges connecting behavior operations is satisfied by a corresponding pair of operation invocations in the trace Must capture information flow in dependence graphs o two key characteristics  the path taken by the data in the program  security labels assigned to the data source and the data sink

Security LabelDescription NameOfSelf The name of the currently executing program IsRegistryKeyForBootLis t A Windows registry key lsiting software set to start on boot IsRegistryKeyForWindows A registry key that contains configuration settings for the operating system IsSystemDirectory The Windows system directory IsRegistryKeyForBugfix The Windows registry key containing list of installed bugfixes and patches IsRegistryKeyForWindows Shell The Windows registry key controlling the shell IsDevice A named kernel device IsExecutableFile Executable file

Main ideas: behavior mining Information gain is used to determine if a behavior is significant. A behavior that is not significant is ignored when constructing the dependency graph Information gain is defined in terms of Shannon entropy and it means gaining additional information to increase the accuracy of determining if a G is in G+ or G- Shannon entropy o H(G+ U G-) corresponds to the uncertainty that a graph G belongs to G+ or G- o partition G+ and G- into smaller subsets to decrease that uncertainty o process called subgraph isomorphism

Main ideas: behavior mining A significant behavior g is a subgraph of a dependence graph in in G+ such that: Gain(G+ U G-, g) is maximized Information gain is used as the quality measure to guide the behavior mining process Some non-significant actions can get passed as significant o these actions may or may not throw off the algorithm that determines if the program is malicious

Main ideas: behavior mining Significant behaviors mined from malware Ldpinch o Leaking bugfix information over the network o Adding a new entry to the system autostart list o Bypassing firewall to allow for malicious traffic Could say any program that exhibits all three of these behaviors should be flagged malicious o This is too specific of a statement i.Doesn't account for variations within a family ii.It is known that smaller subsets of behaviors that only include one of these actions could still be malicious iii.Need discriminative specifications

Main ideas: discriminative specifications Creates clusters of behaviors that can be classified into as characteristic subset o Program matches specification if it matches all of the behaviors in a subset o "Discriminative" in that it matches the malicious but not the benign programs

Main ideas: discriminative specifications Each set of subset of behaviors induces a cluster of samples o Malicious and benign samples are mined are organized into these clusters o Goal: find an optimal clustering technique to organize the malicious into the positive subset and the benign into negative subset

Main ideas: discriminative specifications Three part algorithm o Formal concept analysis o Simulated annealing o Constructing optimal specifications Formal concept analysis o O is a cluster of samples o A is the set of mined behaviors in O o A concept is the pair (A, O) Set of concepts: {c1, c2, c3,..., cN) Behavior specification: S(c1, c2, c3,..., cN)

Main ideas: discriminative specifications Formal Concept Analysis (continued) Begins by constructing all concepts and computes pairwise intersection of the intent sets of these concepts Repeated until a fixpoint is reached and no new concepts can be constructed When algorithm terminates, left with an explicit listing of all of the sample clusters that can be specified in terms of one or more mined behaviors Goal is to find {c1, c2, c3,..., cN} such that S(c1, c2, c3,..., cN) is optimal (based on threshold)

Main ideas: discriminative specifications Simulated annealing Probabilistic technique for finding approximate solution to global optimization problem At each step, a candidate solution i is examined and one of its neighbors j is selected for comparison The algorithm moves to j with some probability A cooling parameter T is reduced throughout process and when it gets to a minimum the process stops

Main ideas: discriminative specifications Constructing Optimal Specifications Threshold t, a set containing positive and negative samples, and a set of behaviors mined with the previous process Called SpecSynth o Constructs full set of concepts o Removes redundant concepts o Run simulated annealing until convergence, then return the best solution

Holmes: Mining an Clustering

Evaluation and Results: Holmes Used six malware families to develop specifications Tested final product against 19 malware families Collected 912 malware samples and 49 benign

Holmes Continued Experiments carried over varying threshold values (t) Demonstrates high sensitivity to system accuracy Perhaps only efficient for a specific subset of malware

Holmes Scalability Worst-case complexity is exponential Behaviors of repeated executions (Stration and Delf) took hours to analyze Scalability for Holmes is a nightmare! “scary and scaled”

USENIX The Advanced Computing Systems Association (Unix Users Group) 2009 article: automatic behavior matching o Behavior graphs (slices) o Tracking data and control dependencies o Matching functions o Performance evaluations Source: Kolbitsch, Clemens. "Effective and Efficient Malware Detection at the End Host." Usenix Security Symposium (2009). Web. 8 Apr

USENIX: Producing Behavior Graphs Instruction log o Trace instruction dependencies o Slicing doesn't reflect stack manipulation Memory log o Access memory locations Partial behavior graph of Netsky (Kolbitsch et al)

USENIX: Behavior Slices to Functions Use instruction and memory log to determine input arguments Identify repeated instructions as loops Include memory read functions We can now compare to known malware

Evaluation Six families used for development (mostly mass-mailing worm) Expanded test set

Performance Evaluation Installed Internet Explorer, Firefox, Thunderbird, Putty, and Notepad on Windows XP test machine Single-core, 1.8 GHz, 1GB RAM, Pentium 4 processor

USENIX Limitations Evading system emulator o USENIX detector uses Qemu emulator o delays o time-triggered behavior o command and control mechanisms Modifying algorithms behavior o A more fundamental change, but cannot be detected using same signatures End-host based system o Cannot track network activity

Questions/Discussion