1 Detection of Injected, Dynamically Generated, and Obfuscated Malicious Code (DOME) Subha Ramanathan & Arun Krishnamurthy Nov 15, 2005.

Slides:



Advertisements
Similar presentations
Chapter 15 Computer Security Techniques
Advertisements

Ensuring Operating System Kernel Integrity with OSck By Owen S. Hofmann Alan M. Dunn Sangman Kim Indrajit Roy Emmett Witchel Kent State University College.
Operating System Security : David Phillips A Study of Windows Rootkits.
Mobile Code Security Aviel D. Rubin, Daniel E. Geer, Jr. MOBILE CODE SECURITY, IEEE Internet Computing, 1998 Minkyu Lee
Chapter 3 (Part 1) Network Security
Software-based Code Attestation for Wireless Sensors.
Silberschatz, Galvin and Gagne  Operating System Concepts The Security Problem A system is secure iff its resources are used and accessed as.
Cyber Threat Analysis  Intrusions are actions that attempt to bypass security mechanisms of computer systems  Intrusions are caused by:  Attackers accessing.
Software Security Threats Threats have been an issue since computers began to be used widely by the general public.
________________ CS3235, Nov 2002 Viruses Adapted from Pfleeger[Chap 5]. A virus is a program [fragment] that can pass on malicious code [usually itself]
Anomaly Detection Using Call Stack Information Security Reading Group July 2, 2004 Henry Feng, Oleg Kolesnikov, Prahlad Fogla, Wenke Lee, Weibo Gong Presenter:
@ NCSU Zhi NCSU Xuxian Microsoft Research Weidong Microsoft NCSU Peng NCSU ACM CCS’09.
Efficient Instruction Set Randomization Using Software Dynamic Translation Michael Crane Wei Hu.
Department Of Computer Engineering
Address Obfuscation: An Efficient Approach to Combat a Broad Range of Memory Error Exploits Sandeep Bhatkar, Daniel C. DuVarney, and R. Sekar Stony Brook.
Henric Johnson1 Chapter 10 Malicious Software Henric Johnson Blekinge Institute of Technology, Sweden
1 Chap 10 Malicious Software. 2 Viruses and ”Malicious Programs ” Computer “Viruses” and related programs have the ability to replicate themselves on.
Vulnerability-Specific Execution Filtering (VSEF) for Exploit Prevention on Commodity Software Authors: James Newsome, James Newsome, David Brumley, David.
Computer Viruses Preetha Annamalai Niranjan Potnis.
Protecting Web 2.0 Services from Botnet Exploitations Cybercrime and Trustworthy Computing Workshop (CTC), 2010 Second Nguyen H Vo, Josef Pieprzyk Department.
Improving Intrusion Detection System Taminee Shinasharkey CS689 11/2/00.
Preventing SQL Injection Attacks in Stored Procedures Alex Hertz Chris Daiello CAP6135Dr. Cliff Zou University of Central Florida March 19, 2009.
WHAT IS VIRUS? NAE GRAND CHALLENGE SECURE CYBERSPACE.
 a crime committed on a computer network, esp. the Internet.
Virus and Antivirus Team members: - Muzaffar Malik - Kiran Karki.
MANAGEMENT ANTIMALWARE PLATFORM Microsoft Malware Protection Center Dynamic Signature Svc Available only in Windows 8 Endpoint Protection Management.
Copyright 2004 Sheng Bai The Classification and Detection of Computer Worms ( survey report) Instructor: Dr. A. K. Aggarwal Session: Winter 2004.
BLENDED ATTACKS EXPLOITS, VULNERABILITIES AND BUFFER-OVERFLOW TECHNIQUES IN COMPUTER VIRUSES By: Eric Chien and Peter Szor Presented by: Jesus Morales.
A virus is software that spreads from program to program, or from disk to disk, and uses each infected program or disk to make copies of itself. Basically.
CHAPTER 14 Viruses, Trojan Horses and Worms. INTRODUCTION Viruses, Trojan Horses and worm are malicious programs that can cause damage to information.
1 What is a computer virus? Computer program Replicating Problematic "Event" Types Detection and prevention.
1 Higher Computing Topic 8: Supporting Software Updated
1 Chap 10 Virus. 2 Viruses and ”Malicious Programs ” Computer “Viruses” and related programs have the ability to replicate themselves on an ever increasing.
10/11/2015 Computer virus By Al-janabi Rana J 1. 10/11/2015 A computer virus is a computer program that can copy itself and infect a computer without.
Ether: Malware Analysis via Hardware Virtualization Extensions Author: Artem Dinaburg, Paul Royal, Monirul Sharif, Wenke Lee Presenter: Yi Yang Presenter:
Virus Detection Mechanisms Final Year Project by Chaitanya kumar CH K.S. Karthik.
Countering Kernel Rootkits with Lightweight Hook Protection Presented by: Hector M Lugo-Cordero, MS CAP 6135 March 24, 2011.
Operating system Security By Murtaza K. Madraswala.
AccessMiner Using System- Centric Models for Malware Protection Andrea Lanzi, Davide Balzarotti, Christopher Kruegel, Mihai Christodorescu and Engin Kirda.
HIPS Host-Based Intrusion Prevention System By Ali Adlavaran & Mahdi Mohamad Pour (M.A. Team) Life’s Live in Code Life.
 Focus on various part of the operating system can achieve the security and protection according to the organization’s requirement.  External and internal.
Week 10-11c Attacks and Malware III. Remote Control Facility distinguishes a bot from a worm distinguishes a bot from a worm worm propagates itself and.
MICHALIS POLYCHRONAKIS(COLUMBIA UNIVERSITY,USA), KOSTAS G. ANAGNOSTAKIS(NIOMETRICS, SINGAPORE), EVANGELOS P. MARKATOS(FORTH-ICS, GREECE) ACSAC,2010 Comprehensive.
Stamping out worms and other Internet pests Miguel Castro Microsoft Research.
RIVERSIDE RESEARCH INSTITUTE Deobfuscator: An Automated Approach to the Identification and Removal of Code Obfuscation Eric Laspe, Reverse Engineer Jason.
Dynamic Instruction Sequences Monitor for Virus Detection Jianyong Dai, Ratan Guha, Joohan Lee Wednesday, January 28, 2009 Cho, Ho-Gi.
November 19, 2008 CSC 682 Use of Virtualization to Thwart Malware Written by: Ryan Lehan Presented by: Ryan Lehan Directed By: Ryan Lehan Produced By:
Search Worms, ACM Workshop on Recurring Malcode (WORM) 2006 N Provos, J McClain, K Wang Dhruv Sharma
Computer Systems Viruses. Virus A virus is a program which can destroy or cause damage to data stored on a computer. It’s a program that must be run in.
CE Operating Systems Lecture 2 Low level hardware support for operating systems.
Introduction Program File Authorization Security Theorem Active Code Authorization Authorization Logic Implementation considerations Conclusion.
14.1 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts with Java – 8 th Edition Protection.
Tamper Resistant Software: An Implementation By David Aucsmith, IAL In Information Hiding Workshop, RJ Anderson (ed), LNCS, 1174, pp , “Integrity.
Embedded Real-Time Systems Processing interrupts Lecturer Department University.
Antivirus Software Technology By Mitchell Zell. Intro  Computers are vulnerable to attack  Most common type of attack is Malware  Short for malicious.
Protecting Computers From Viruses and Similarly Programmed Threats Ryan Gray COSC 316.
Cosc 4765 Antivirus Approaches. In a Perfect world The best solution to viruses and worms to prevent infected the system –Generally considered impossible.
Vigilante: End-to-End Containment of Internet Worms Manuel Costa, Jon Crowcroft, Miguel Castro, Antony Rowstron, Lidong Zhou, Lintao Zhang and Paul Barham.
CMSC 345 Defensive Programming Practices from Software Engineering 6th Edition by Ian Sommerville.
Techniques, Tools, and Research Issues
Operating system Security
WHAT IS A VIRUS? A Computer Virus is a computer program that can copy itself and infect a computer A Computer Virus is a computer program that can copy.
Chap 10 Malicious Software.
Computer-System Architecture
Chap 10 Malicious Software.
CSC-682 Advanced Computer Security
Malicious Software Slide Set #5 Textbook Chapter 6 Clicker Questions
Operating System Concepts
Following Malware Execution in IDA
Introduction to Internet Worm
Presentation transcript:

1 Detection of Injected, Dynamically Generated, and Obfuscated Malicious Code (DOME) Subha Ramanathan & Arun Krishnamurthy Nov 15, 2005

2 Presentation Outline Introduction to Malicious Code and DOME Introduction to Malicious Code and DOME Areas of Coverage Areas of Coverage Detection Techniques Detection Techniques Proof of Concepts Study Proof of Concepts Study Deployment Options Deployment Options Related Work Related Work Conclusion Conclusion

3 Introduction to Malicious Code (MC) Definition Definition Any type of code that can potentially harm computers and networks. Any type of code that can potentially harm computers and networks. Classes of MC Classes of MC Injected Code – Code that is introduced into a process’ address space at runtime. Injected Code – Code that is introduced into a process’ address space at runtime. Worms using buffer-overflow attacks Worms using buffer-overflow attacks Dynamically Generated Code – Code that is created by a process at runtime. Dynamically Generated Code – Code that is created by a process at runtime. Polymorphic Viruses Polymorphic Viruses Obfuscated Code – Code that is present in the process original code but whose true intentions are hidden with obscure calculations and data manipulation. Obfuscated Code – Code that is present in the process original code but whose true intentions are hidden with obscure calculations and data manipulation.

4 Introduction to DOME A host-based technique for detecting several general classes of malicious code in software executables. A host-based technique for detecting several general classes of malicious code in software executables. Uses static analysis to identify the locations of system calls within the software executables, and monitors the executables at runtime to verify the location. Uses static analysis to identify the locations of system calls within the software executables, and monitors the executables at runtime to verify the location. Simple to understand and implement. Simple to understand and implement.

5 Limitations of DOME Cannot detect unobfuscated viruses and trojans whose code is embedded in the software executable, prior to the pre-processing by DOME Cannot detect unobfuscated viruses and trojans whose code is embedded in the software executable, prior to the pre-processing by DOME Limited to executable MC that uses Win32 functions Limited to executable MC that uses Win32 functions Does not work for worms that spread using techniques other than code injection, such as script-based worms or worms that infiltrate through drive sharing Does not work for worms that spread using techniques other than code injection, such as script-based worms or worms that infiltrate through drive sharing So, to ensure full protection from the MC threat, a system based on DOME should be deployed in conjunction with other detection-response systems So, to ensure full protection from the MC threat, a system based on DOME should be deployed in conjunction with other detection-response systems

6 DOME Areas of Coverage DOME is designed to detect the three classes of MC. DOME is designed to detect the three classes of MC. Five Assumptions Five Assumptions 1. Any code that can be classified in one of the three classes is assumed to be malicious. 2. MC interacts with the Operating System. 3. When interacting with the OS, MC uses the Win32 APIs. 4. If MC hides itself from detection by using dynamic code generation and obfuscation, its Win32 API usage is hidden as well. 5. Software executables that are to be protected has to be easily disassembled, and their Win32 APIs can be effectively monitored at run time.

7 DOME Detection Technique Preprocessing Preprocessing Disassemble software executables and analyze them to identify the instructions that call into Win32 APIs. Disassemble software executables and analyze them to identify the instructions that call into Win32 APIs. Virtual addresses of these intrusions and API names are recorded into an output file. Virtual addresses of these intrusions and API names are recorded into an output file. Monitoring and Detection Monitoring and Detection Monitor Win32 API calls at runtime. Monitor Win32 API calls at runtime. During the call, identify the instruction that produced the call and the address within the executable. During the call, identify the instruction that produced the call and the address within the executable. Validate the instruction address and the API name against the model generated during the preprocessing step. Validate the instruction address and the API name against the model generated during the preprocessing step. If it doesn’t match, then signal that a malicious code was detected. If it doesn’t match, then signal that a malicious code was detected.

8 Pre-processing Step Software executables are disassembled and analyzed to identify the instructions that call into Win32 APIs. Software executables are disassembled and analyzed to identify the instructions that call into Win32 APIs. The virtual addresses of these instructions and the API names are then recorded. Also record the addresses of instructions that occur immediately after the identified Win32 API calls – these are the return addresses for the Win32 API calls The virtual addresses of these instructions and the API names are then recorded. Also record the addresses of instructions that occur immediately after the identified Win32 API calls – these are the return addresses for the Win32 API calls The identification mechanism should be designed to differentiate between normal compiler-generated code, but none of the Win32 API calls that are intentionally hidden. The identification mechanism should be designed to differentiate between normal compiler-generated code, but none of the Win32 API calls that are intentionally hidden.

9 Monitoring Win32 API calls Number of methods can be used to monitor the Win32 API calls made by processes. Number of methods can be used to monitor the Win32 API calls made by processes. Proof-of-concept study uses direct patching method implemented by the Detours package Proof-of-concept study uses direct patching method implemented by the Detours package Detours instruments the DLLs containing the Win32 APIs at load time. Detours instruments the DLLs containing the Win32 APIs at load time. By directly patching the entry point of each Win32 API, all Win32 APIs can be monitored. By directly patching the entry point of each Win32 API, all Win32 APIs can be monitored.

10 DOME Detection Technique (Monitoring using Wrappers Example) The process makes a call into the API function (1). The process makes a call into the API function (1). The first instruction is an unconditional jump to the wrapper (2). The first instruction is an unconditional jump to the wrapper (2). The wrapper may execute pre-stub code before returning control to the Win32 API body (3 & 4). The wrapper may execute pre-stub code before returning control to the Win32 API body (3 & 4). After the API body finishes executing, control is returned to the wrapper (5), which may execute post-stub code before returning to the caller (6). After the API body finishes executing, control is returned to the wrapper (5), which may execute post-stub code before returning to the caller (6). The pre-stub code is where DOME validates the Win32 API call against the information identified in the preprocessing step. The pre-stub code is where DOME validates the Win32 API call against the information identified in the preprocessing step.

11 DOME Detection Technique (Handling Bypassability) DOME Bypassability DOME Bypassability MC can forge return address on top of runtime stack, making it appear like the address came from a legitimate program MC can forge return address on top of runtime stack, making it appear like the address came from a legitimate program Solution: Perform runtime stack verification. Solution: Perform runtime stack verification. MC can use legitimate software’s own instructions and then supply its own malicious arguments. MC can use legitimate software’s own instructions and then supply its own malicious arguments. Solution: Identifying and recording static Win32 API arguments during preprocessing and then validating them at runtime. Solution: Identifying and recording static Win32 API arguments during preprocessing and then validating them at runtime. Wrapper Bypassability Wrapper Bypassability If a MC detects a wrapper, it can manipulate the memory and disable the wrappers before calling the APIs. If a MC detects a wrapper, it can manipulate the memory and disable the wrappers before calling the APIs. MC can also directly call into the kernel, avoiding the Win32 API calls and wrappers. MC can also directly call into the kernel, avoiding the Win32 API calls and wrappers. Solution: Add a kernel level authentication mechanism that verifies that the APIs are reached only after the execution has passed through the unmodified wrappers. Solution: Add a kernel level authentication mechanism that verifies that the APIs are reached only after the execution has passed through the unmodified wrappers.

12 Proof of Concept Study (Goals) 1. It is possible to identify API calls in real-world software using static analysis. 2. It is possible to monitor API calls at runtime and to identify the instruction responsible for the observed API calls. 3. DOME is able to accurately distinguish between the normal code and the malicious code. Provided that the above 2 assertions are true. Provided that the above 2 assertions are true.

13 Proof of Concept Study (Tools Used) IDA PRO Disassembler (Preprocessing Step) IDA PRO Disassembler (Preprocessing Step) Disassembling executables. Disassembling executables. Identifying and annotating instructions making Win32 API calls. Identifying and annotating instructions making Win32 API calls. Detours Wrapper Package (Monitoring Step) Detours Wrapper Package (Monitoring Step) Monitors Win32 API calls by directly patching each entry point. Monitors Win32 API calls by directly patching each entry point.

14 Proof of Concept Study (Sample Data – Benign)

15 Proof of Concept Study (Sample Data – Malicious)

16 Proof of Concept Study (Sample Output File)

17 Proof of Concept Study (Results) Successful in determining benign executables. Successful in determining benign executables. Did not find any unexpected API calls. Did not find any unexpected API calls. Only false positives noticed were due to dynamic binding of APIs. Only false positives noticed were due to dynamic binding of APIs. Successful in determining malicious executables. Successful in determining malicious executables. Found Win32 API calls during runtime of executables, but not found during preprocessing. Found Win32 API calls during runtime of executables, but not found during preprocessing. Gives users better understanding of how malicious codes work. Gives users better understanding of how malicious codes work. Even detected the W32-Simile Virus Even detected the W32-Simile Virus

18 Proof of Concept Study (W32-Simile Virus Detection)

19 Deployment Options Online Detection and Blocking Online Detection and Blocking Offline Software Scanning Offline Software Scanning Online and Offline Analysis Online and Offline Analysis

20 Deployment Options (Online Detection and Blocking) Blocking simple and complex viruses at real-time. Blocking simple and complex viruses at real-time. Legitimate code that uses obfuscation can still use API calls by giving DOME early notification. Legitimate code that uses obfuscation can still use API calls by giving DOME early notification. New software that was not preprocessed can still use API calls by allowing DOME to read the code and use local static analysis to determine whether that API call can be identified. New software that was not preprocessed can still use API calls by allowing DOME to read the code and use local static analysis to determine whether that API call can be identified.

21 Deployment Options (Offline Software Scanning) Similar to anti-virus scanning. Similar to anti-virus scanning. Preprocesses designated executables, then launches them to see if they produce Win32 API calls that were not identified. Preprocesses designated executables, then launches them to see if they produce Win32 API calls that were not identified. Blocks the Win32 API calls if not identified. Blocks the Win32 API calls if not identified. Executives need to be driven through all execution paths. Executives need to be driven through all execution paths. Not many practical application-independent solutions. Not many practical application-independent solutions.

22 Deployment Options (Online and Offline Analysis) Not only detects malicious code, but also pinpoints instructions belonging to it. Not only detects malicious code, but also pinpoints instructions belonging to it. Can be used to generate detection signatures, predict propagation vectors, or identify code lineage and perform attribution. Can be used to generate detection signatures, predict propagation vectors, or identify code lineage and perform attribution. Allows users to understand how malicious code works. Allows users to understand how malicious code works.

23 Related Work (Types of MC Detection) Misuse Detection Misuse Detection Attempts to find code that have malicious behaviors. Attempts to find code that have malicious behaviors. Anomaly Detection Anomaly Detection Determines the normalcy of the behavior. Determines the normalcy of the behavior. DOME is an anomaly detection technique!!!

24 Conclusion DOME uses static analysis to identify locations of Win32 API calls within executables. It then uses these locations as a model of which Win32 calls are allowed to occur at run-time. DOME uses static analysis to identify locations of Win32 API calls within executables. It then uses these locations as a model of which Win32 calls are allowed to occur at run-time. Very effective in finding injected, dynamically generated, and obfuscated MC. Very effective in finding injected, dynamically generated, and obfuscated MC. Future implementation will try to detect MC that bypasses DOME. Future implementation will try to detect MC that bypasses DOME.

25 Works Cited Jesse C. Rabek, Roger I. Khazan, Scott M. Lewandowski, Robert K. Cunningham, "Detection of injected, dynamically generated, and obfuscated malicious code", Proceedings of the 2003 ACM workshop on Rapid Malcode, Washington, DC, October, 2003, pp Jesse C. Rabek, Roger I. Khazan, Scott M. Lewandowski, Robert K. Cunningham, "Detection of injected, dynamically generated, and obfuscated malicious code", Proceedings of the 2003 ACM workshop on Rapid Malcode, Washington, DC, October, 2003, pp