Dean Carlson and Beth Anne Byrd CpSc 420

 What is reverse engineering?  Brief History  Usefulness  The process  Bagle Virus example

 “[T]he process of analyzing a subject system to create representations of the system at a higher level of abstraction” (Chikofsky, 1990).  Going through the software development cycle backwards

 Started as analyzing hardware in an attempt to gain an advantage.  The first time this was applied to a piece of malware was in  Bernt Fix disassembled and neutralized the Charlie virus.

 Analysis of a product  Recreating lost or nonexistent documentation  Academic use  Curiosity  With Malware  Contain it  Remove it  Prevent it

 Diffuse “time bombs”  Conficker virus ▪ The Y2K of today

1. Set up a controlled, isolated laboratory 2. Perform behavioral analysis to examine the specimen’s interactions with its environment. 3. Perform static code analysis to further understand the specimen’s inner-workings. 4. Perform dynamic code analysis to understand the more difficult aspects of the code. 5. If necessary, unpack the specimen. 6. Repeat steps 2, 3, and 4 (order may vary) until sufficient analysis objectives are met. 7. Document findings and clean-up the laboratory for future analysis.

 HOST:  Windows XP in Virtual Machine  DataRescue IDA Pro  Microsoft Visual C++ ▪ Dumpbin  UltraEdit  SERVER  Solaris 9 (SPARC)  Snoop  BIND (DNS)  GCC  GDB

 The and DNS programs on the server were setup to log all of their activity and network traffic in order to see the virus interact with the server.

 Open in IDA Pro  Breaks it down into assembly and hex

 Open in dumpbin to determine type  PE (Portable Executable)

 Walk through the virus step by step with a debugger and look at register values. Especially EAX, EIP, ZF bit of EFLAGS  EAX = return values from functions  ZF = flag used for comparisons and decisions  EIP = useful for thread usage

 Use IDA to chart subroutines

 Use IDA to identify function parameters and variables  arg_8 can be accessed by adding “10h” to the EBP Register

 Multiple Thread  Extended Instruction Pointer (EIP) doesn’t follow new threads unless specified

 The Bagle virus was not packed  Compressed or encrypted  It also was not polymorphic  Changing the assembly, usually by inserting “noop” thus changing the virus signature but not changing the effectiveness  The Bagle virus has many removal tools

 Reverse engineering malware started in 1987  It is good to contain, remove, and prevent malware  7 steps 1. Set up lab 2. Behavioral analysis 3. Static code analysis 4. Dynamic code analysis 5. Unpack 6. Repeat steps 2, 3, and 4 7. Document and clean-up