Part 5: Anti-Reverse-Engineering Chapter 15: Anti-Disassembly Chapter 16: Anti-Debugging Chapter 17: Anti-Virtual Machine Techniques Chapter 18: Packing and Unpacking

Chapter 15: Anti-Disassembly

Anti-Disassembly 1. Understanding Anti-Disassembly 2. Defeating Disassembly Algorithms 3. Anti-Disassembly Techniques 4. Obscuring Flow Control 5. Thwarting Stack-Frame Analysis

1. Understanding Anti-Disassembly Special code to cause disassembly analysis to produce incorrect program listings Goal is to delay or prevent analysis of malicious code Thwart automated and manual analysis Tricking disassembly at an incorrect offset Naïve disassembly p. 328, Loc. 8288 Corrected disassembly p. 329, Loc. 8300

2. Defeating Disassembly Algorithms Two types of algorithms Linear disassembly Iterate over a block of code, disassembling one instruction at a time linearly Decode blindly from start to end, ignores flow-control instructions that cause only a part of the buffer to execute Example previously on p. 328-329 Bogus, injected opcode 0xE8 assumed to be “call” Next 4 bytes assumed to be target (can instead contain malicious code) Example p. 330, Loc. 8344 Jump table correctly disassembled Turns to code in a linear disassembler (p. 331, Loc. 8357)

Defeating Disassembly Algorithms Two types of algorithms Flow-oriented disassembly Builds list of locations to assemble by examining code from entry Example p. 332, Loc. 8368: Decoding correctly stops after unconditional jmp p. 332, Loc. 8388: Linear disassembly parses ‘Failed’ as code p. 333, Loc. 8401: Call-pop to get pointer to "hello" into a register p. 334, Loc. 8415: Instructions after call disassembled first parse string as code p. 334, Loc 8428: Manual cleanup Issues with flow-oriented disassembly On conditional branch, which branch should be followed first? On a call, do you disassemble the call or instructions after first? Arbitrary results based on your choice!

3. Anti-Disassembly Techniques Fake conditionals Take advantage of choice in disassembly Jump instructions with the Same Target Back-to-back conditional jumps with the same target Consider Figure 15-2, p. 335, Loc. 8477 Unconditional jump that *always* skips bogus 0xE8 Confused as code if fall-through followed firstin flow-disassembler Incorrect disassembly at p. 335, Loc. 8448 Correct disassembly at p. 335, Loc. 8462 Toggle bytes from code to data using “C” and “U” Jump instruction with a constant condition Figure 15-3, p. 336, Loc. 8505 XOR reg,reg followed by jz Incorrect disassembly: p. 335, Loc. 8477 Correct disassembly: p. 336, Loc. 8491 Note: Methods use a “rogue” call/jmp byte (0xE9 or 0xE8)

Anti-Disassembly Techniques Impossible disassembly Using a single byte in two instructions Disassembler limited to picking one interpretation, but processor can use both Inward jump (Figure 15-4, p. 337, Loc. 8517) More complex case (Figure 15-5, p. 338, Loc. 8533) Incorrect disassembly (p. 338, Loc. 8541) Correct disassembly (p. 339, Loc. 8555) Must manually patch code or NOP out bogus instructions IDAPython script on p. 339, Loc. 8574

4. Obscuring Flow Control Function pointers Locations resolved at run-time Hard to statically reverse engineer p. 341, 2 calls to sub_4011C0 via function pointers not labeled Return pointer abuse Modify return value on stack at run-time (return-oriented programming) call $+5 coupled with rogue ret instruction on p. 342 retn makes it appear sub_4011C0 is finished, but control goes just beyond retn instruction

Obscuring Flow Control Misusing structured exception handlers SEH allows program to handle error conditions intelligently Uses a stack to manage (FS segment register) Example on p. 346 Craft pointer to exception routine (0x401080) via top two instructions Push address onto stack as exception handler in third instruction Trigger exception (divide-by-zero). Routine at 0x401080 is not disassembled Use IDA to manually disassemble

5. Thwarting Stack-Frame Analysis Stack-frame analysis dependent upon compiler used Calling conventions vary Custom management also possible such as management using esp directly (Listing 15-1, p. 347) does not use ebp, breaking IDA Pro analysis cmp instruction sets up a fake conditional, but IDA Pro traces incorrect branch Assumes “add esp, 104h” is actually executed and shows esp getting into an incorrect range (at -F8) cmp should never trigger

Talk https://www.youtube.com/watch?v=wdFLK_eX0QY

In-class exercise Lab 15-01, 15-02

Chapter 16: Anti-Debugging

Anti-Debugging Anti-analysis technique for malware to recognize when it is under the control of a debugger Slow down analysis as much as possible to increase window of vulnerability Hundreds of techniques

Anti-Debugging 1. Windows Debugger Detection 2. Identifying Debugger Behavior 3. Interfering with Debugger Functionality 4. Debugger Vulnerabilities

1. Windows Debugger Detection Using the Windows API IsDebuggerPresent() returns 0 if no debugger attached by searching the Process Environment Block for field IsDebugged CheckRemoteDebuggerPresent() allows one to check the IsDebugged flag on other processes NTQueryInformationProcess using value ProcessDebugPort

Windows Debugger Detection Using the Windows API OutputDebugString() (Listing 16-1, p. 353) Get errorValue initially Call function If no debugger attached when attempting to output debug string, errorValue changed to indicate Do the malicious thing

Windows Debugger Detection Direct checks of structures in memory Bypass Windows API Preferred by malware since API calls can be hooked by anti-virus

Windows Debugger Detection Direct checks of structures in memory BeingDebugged flag Load PEB structure address fs:[30h] Access BeingDebugged at offset 0x2 (Listing 16-2, Table 16-1, p. 354)

Windows Debugger Detection Direct checks of structures in memory Flags indicate if process heap was created by the debugger Debugger heap check Get address of process’s first heap (ProcessHeap) by loading value at 0x18 into PEB structure, then accessing flag field at 0x10 (XP) or 0x44 (Win7) (Listing 16-3, p. 355)

Windows Debugger Detection Direct checks of structures in memory Heap management different for debugged programs NtGlobalFlag check Specified at 0x68 offset in PEB. Set to 0x70 if debugged (Listing 16-4, p. 355)

Windows Debugger Detection System artifacts API hooks (OllyDbg detour of OpenProcess) Registry values used by debuggers (HKLM\....\AeDebug) Window names (e.g. OLLYDBG) Debugging services running on system File system artifacts Memory artifacts (e.g. OllyDbg stores some strings at 0x004B064B)

2. Identifying Debugger Behavior INT scanning INT 3 inserted by debugger to temporarily replace an instruction so that debug exception handler can run when software breakpoints are hit (Opcode 0xCC) Search for 0xCC in code (Listing 16-6, p. 357) Performing code checksums Malware performs checksum on its code pages and exits if tampering detected Get address Set address to start of code Check 4096 bytes for 0xCC 0xCC found!

Identifying Debugger Behavior Timing checks Malware takes timestamps and exits if there is a lag Especially effective when taken before & after an exception Implemented via rdtsc instruction (Listing 16-7, p. 358), QueryPerformanceCounter, or GetTickCount (Listing 16-8, p. 359)

Identifying Debugger Behavior Debugger artifacts INT 1 osed to single-step program in debugger Overwrites 6 bytes beyond current sp with return values for IP, CS, and Flags Put canary just beyond stack and check that it survives PUSH AX POP AX DEC SP POP BX CMP AX,BX JNE CODE_IS_TRACED Put value on stack Get stack pointer to point back to it Read it again If not the same, then single stepping has clobbered it

Identifying Debugger Behavior Debugger artifacts Force INT 1/INT 3 tracing to disable essential functions Hide critical value (e.g. decryption key) on stack directly without modifying stack pointer Debugger overwrites value if it runs Registers/flags saved by debugger on context switches Debug registers (DR0-DR7) saved on stack on context switch Set handler and force exception (divide by zero) Read and write values directly Execute exception with Trap flag set No debugger = SEH occurs since no one handles trap Debugger attached = SEH will not occur (trap sent to debugger)

3. Interfering with Debugger Functionality Using TLS (thread local storage) callbacks Debuggers typically execute until program entry point defined by the PE header TLS implemented in an executable contains a .tls section that can be executed before program entry point Malware can hide functionality in TLS Must configure debugger to pause before TLS callback code

Interfering with Debugger Functionality Using exceptions Debuggers typically configured to trap exceptions and not pass them through to program Malware probes to ensure exceptions are passed through quickly Must configure debugger to pass them through automatically

Interfering with Debugger Functionality Inserting interrupts Inserting a long loop of INT 3 instructions or 0xCD03 (STATUS_BREAKPOINT) to generate an INT 3. Trolls the reverse-engineer Doing the same with INT 2D (kernel debugger breakpoint)

Interfering with Debugger Functionality Putting payload in interrupt handler Having a debugger attached results in disabling malware Make malicious code be a part of an SEH handler INT 3 without debugger invokes SEH handler to execute malicous code INT 3 with debugger goes to next instruction unless debugger configured appropriately (Listing 16-9, p. 363) Place SEH on stack (continue) Trigger exception Code that does nothing and exits Malicious code

Interfering with Debugger Functionality Use interrupt handling to obfuscate execution Example: Running line of code Hook INT 1 Decrypt next instruction, encrypt previous one Only one instruction decrypted in memory at a time Hard to analyze Themida, last level of Microcorruption

Interfering with Debugger Functionality Modifying expected interrupt behavior Continually overwrite Interrupt Vector of INT 1/3 instructions to point to garbage code to crash debugger Turn off keyboard interrupts IN AL, 20h OR AL, 02 OUT AL, 20 <malicious code> IN AL, 20 AND AL, NOT 2 OUT AL,20

4. Debugger Vulnerabilities PE header vulnerabilities OllyDbg follows specifications of PE headers more strictly than Windows. Crashes on malformed headers that will run without debugger Code vulnerabilities Pass malformed string to crash debugger OutputDebugString vulnerable to format string vulnerability in OllyDbg v. 1.1. See fuzzing lecture Exploit instructions that OllyDbg handles differently than CPU to crash debugger Exploit exceptions that OllyDbg handles differently than CPU to crash debugger (memory handling)

In-class exercise Lab 16-01

Chapter 17: Anti-Virtual Machine Techniques

Anti-Virtual Machine Techniques Virtual machines initially used only by malware analysts Malware benefited from detecting VM (especially VMware) and shutting down to escape analysis Rollback recovery easy Portability

Anti-Virtual Machine Techniques 1. VMware Artifacts 2. Vulnerable Instructions 3. Tweaking Settings 4. Escaping the Virtual Machine

1. VMware Artifacts Process listing (Figure 17-1, p. 370)

VMware Artifacts Filesystem Registry keys (p. 371) (e.g. C:\Program Files\VMware\VMware Tools) Registry keys (p. 371)

VMware Artifacts Networking MAC addresses assigned for use by IEEE for VMware NICs begin with 00:0C:29 Memory (invariant strings in VMware virtual machine)

VMware Artifacts Example code to check Must disable checks Listing 17-1, p. 372 Must disable checks Patch condition on branch to bypass in debugger Use hex editor to modify VMware string Uninstall VMware tool being checked

2. Vulnerable Instructions Descriptor Table Instructions 3 special x86 registers for pointing to machine-wide data structures IDTR: points to Interrupt Descriptor Table Register GDTR: points to Global Descriptor Table Register (memory lookups) LDTR: points to Local Descriptor Table Register (unused in Windows) Guest VM must have a different location for these tables than Host VM VM software creates separate locations But, Guest VM can directly execute x86 instructions that directly access underlying registers to check for inconsistency (user-mode instructions not caught by hypervisor) Must NOP out these checks

2. Vulnerable Instructions Examples of Descriptor Table checks Red Pill x86 instruction sidt loads value of IDTR (Listing 17-2, p. 374) VMware sets 5th byte of IDTR to 0xFF No Pill x86 instruction sldt loads value of LDTR Zero on non-virtualized, Non-zero on virtualized x86 instruction sgdt loads value of GDTR

Vulnerable Instructions Querying the I/O communication port (Phatbot, Storm) VMware virtualizes I/O ports Port can be queried to detect presence of VMware Obtaining VMware version via IO port (Listing 17-3, p. 376) Must NOP out the check

Vulnerable Instructions Common Anti-VM instructions sidt, sgdt, sldt, smsw, str, in, cpuid 20 instructions designated by VMware as “not virtualizable” Malware will only use them if checking for VMs

3. Tweaking Settings VMware provides options to hide itself from malware Listing 17-5, p. 379 Protects against all checks implemented by ScoopyNG, a free VMware detection tool Last-resort since performance will crater if used

4. Escaping the Virtual Machine Exploiting VMware bugs to crash host or run code in it Prior exploits (now patched) include shared folder feature, drag-and-drop functionality in VMware Tools, shared clipboard, VM display function

Talk Blue Pill, Don't Tell Joanna

In-class exercise Lab 17-01

Chapter 18: Packers and Unpacking

Packers Used to shrink and encrypt malware to thwart detection by antivirus Original executable transformed to a unique self-extracting one via compression, encryption, and obfuscation Difficult to reverse-engineer Prevents static analysis since malware must be unpacked and decrypted before analysis Employs anti-disassembly, anti-debugging, and anti-VM techniques to make analysis difficult

Packers

Packers and Unpacking 1. Packer Anatomy 2. Identifying Packed Programs 3. Unpacking Options 4. Tips and Tricks for Common Packers 5. Packed DLLs

1. Packer Anatomy Unpacking Stub Small piece of code loaded by the operating system just as a normal program Unpacking stub then loads original program Step 1: Unpacking original executable into memory Loader reads PE header and copies packed sections into allocated memory normally Unpacking code called and does the same for packed code

Packer Anatomy Step 1 (cont.) Within unpacking routine, unpacking loop Save flags and all registers (PUSHFD, PUSHAD), call unpacking routine Within unpacking routine, unpacking loop 0040AC44 FFFF INVALID 0040AC4C 9C PUSHFD 0040AC4D 60 PUSHAD 0040AC4E E802000000 CALL 0040AC55 **If you step over this CALL using F10, the program will run. Thus, reload the program and step into this CALL using F8 next time. aaaaaaaa ... wwwwwwww xxxxxxxx JNZ zzzzzzzz <-- Loop back to aaaaaaaa yyyyyyyy JMP aaaaaaaa zzzzzzzz Rest of unpacking code

Packer Anatomy Step 2: Resolving imports of original executable On normal binaries, loader reads PE header to find library functions to import and their addresses Unless packed code's imports included in unpacking code's import section, unpacker must resolve imports manually using LoadLibrary and GetProcAddress (will revisit in Chapter 19)

Packer Anatomy Step 3: Transfer of execution to original execution point (OEP) POPAD followed by tail jmp to entry point (Listing 18-1, p. 392) Note: empty bytes after JMP and huge offset IDA Pro can identify JMP goes to garbage and flags it red (Figure 18-5, p. 393)

2. Identifying Packed Programs Simple indicators Program with few imports and imports are LoadLibrary and GetProcAddress IDA Pro recognizes a small amount of code Presence of UPX0 section (a specific packer) Abnormal section sizes Entropy calculation Disorder in a program much larger in encrypted and compressed payloads

3. Unpacking Options Automated static unpacking Specific to a packer (i.e. you must know which packer was used) Apply decompression and decryption based on packer type PE Explorer Supports NSPack, UPack, and UPX Automated dynamic unpacking Program is run and unpacker stub is allowed to unpack original executable Tool attempts to find when tail jump is reached, then memory is dumped and original program written to disk Fails if the end of unpacking stub is not identified properly Not many publicly available tools for this

Unpacking Options Manual dynamic unpacking Option #1: Discover packing algorithm and write a program to run it in reverse Option #2: Run unpacking stub Find OEP via stack trace Registers pushed before entry to unpacking stub Set breakpoint on %esp accessing those stack locations to catch their restoration (indicates a jump to OEP forthcoming) Find OEP via iteration Break at end of main unpacking loop, step to tail jump Then, break and dump the process out of memory (Listing 18-2 and 18-3, p. 393-394)

Unpacking Options

Unpacking Options Issues in dumping unpacked binaries Import table of function names and addresses not reconstructed Listing 18-4, p. 396 functions not labeled Addressed via manual import table patching Patch PE header to reconstruct import tables Cross-reference between OllyDbg and IDA Pro to patch import table with function names OllyDump plug-in for OllyDbg (performs OEP identification, import table reconstruction, entry point patching) ImpRec (Import Reconstructor tool) when OllyDump fails to build a proper import table

4. Tips and Tricks for Common Packers UPX (Ultimate Packer for eXecutables) Open-source Designed for compression not for security OllyDump finds easily using heuristics previously described PECompact Similar to UPX, uses a tail jump of jmp *eax ASPack Uses self-modifying code to thwart analysis

Tips and Tricks for Common Packers Petite Uses single-step exceptions to break into debugger Must pass single-step exceptions back to Petite or employ hardware breakpoints to find OEP WinUpack Uses PUSH followed by RET for tail jump Placed in the middle of stub (Listing 18-5, p. 399)

Tips and Tricks for Common Packers Themida Secure packer employing anti-debugging, anti-analysis, and anti-VM techniques Contains a kernel component making it difficult to follow Runs code continuously Use ProcDump to dump memory without attaching debugger Blacklisted and identified as malware whenever found

5. Packed DLLs Similar to executables Unpacking stub contained in DllMain DllMain unpacks original DLL Some debuggers execute DllMain before breaking Need to set IMAGE_FILE_HEADER values to cause DLL to be interpreted as executable to prevent

In-class exercise Lab 18-1