1. RAT-based APT Detection for Provenance Graph Analytics
Northwestern University
Jan. 15, 2016

2. Internal reconnaissance
Motivation
~300
APT
White
Papers
[1]
Most APTs are on Windows and involves malicious RATs
Malicious
RAT
Plant RAT on Victim’s Machine
APT lifecycle [2]
~300 white papers about APT attacks suggest that a RAT is leveraged in nearly every APT attack against Microsoft Windows systems.
Thus, we focus on identifying the existence of malicious RATs in the system for APT detection. 
A piece of software that allows a remote "operator" to control a system as if he has physical access to that system.
Initial compromise
Establish foothold
Escalate privileges
Internal reconnaissance
Move laterally
Maintain presence
Complete mission
[1]
[2] APT1: Exposing One of China‘s Cyber Espionage Units, Mandiant, 2013. 2 2

3. What is a RAT?
RAT stands for Remote Administration Tool, a piece of software that allows a remote "operator" to control a system.
Interpreted as remote administration Trojan in some white papers.
A malicious RAT is a backdoored spying horse zombie, in essence.
Trojan - Disguises its identity as legitimate program.
Backdoor – Enables remote control and receives commands.
Spyware – Gathers information stealthily and sends back to master.
Zombie – Behaves like a zombie (allows its master to perform operation through it).
Malicious RAT is equivalent to Remote Administration Trojan. But with a covert effect 3 3

4. First malicious RAT and Today
Back Orifice is one of the first RATs, released in 1998.
It includes a GUI control panel.
Spawn/kill processes, play audio files and capture videos.
Malicious RAT Today
An abundance of different RATs:
NetBus, iControl, Sub Seven, Beast Trojan, Bifrost, Blackshades, DarkComet, LANfiltrator, Win32.HsIdir, Optix Pro, …
RAT has become a service available to people without much computer knowledge.
Back Orifice was designed with a client–server architecture. 4 4

5. Architecture of a malicious RAT
Client/Server architecture
Server program installed on victim’s machine
Client program with GUI installed on C&C server
Server program is remotely controlled by Client program
RAT Server (unobtrusive)
RAT Client (GUI)
Control
Back Orifice was designed with a client–server architecture.
Response
C&C Server
Victim’s Machine 5 5

6. Typical Features of a malicious RAT
VM detection
Antivirus tool detection/disabling
File/registry/process manipulating
Shell control
Credential stealing
Webcam/microphone access
Keystroke logging
Screen capture
Network monitoring
...
Features of DarkComet 6 6

7. Malicious RAT vs Bot Targeted attack vs. random attack
APT are targeted attacks with reconnaissance while most botnets are kind of random attacks, relying on port scan for victim discovery.
Human manual operation (w/ vs w/o)
RAT is much more complex and powerful, and must require human operation, while bots are usually automatic programs without requiring human-driven activities.
Attack Intention
RATs are mainly used for spying on victims while bots are typically harvested for orchestrated attacks.
Add some survey results (reference), enterprise RAT, what the industry do with RAT detection.
So RAT usually does not automatically discovery victims.
Bots usually contains exploits and is able to infect victim by itself, while RAT is usually downloaded to the victim after a successful exploitation 7 7

8. Malicious RAT vs. Legitimate Program
Malicious RAT vs Enterprise/Commercial RAT
They resemble each other, except a few features such as VM detection and antivirus tool detection/disabling.
Malicious RAT vs other Legitimate Program
Malicious RATs have many distinct features that legitimate programs do not have:
Keystroke logging
Network connection monitoring
File/registry/process manipulating
Etc.
RATs always stay hidden while legitimate programs typically interact with users and show itself by displaying windows, buttons, dialog boxes and so on.
Base on behavior and other characteristics. Focus on these features . Differentiate the malicious from other programs using these features. 8 8

9. Related Work Very few academic works related to RAT detection
Z. Chen, P. Wei, and A. Delis. Catching Remote Administration Trojans (RATs). In Software Practice and Experience, 2008.
K. T. Gardåsen. Detecting Remote Administration Trojans through Dynamic Analysis using Finite-State Machines. Master thesis, 2014.
Industrial efforts on APT detection
Redline [1], a memory forensics tool designed by FireEye, provides host investigative capabilities to identify malicious activities through memory and file analysis.
Volatility [2] - an open source memory forensics framework for incident response and malware analysis
-> bullets, add industry efforts on RAT detection,
Zhongqiang et al. in 2008 performed packet inspection, tracked network connections, and leveraged proprietary communication features of the known RATs and heuristic rules for RAT detection. One major weakness is that their method cannot be used to capture new families of RATs.
One recent work is a 2014 master thesis. The author considered the behavior of surreptitiously monitoring the webcam, microphone, and keystrokes without user interactive as an indicator of the presence of a RAT, and proposed to use finite state machine for detecting malicious behavior by modeling the triggered API calls. However, it is too absolute to determine a RAT based on only three kinds of behaviors. 
Redline is the only industrial tool we found which is designed for APT detection. 9
[1] Redline:
[2] Volatility: 9

10. Our Detection Observations: We perform RAT-based APT detection by
API calls reveal the behavior of a RAT no matter how it disguises itself and how its code is obfuscated.
API calls are stable for a specific behavior most of the time. Similar behavior triggers similar API calls.
We perform RAT-based APT detection by
modeling the behavior of RATs by the API calls
creating the provenance graph based on the API call traces
developing signature-based and statistics-based methods for spotting suspicious behaviors
Detecting RATs is very difficult due to the fact that they resemble commercial remote administration software. Monitoring system processes to detect the execution of malicious activity has proven to be an effective approach for sniffing out a rat.
A set of API calls statically correspond to a specific behavior most of the time, which makes identifying malicious behaviors based on API calls possible. 10 10

11. Trace Collection Experiment Setup
Windows VM
Windows VM
RAT victim
RAT controller (C&C server)
Control
Response
process RAT
API capture tool
Our API calls were captured using two Windows 7 x64 virtual machines which were connected
through a virtual network, with one being the RAT controller and the other the victim.
WinAPIOverride
ApiOverride.dll
RAT Code
kernel32.dll, ntdll.dll, …
ApiCapture dll
Inject dll into RAT

12. Trace analysis: Screen Capture
DarkComet
WSARecv(0x BC, …)
// Get Instruction from C&C server (attacker).
CreateDC(…)
GetDeviceCaps(…)
CreateCompatibleDC(…)
CreateCompatibleBitmap(…)
GetDC()
SelectObject()
Share the same socket descriptor
ReleaseDC()
DeleteDC()
// send all the collected information back to the attacker
Send(0x BC, …)
WinAPI tracing: Attacking flow of DarkComet (RAT) 12 12

13. Trace analysis: Screen Capture
Pandora
WSARecv(0x B14, …)
// Get Instruction from C&C server (attacker).
GetDC(…)
Different RATs share similar API call patterns in similar behavior.
GetDeviceCaps(…)
CreateCompatibleDC(…)
SelectObject()
Pandora (RAT): Screen capture (Remote desktop)
ReleaseDC()
Share the same socket descriptor
DeleteDC()
// send all the collected information back to the attacker
Send(0x B14, …)
WinAPI tracing: Attacking flow of Pandora (RAT) 13 13

14. RAT Behavior Detection
Generate signature for RAT’s screen capture behavior:
Signature-based detection:
We generate signature based on our trace analysis.
WSARecv(fd, …)
GetDC() / CreateDC()
GetDeviceCaps()
CreateCompatibleDC …
Send()
WSARecv(fd, …)
Critical
WinAPI
Calls
Send(fd, …)
Identify critical path
Graph
signature
based
detection 14 14

15. RAT Behavior Detection (cont’d)
Statistics-based detection:
Single feature/behavior cannot alone differentiate malicious RATs from enterprise RATs.
However, they typically differ in terms of characteristics of API calls such as the frequency and the time interval between API calls.
With traces of API calls collected from real-world malicious RATs, enterprise RATs, and other legitimate programs, we develop statistics-based detection. 15 15

16. Conclusion
We identify that malicious RATs are used in most APT attacks on Windows systems.
We propose signature-based and statistics-based method to detect malicious RATs for APT detection for provenance graph analytics.
Detecting RATs is very difficult due to the fact that they resemble commercial remote administration software. Monitoring system processes to detect the execution of malicious activity has proven to be an effective approach for sniffing out a rat. 16 16

17. Thanks!
Questions? 17 17

18. Trace analysis: Get OS Information
DarkComet
WSARecv(0x BC, …)
// Get Instruction from C&C server (attacker).
GlobalMemoryStatus(…)
// Get global memory status.
NtQuerySystemInformation(0x0, …)
// Get detailed system information by classes, the 1st parameter indicate a unique “class” of system information
NtQuerySystemInformation(0x3, …)
Share the same socket descriptor
NtQuerySystemInformation(0x2, …)
Send(0x BC, …)
// send all the collected information back to the attacker
WinAPI tracing: Attacking flow of DarkComet (RAT) 18 18

19. Trace analysis: Get OS Information
DarkComet
WSARecv(0x BC, …)
WSARecv(fd, …)
GlobalMemoryStatus(…)
Critical
WinAPI
Calls
Graph
signature
based
detection
NtQuerySystemInformation(0x0, …)
NtQuerySystemInformation(0x3, …)
NtQuerySystemInformation(0x2, …)
Send(0x BC, …)
Send(fd, …) 19 19