October 20-23rd, 2015 Sandboxing and Reasoning on Malware Infection Trees Kris Ghosh 1, Jose Morales 2, Will Casey 2 and Bud Mishra 3 1. Miami University 2. Carnegie Mellon University SEI CERT 3. Courant Inst. New York University

October 20-23rd, 2015 Outline  Problem statement -Objective and goal  Methodology  Case study  Future directions

October 20-23rd, 2015 Setting  Malware is a serious problem, and increasingly threatens trust and safety. Because it also employs deception it confounds risk estimation.  Formal understanding of high level behaviors of expressed in malware: expertise  logic formulae.  for example infection process via Malware Infection Trees (MiT).  Reasoning on MiT using temporal logic.  Challenges: Deluge of data and Imprecise observations such as the processes and files of an infection.

October 20-23rd, 2015 Previous Work  Model checking  Malware infection trees  Agent based trace learning.  Recommendation verification system. 4

October 20-23rd, 2015 Hypothesis  Create a reasoning formalism on Malware Infection Trees using Model Checking under uncertainty 5

October 20-23rd, 2015 Findings 6  The formalism incorporating model checking on a simple case study was computational feasible.

October 20-23rd, 2015 Model Checking System  Kripke Structure, M. (A labeled graph structure) Property/specification  temporal Logic formula, ø Model Checking Problem : Does the model M satisfy the property, ø? Challenge in model checking: state explosion problem

October 20-23rd, 2015 Kripke Structure A Kripke Structure where  S is a finite and non empty set of states.  S 0 ⊆ S is the initial state.  R is a transition relation, R ⊆ S ∗ S such that for each s ∈ S there is at least one s′ ∈ S and (s,s’) ∈ R.  L : S  2 AP. L represents the labeling function and AP represents the set of atomic propositions An edge labeled(E)-Kripke structure has labeling on the edges. M e = where L e : R  E and E is the set of edge labels.

October 20-23rd, 2015 Model Checking : Railroad Crossing Property- Safety: No green light for the train when the gate is open. AG (green and open) Property-Liveness: The train must not wait forever. EF(green) a = green and open b = red and open c = red and closed

October 20-23rd, 2015 Malware Infection Trees(MiT)-1  Captures the infection strategies of malware  Show the relationships between files and processes involved in malicious activity  Constructed using abstract rules  Focuses on file and memory infection  Self replication Into a new file Into an existing file  Code injection Direct code into another process’s memory Load dll into another process’s memory

October 20-23rd, 2015 MiT ….  Can be built in real time and post infection  Abstract construction rules apply to any OS  Identification vectors:  SSDT hooking  Minifilters  API scraping (PIN)  Strace (linux)  Log files MiT graph of Backdoor.Win32.Poison

October 20-23rd, 2015 Figure 1. Model Checking on Model Abstractions of MiT Model Checking MiT

October 20-23rd, 2015 Preliminaries- MDP An MDP where  S is a finite set of states.  S 0 is the initial state.  A is the finite set of actions. 1  P : S ∗ A ∗ S  [0, 1] and for all a ∈ A; for all s ∈ Σ s′ ∈ S P(s, a, s′) = 1  L : S  2 AP. L represents the labeling function and AP represents the set of atomic propositions. MDP is a generalization of a DTMC (Discrete Time Markov Chain)

October 20-23rd, 2015 Model Abstraction of MiT  Construction of transition system (Kripke Structure) is directly from MiT.  Labels on the states are files.  Labels on the edges are process or file rules.  An edge, represents file m’ is created by file m by rule z.  The model is total by creation of an ∊- transition where there is a transition(edge) on the state (self loop)

October 20-23rd, 2015 Modeling of MiT construction Rules Modeling the transition, f  f’ and how it fits the MiT rules. Infection via self replication (f1): The file, f is transformed to f’ under the rule, f1. Infection via arbitrary file creation (f2) :The file, f is transformed to f’ under the rule f2. Infection via arbitrary file write modification (f3) : The rule is f3. Infection via dynamic code injection of a current running process (p1) : A static file, f, an image of a process transforms into a file f’ under the rule, p1. Infection via process spawning(p2): Identical reasoning of p1

October 20-23rd, 2015 Case Study: MiT Graph of Back-Door, Win 32.Poison

October 20-23rd, 2015 Model using DTMC Every transition from a state are represented as equally likely events. Example: The fragment of the PRISM code (poison = true) -> 0.2: (one_bat’ = true) + 0.2: (wscript_exe’ = true) + 0.2:(one_reg’ = true) + 0.2:(one_vbs’ = true) + 0.2: (svchest_exe’ = true); The transitions from the state labeled with poison to the state labeled with one_bat, wscript_exe, one_reg, one_vbs and svchest_exe. The probabilities of 0.2 illustrates the event to reach each of the states are equally likely.

October 20-23rd, 2015 Model Using MDP  Nondeterminsm in the Model Each transition with same labels are assumed to be equally likely.  [p1](poison = true) -> (wscript_exe’ = true);  [f2](poison = true) -> 0.5:(one_bat’ = true) + 0.5:(one_reg’ = true);  [f1](poison = true) -> 0.5:(one_vbs’ = true) + 0.5:(svchest_exe’ = true); The transitions from the state labeled with poison to the state labeled with wscript_exe, one_bat, one_reg, one_vbs and svchest_exe. The transitions that has identical edge labels are assumed to the equally likely.

October 20-23rd, 2015 Sample queries evaluated.  Query 1: The maximum probability to reach a state labeled with attrib exe is PCTL formula for the MDP model is, Pmax? = (true U (attrib_exe = true))  Query 2 The probability that svchvst exe is less than.1. PCTL formula, P<0:1(F(svchvst _exe = true))  Query 3: The probability that reg exe is within 2 steps from wscript exe is atleast PCTL formula, P ≥ 0:98 [(wscript exe = true)U<=2(reg_exe = true)].  Query 4: From the initial state, Poison the probability that wscript exe is in constructed before ping_exe greater than equal to PCTL formula, (poison = true) => P ≥ 0:99 [(wscript exe = true)U(ping exe = true)] The times to generate the simulation model and evaluate the queries were less than one second.

October 20-23rd, 2015 Impact: 20  A Sandboxing and reasoning technique to evaluate potential malware infection.

October 20-23rd, 2015 Broader Contributions  Creates a predictive tool for malware infection  Addresses uncertainty in the prediction.  Can be create a formalism that is able to answer the potential malware infection? If yes, is there a statistical bound in the process 21

October 20-23rd, 2015 Summary and Conclusions  A formalism using model checking on MiT has been created.  Uncertainty has been incorporated using stochastic models.  Computational Feasibility on a simple case study has been evaluated. 22

October 20-23rd, 2015 Remaining Questions  Future directions include performing analysis on large sized MiT with Bayesian statistical model checking.  Causality of the events for creation of files and processes will be in studied in detail  Reasoning using Bayesian statistical model checking is a way to compute threshold values of probabilities which could predict the potential of existence of malware. Detection of potential suspicious creation of file and process in a sandboxing environment will be addressed.

October 20-23rd, 2015 Acknowledgements This material is based upon work funded and supported by the Department of Defense under Contract No. FA C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. This material hasbeen approved for public release and unlimited distribution. DM

October 20-23rd, 2015 Thank You  Q &A

October 20-23rd, 2015 Kris Ghosh Jose Andre Moraeles Willam Casey Bud Mishra 26 Contact Details