CompSci 725 Presentation by Siu Cho Jun, William

CompSci 725 Presentation Playing Inside the Black Box: Using Dynamic Instrumentation to Create Security Holes by Barton P. Miller, Mihai Christodorescu, Robert Iverson, Tevfik Kosar, Alexander Mirgorodskii, Florentina Popovici

CompSci 725 Presentation Computer Sciences Department, University of Wisconsin Paradyn Project Exploring new approaches to build scalable tools for monitoring or tuning of parallel program performance.

Outline Introduction Background Demo 1 Demo 2 Conclusion The Aim of this paper Background What is “DynInst”? Demo 1 Demo 2 Conclusion

Introduction Programs in execution have long been considered to be immutable objects Use “DynInst API” to build tools Operate on binary codes during execution

Introduction Show how Dynamic Instrumentation techniques can be used to subvert system security through vulnerabilities Provide some suggestions to compromise those vulnerabilities

Background What is “DynInst”? A post-compiler program manipulation tool Provides an Application Program Interface (API) for programming instrumentation tools with the dynamic instrumentation technology C++ class library

What is “DynInst”? (cont.) Background What is “DynInst”? (cont.) Allows tool builders build tools that can make insertion and modification to a running program Without re-compile, re-link, or even re-execute the program Machine independent

What can tools built with “DynInst API” do? Background What can tools built with “DynInst API” do? (1) inspect a running process, obtaining structural information about the program; (2) control the execution of the program; (3) cause new libraries to be dynamically loaded into the process' address space;

What can tools built with “DynInst API” do? Background What can tools built with “DynInst API” do? (cont.) (4) splice new code sequences into the running program and remove them; (5) replace individual call instructions or entire functions.

Background Terms: Mutator Mutatee Point Snippet The tool built for performing instrumentation Mutatee The program to be instrumented Point A location in a program where instrumentation can be inserted Snippet A piece of executable code to be inserted into a program at a point

Background How it works? p.s. Image extracted from the paper “Playing Inside the Black Box: Using Dynamic Instrumentation to Create Security Holes”

Background How it works? p.s. Image extracted from the paper “An API for Runtime Code Patching”

Demo 1 The Lurking Condor Job Goal To expose security vulnerabilities in a distributed computing environment

Demo 1 Background Info. Platform: Unix with Condor High-Throughput Computing System allows users to schedule and run application programs on idle hosts in widely distributed environment Users do not need to have account and privileges on other hosts

Demo 1 Condor SM: Submitting Machine EM: Execution Machine

Demo 1 Attack strategy

Demo 1 Attack strategy

Demo 1 Attack strategy

Demo 1 Attack strategy

Demo 1 Attack strategy

Vulnerability of Condor System Demo 1 Vulnerability of Condor System Condor starts a shadow process on the SM when it starts the submitted program on the remote EM The shadow process receives remote system calls from the EM and executes with the normal privileges of the submitting user

Vulnerability of Condor System (cont.) Demo 1 Vulnerability of Condor System (cont.) EM might subvert the submitted program and cause it to make inappropriate and malicious requests to the user's SM Condor use the same user id for every job on the EM, this allow the lurker process to access subsequently arriving jobs

Demo 1 Suggestions: On SM On trusted EM Create “Sandbox” around the shadow process On trusted EM Restrict the use of system call on EM Clear up after a job migrated or completed Use multiple user ID’s Malicious EM can bypass any of those measures

Subverting License Checking Demo 2 Subverting License Checking Goal Use DynInst tools to bypass license checking And attain full program functionality even when the license data could not be obtained

Demo 2 Background Info. Target application: Framemaker word processing tool from Adobe Platform: UltraSPARC IIi running Solaris License checking method in Framemaker: Obtain license data from license server and validate it

Demo 2 Attack Strategy (1) See the program as a black box and capture the I/O operations to located those were specific to contacting the license server Attach DynInst to Framemaker and trace all library functions that performs I/O operations Replace the open, read, write, send, and recv library function with custom versions which are modified to copy their data into a mirror file

Demo 2 Attack Strategy (2) Trace the flow control within the program to understand where the license checking is performed By tracing the control flow for the cases: the license server is successfully contacted the license server cannot be contacted

Demo 2 Attack Strategy (3) Determine the functions to be skipped or replaced to avoid the failure of the license check

Demo 2 Attack toolkit (1) Function Call Tracer The depth of the call stack, order of calls, and return values from each function are reported (2) Function Argument Parser For tracking the type and name of each parameter to a function in the application

Demo 2 Attack toolkit (3) Java to DynInst Compiler (JavaD) Since the DynInst API calls operate at the machine language level Write snippet in Java and translate to DynInst calls To simplify the task of creating code snippets

Demo 2 Result Successfully bypassed the licensing checks by the following steps: (1) Allowed the retrieval of the license data to fail. (2) Prevented FrameMaker from entering demo mode by deleting the function call of ChangeProductToDemo.

Demo 2 Result (cont.) (3) Bypassed the first license data validation by skipping over the sequence of code that performed it. (4) Modified all later license data validations to always succeed, regardless of the presence of the license data in memory.

Demo 2 Suggestion Basic code obfuscation techniques can be used: obscure naming of modules and functions violating modularity by having many implementations of the same functionality especially for the license checking function Include error reporting code

Conclusion With DynInst library: It is easy to monitor and control almost any running program. It is also easy to make arbitrary changes that program's behavior.

Conclusion More significant study is needed for Safe remote execution which includes: preventing inappropriate operations preventing undetected modification or spoiling of computational results

Question We have seen several tamper-proofing/tamper-resistance technique: Software Guards Obfuscation Software aging How well do they serve in protecting software?