1. MIT/Determina Application Communities, page 1 Approved for Public Release, Distribution Unlimited - Case 9649 Collaborative learning for security and repair in application communities MIT & Determina PM: Lee Badger DARPATech, August 2007 (Other AC performer: SRI) Self-defending COTS Software Approved for Public Release, Distribution Unlimited - Case 9649

2. MIT/Determina Application Communities, page 2 Approved for Public Release, Distribution Unlimited - Case 9649 Application Communities Many communities have largely identical installations (e.g., Windows, Office)‏ Problem: A single vulnerability can be exploited throughout the community Goal: Protection against exploitation of known or unknown software vulnerabilities in communities of COTS components Automatically (no need for pre-generated signature) Protect first Survive and maintain functionality

3. MIT/Determina Application Communities, page 3 Approved for Public Release, Distribution Unlimited - Case 9649 Collaborative learning approach Initially: all community members are susceptible to an unknown attack Perform machine learning to infer normal behavior When an attack occurs: 1.Detect attacks on any member of the community 2.Analyze additional attacks to localize 3.Generate potential fixes 4.Evaluate fixes 5.Distribute successful fixes Finally: all community members are automatically protected from the attack

4. MIT/Determina Application Communities, page 4 Approved for Public Release, Distribution Unlimited - Case 9649 The Community: A Problem and an Opportunity A community contains many vulnerable members A community also has significant advantages Increased Accuracy – A community exercises more behaviors during learning Amortized Risk – A problem in one member can lead to a solution for the rest Shared Burden – Use expensive monitoring techniques by distributing the burden across the members

5. MIT/Determina Application Communities, page 5 Approved for Public Release, Distribution Unlimited - Case 9649 Collaborative Learning Framework Provide best possible out-of-box proactive protection against vulnerabilities Protect (and take advantage of) a community Automatically repair vulnerabilities for improved continuity Use dynamically learned constraints Key Features: Proactive protection (Memory Firewall) for unknown vulnerabilities Attack detector based constraint checking focuses repair on exploited vulnerabilities Code injection and Denial of Service (crashes) vulnerabilities are protected and repaired Other detectors can be added to the framework Supports arbitrary x86 binaries Adaptive: Repairs that perform poorly are removed

6. MIT/Determina Application Communities, page 6 Approved for Public Release, Distribution Unlimited - Case 9649 … Collaborative Learning System Architecture Constraints Patch/Repair Code Generation Patches … Client Workstations (Learning)‏Client Workstations (Protected)‏ Patches Patch/Repair results Merge Constraints (Daikon)‏ MPEE Data Acquisition Client Library Application Learning (Daikon)‏ Sample Data MPEE Application Memory Firewall Live Shield Evaluation (observe execution)‏ MPEE Application Memory Firewall Live Shield Evaluation (observe execution)‏ MPEE Data Acquisition Client Library Application Learning (Daikon)‏ Sample Data Central Management System

7. MIT/Determina Application Communities, page 7 Approved for Public Release, Distribution Unlimited - Case 9649 Protection Stages Learning – Constraints are learned throughout the community Monitoring – Applications are monitored for attacks (Memory Firewall and crashes). Logging – Constraints near attack locations are analyzed. Repairs are generated for constraint violations that are correlated with attacks. Repair Evaluation – Effective repairs are distributed, ineffective repairs are discarded.

8. MIT/Determina Application Communities, page 8 Approved for Public Release, Distribution Unlimited - Case 9649 Merge Constraints Monitor Learn x < 18x < 20x < 14 x < 20

9. MIT/Determina Application Communities, page 9 Approved for Public Release, Distribution Unlimited - Case 9649 Attack Location Enable logging of constraints related to attack Monitor Learn Monitor Security Server

10. MIT/Determina Application Communities, page 10 Approved for Public Release, Distribution Unlimited - Case 9649 Security Server Constraint/Attack Information Constraint Information Monitor Learn Monitor Logging

11. MIT/Determina Application Communities, page 11 Approved for Public Release, Distribution Unlimited - Case 9649 Constraint/Repair Information Distribute possible repairs Monitor Learn Monitor Logging Monitor Eval Repairs Security Server

12. MIT/Determina Application Communities, page 12 Approved for Public Release, Distribution Unlimited - Case 9649 Monitor Learn Monitor Logging Patch Activation Information Monitor Eval Repairs Monitor Secure Security Server Distribute most successful repair(s)‏

13. MIT/Determina Application Communities, page 13 Approved for Public Release, Distribution Unlimited - Case 9649 Learning Applications are instrumented throughout the community Only a small percentage of an application is instrumented on each machine Constraints are found locally and then merged centrally Constraints are learned at the basic block level Variables from multiple basic blocks in a function can be used Loop invariants and flow dependent invariants can be found Built on Determina’s client library Low overhead No visible change to client programs

14. MIT/Determina Application Communities, page 14 Approved for Public Release, Distribution Unlimited - Case 9649 Monitoring – detect attacks/bugs Current detectors Code injection (Determina’s Memory Firewall)‏ Crashes (denial of service)‏ Address violations Divide by zero Assertion checks Low overhead, no false positives Constraint violations are not an attack! Attack locations are sent to central server Framework supports additional detectors Unusual code execution Heap consistency checker User complaints

15. MIT/Determina Application Communities, page 15 Approved for Public Release, Distribution Unlimited - Case 9649 Memory Firewall as Detector NETWORK KERNEL Make payment Change prefs Read statement Write RecordUpdate Registry Open port HIJACK Program Counter HIJACK Program Counter COMPROMISE ENTER call br jmp ENTER Monitoring is simple – Port monitoring or system call monitoring Don’t know good guy from bad guy – Only “known criminals” can be identified Even known bad guys are hard to detect – Encrypted channels Used by IDS, Application Firewalls HIJACK “Catch in the act of criminal behavior” All programs follow strict conventions – ABI (Application Binary Interface) ‏ – The Calling Convention (MS/Intel) ‏ Currently no enforcement All attacks violate some of these conventions COMPROMISE Monitoring can be done – System call monitoring Hard to distinguish between actions of a normal program vs. a compromised program – Leads to false positives Used by “System Call Interception” HIPS systems SYSTEM & APPLICATION MEMORY 3) ABI Violation 1) NO ABI Violation 2) NO ABI Violation Attack Code

16. MIT/Determina Application Communities, page 16 Approved for Public Release, Distribution Unlimited - Case 9649 Memory Firewall Security Policies Goal: Closest approximation to programmer intent Infer Control Flow Graph nodes and edges ISA requirements – x86 minimal OS requirements – page RW- ABI requirements – imports, exports, SEH Calling conventions (inter-module)‏ Compiler idiosyncrasies – C, gcc C, C++, VB, Delphi Restricted Code Origins – Prevent code injection Restricted Control Transfers -Prevent code reuse attacks For x86 User mode: RET IND CALL IND JUMP

17. MIT/Determina Application Communities, page 17 Approved for Public Release, Distribution Unlimited - Case 9649 Logging - Correlate constraints and attacks Logging is enabled for constraints related to the attack across the community Overhead is low only related constraints are enabled for logging distribute logging over the community Send results to a central server for analysis A critical constraint is one that is violated if and only if there is an attack Repairs are created for each critical constraint There may be more than one possible repair for each constraint

18. MIT/Determina Application Communities, page 18 Approved for Public Release, Distribution Unlimited - Case 9649 Attack / repair example Attack exploits the C++ implementation of a Javascript system routine The type of the Javascript argument is not checked. System routine casts to a C++ object, calls a virtual method The object has a virtual table entry that points to injected code Violated constraint is found at the method call JSRI Address is in a set of legal method addresses Possible repairs Ignore the call Call one of the known valid methods Return early No repair

19. MIT/Determina Application Communities, page 19 Approved for Public Release, Distribution Unlimited - Case 9649 Evaluate repairs Server creates patches for each possible repair for each correlated constraint Server distributes each patch to a subset of the community When a patch is activated (the constraint is violated), the community member evaluates it and sends the results to the central server Is the attack avoided? Does the program exhibit other problems? Central server analyzes results The most successful patch is distributed and other patches are abandoned

20. MIT/Determina Application Communities, page 20 Approved for Public Release, Distribution Unlimited - Case 9649 Conclusion Critical vulnerabilities are recognized Code injection Denial of service Framework can be extended to other detectors Vulnerability is closed (repaired)‏ This attack will fail in the future Overhead is low Detector overhead is low Only constraints associated with attacks are logged Effective on legacy x86 binaries