DICOTS and StackGuard: Two current approaches to tolerating malicious code Carl Landwehr Mitretek Systems, Inc Colshire Dr. McLean, VA (703)

2 DICOTS: Data Integrity for COTS Databases Research project at Naval Research Lab Based on Integrity Cluster concept (J. McDermott) Developed and prototyped by B. Montrose and L. Day

3 What is the Threat? Malicious code embedded in COTS products may compromise the integrity of this data. Select About from help menu Ctrl-drag “e” graphic onto Earth Release Ctrl Key Ctrl-drag “e” again to push text out of the way Press “unlock” button Return “e” to Earth Type X97:L97 Press Enter then Tab Hold Ctrl-Shift while you click on the Chart Wizard Button in the toolbar Open a New Sheet Press the F5 key

4 Majority Vote Technical Approach Diverse & Redundant Databases with a Trustworthy Voter

5 Supporting Research Integrity Clusters Integrity Cluster: diverse, redundant data-storing components accessed and controlled exclusively by an Integrity Controller. The Integrity Controller replicates transactions to all of its attached data storages in a one-copy serializable fashion. Data integrity is verified by comparing replicas. Integrity Cluster Criticisms Maintaining at least three redundant replicas at one site is impractical. Remedy: DICOTS can be made to work with existing distributed databases. Performance cost (from serialization) is intuitively high. Remedy: measure the actual performance, assess, seek speed-ups

6 Technical Approach Consistency Among Replicas

7 Sybase Host Sybase DB Client Host Sybase ODBC Driver COTS ODBC Software Open Database Connectivity Technical Solution Vendor provides ODBC driver designed to run directly on DBMS Practical implementation of DICOTS requires network between client and database server

8 COTS ODBC Software Client Host DICOTS Operation Sybase Host Sybase DB Oracle Host Oracle DB Informix Host Informix DB Sybase ODBC Driver Oracle ODBC Driver Informix ODBC Driver Custom ODBC Driver ODBC Proxy Interface DICOTS implements custom ODBC driver as DLL Application generates std database calls; DLL packages data, initiates TCP/IP connection, and transfers to proxy Proxy polls incoming links round robin When transaction found, it accepts and initiates it at all three back ends Proxy waits for all backends to finish If data returned, proxy votes it and reports failure if appropriate (note format issues) Then accept another transaction COTS ODBC Software Client Host Custom ODBC Driver COTS ODBC Software Client Host Custom ODBC Driver

9 DICOTS preliminary results Feasible to construct the architecture Partial set of ODBC API implemented Some compatibility issues encountered Benchmarking in progress Initial evidence: retrieval times considerably longer than update Current implementation represents baseline, near worst case

10 StackGuard: Tolerating Stack Smashing Attacks Stack Smashing Attacks are Legion Vulnerabilities attributable to buffer overflow/stack smashing attacks in CERT advisories since 1997: –1997: 14 of 28 –1998: 7 of 12 –1999: 5 of 12 (to Oct 1) Effective countermeasure to stack smashing could eliminate many actual attacks

11 Problem: Buffer Overflow Attack Attacker feeds a big string to an input routine that does not do bounds checking String over-writes return address String injects code Function return jumps to injected code Program Text [write protected] Data (static) Stack [read/write OK] Low address High address Buffer Return Address … Other local varioable... Stack frame Normal Stack Execution point Calling stack frame

12 After a Successful Attack Return address points into buffer Buffer contains attack code Executing return invokes attack code Program Text [write protected] Data (static) Stack [read/write OK] Low address High address Attack code Return Address … Other variable... Stack frame Smashed Stack

13 Problems for the attacker Hacker must find: program that runs with system privileges (e.g., root privilege) and fails to check the length of its input Hacker needs to know: –Address of the start of the buffer within the stack, to set up return address properly –Distance between start of buffer and the return address in the stack, to write a string long enough to rewrite the return address –The actual attack code (binary instructions) to create a new process [exec(“sh”)] Hacker must be able to enter entire attack string as single string input (typically, no embedded nulls) –May have to specialize machine instruction to avoid nulls in attack code

14 Attacker tricks By starting the attack code with NOPs, the rewritten return address only need be approximate By writing the guessed return address several times, the guessed displacement from buffer start to return address need only be approximate Low address High address NOP... NOP Attack Code Return Address

15 StackGuard Solution: Detect Stack Smashing As it Happens Crispin Cowan, Calton Pu, Oregon Grad Inst. StackGuard is a compiler that emits robust bug tolerant code –Stack smashing vulnerabilities are not repaired, nor are the attacks prevented Instead, program detects attack in progress –Stack smash corrupts running function’s activation record –StackGuard code generator detects corruption before return, preventing attacker from getting control

16 StackGuard 1: Defend Programs with a Stack Integrity Check Stack smash goes through –Attack code injected –Return address altered But Stack smash also smashes the Canary –Function checks for Canary before returning –If Canary smashed, program halts instead of yielding control to the attacker Low address High address Buffer Return Address Canary Stack frame Normal Stack Calling stack frame Other local variables...

17 Canary Integrity Problem: –attacker customizes attack string so that string over-write replaces proper canary word Original solution: random canaries –Choose random canary values at exec() time Requires lookup on each function call and return –Slow –Can’t be used with shared libraries

18 Faster, Better Canary Integrity: StackGuard 1.1 New solution: “terminator” canary –canary word is Null, CR, LF, and -1 –I.e. the common termination symbols for C string functions Hard for attacker to embed a termination symbol in the attack string Improvement: static canary lookup –Faster –Can be used with shared libraries

19 What this solution requires Recompile vulnerable programs with StackGuard compiler –Changes call sequence to add canary to stack –Changes return sequence to check canary before returning If canary intact, return normally If canary not intact, abort the process Programs not recompiled by Stackguard are not protected, but not affected (no compatibility issue)

20 How good is the protection? Possible to rewrite return address randomly in some situations, but much more difficult; virtually impossible for this kind of attack Overhead for StackGuard 1.1 (fixed canary) is barely measurable (time and space)

21 References McDermott, J. and Froscher, J. “Practical Defenses Against Storage Jamming,” in Proceedings of the 20th National Information Systems Security Conference, Baltimore, MD, October 1997, pp McDermott, J. Integrity Clusters. Submitted for publication, Journal of Computer Security, 1999 Aleph One. Smashing Stacks for Fun and Profit. Phrack V. 7, issue Cowan, Crispin, Calton Pu, et. Al. StackGuard Automatic Adaptive Detection and Prevention of Buffer Overflow Attacks. Proceedings 7th USENIX Conf. Available at