Encoded PC: Self Protection from Buffer Overflow Attacks Akhilesh Tyagi Depts: Elec. & Computer Engg; Computer Science Iowa State University

ATIAS Program (FRT#1) Prof. Gyung-Ho Lee Prof. Akhilesh Tyagi 2 Graduate students

Project Goals Protection of Program Pointers: (1) return address on AR (2) function pointers Explore the role of microarchitecture and compiler. The closest related project is StackGuard and PointGuard.

The Main Idea Memory is a public area – open to attacks. Any Program Counter (PC) bound value v encoded through an encoding function e(v). Decoded d(e(v)) = v before being placed in PC.

Return Address: A valid PC bound value goes through both encoding, e, and decoding, d, functions in a PC-Memory-PC roundtrip. A compromised value only goes through the decoding function d redirecting the attack to d(A) instead of the intended address A.

Microarchitectural Version The set of actions taken on linking jumps/branches includes: MEM[$sp] e($PC); Encoding Function: e($PC) = $PC $fp;

Decoding: Instructions to pop the return address into PC can decode: jr $ra or rts $PC $ra $fp

Encoding/Decoding Key Selection Direct protection: In order to protect an object at address A, it is not sufficient to protect the surrounding addresses: A+i and A-i for i=1,2, …, k. Dynamically variable keys: A new value for each protected object instance.

Key characteristics continued: Non-reproducibility: Attacker has access to identical software and hardware environment. Key = $fp random # Register-resident keys: Most paranoid model is to assume that all memory- resident values are susceptible.

Return Address Stack Most ILP microarchitectures include 8-16 deep stack of most recent return addresses for return address prediction. Covers over 90% of nesting depths in practice. It can also be used to supplement the encoding/decoding schemes.

Compiler-assisted Version The microarchitecture actions can also be implemented in the compiler. jal / jsr actions can go into the prologue for the procedure. jr / rts actions go into the epilogue.

Current Project Status Project started Nov 1, Gcc modified to incorporate the encoding/decoding. Linux and libraries recompiled with the PC encoding gcc. Conceptual development for function pointer protection.

Function Pointer Protection Similar encoding/decoding: *fp = address replaced by *fp = e(address) = address key. Dereferencing leads to decoding: foo = *fp replaced by foo = d(*fp) = *fp key or (*fp)() replaced by ((*fp) key)(). Microarchitecture alone does not suffice: assignment to a function pointer results in a store – compiler needs to distinguish the function pointer assignments.

Func. Pointer Protection Contd. Key selection: many choices to satisfy the stated properties. Some combination of PID, FP address, compiler generated call path signature will work. Where/when to encode and decode? Compiler can attribute declarations such as (void) (*fp)() to type a subset of pointers as function pointers.

Func. Pointer Protection Contd. Decoding instructions can be inserted at the dereferencing points for the tagged function pointers or inside the prologue of the function. Encoding at the linking/loading time or through PC-bound static analysis for limited cases.

Conclusions Microarchitecture offers efficiency, transparency, and a truly private encoding key. The approach may be extensible to larger objects than the program pointer objects.