Verification of Software Security Properties Shahyad Sharghi and Amir Masoumzadeh Sharif University of technology shahyads@yahoo.com amir.mz@gmail.com 1384/04/26

Outline Some examples about security properties Objective Preliminaries Approach Conclusion

Example-1: Safety Property Any call to chroot should be immediately followed by a call to chdir(“/”). An Example of violation: // Here the current directory is “/var/ftp” chroot(“/var/ftp/pub”); filename = read from network(); fd = open(filename, O RDONLY); Current directory remains /var/ftp. A malicious user may ask the program to open the file ../../etc/passwd successfully even though this is outside the chroot jail and the programmer probably intended to make it inaccessible.

Example-1: FSA Any call to chroot should be immediately followed by a call to chdir(“/”). FSA for the above property: chdir chroot other

Example-2: Safety Property A call to stat(f) should not be followed immediately by a call to open(f) other stat(f) open(f)

Example-3: Safety Property A privileged process should not make certain system calls that run untrusted programs without first dropping all privileges. unpriv priv seteuid(!0) seteuid(0) other noexec exec other execl()

Example-3: A Violation An interprocedural path dependent error: the path [m1d0d2d3d4m2m3] satisfies Property the path [m1d0d1m2m3] violates it.

Objective Build a program analysis tool for finding a wide range of security vulnerabilities in large programs efficiently.

Chomsky’s hierarchy All languages type 0 or recursively enumerable languages decidable languages (turing machine) type 1 or context sensitive languages type 2 or context free languages (pushdown automata) type 3 or regular languages (finite state automata)

Properties of context-free languages An alternative and equivalent definition of context-free languages employs non-deterministic push-down automata: a language is context-free if and only if it can be accepted by such an automaton. The union and concatenation of two context-free languages is context-free; the intersection need not be. The reverse of a context-free language is context-free, but the complement need not be. Every regular language is context-free because it can be described by a regular grammar. The intersection of a context-free language and a regular language is always context-free. There exist context-sensitive languages which are not context-free. To prove that a given language is not context-free, one may employ the pumping lemma for context-free languages.

Approach Program = pushdown automaton Security property = finite state automaton Finding the composite PDA (the Intersection of the FSA and PDA) Check the resulting model Advantages of the approach: It is sound It is broadly applicable to classes of vulnerabilities It is efficient and scalable

Hackers behave like water, taking the path of least resistance

Problem Determine if there exists any execution path through the program that contain a sequence of operations that violate a security property.

Modeling Program – Step1 Step1: Parse the source program to build CFG. Each edge in the CFG represents a statement in the program and each node in the CFG represents a program point. The parser is based on GCC. If the program consists of multiple source files, MOPS merges the multiple CFGs each of which is generated from one source file into a single CFG. As source programs get larger, the sizes of their CFGs increase rapidly. For example, sendmail 8.12.0 with 53k lines of code, has a CFG with 182k nodes and 197k edges.

Modeling Program – Step 2 Step 2: Reduce the CFG by eliminating the parts not relevant to the security property being analyzed. A relevant function is a function that contains at least one relevant statement, one that may trigger a state change in the security model or that is a call to a relevant function.

A sample program and associated PDS void m() { double d = drand48(); if (d < 0.66) { s(); go_right(); if (d < 0.33) m(); } else { go_up(); m(); go_down(); void s() { if (drand48() < 0.5) return; go_up(); m(); go_down(); main() { srand48(time(NULL)); s(); <p,m0> → <p,m2> <p,m2> → <p,m3> <p,m3> → <p,s0m4> <p,m4> → <p,m5> <p,m5> → <p,m1> <p,m5> → <p,m6> <p,m6> → <p,m0m1> <p,m2> → <p,m7> <p,m7> → <p,m8> <p,m8> → <p,m0m9> <p,main0> → <p,main2> <p,main2> → <p,s0main1> <p,main1> → <p,main1> <p,m9> → <p,m1> <p,m1> → <p,ε> <p,s0> → <p,s2> <p,s0> → <p,s3> <p,s3> → <p,ε> <p,s2> → <p,s4> <p,s4> → <p,m0s5> <p,s5> → <p,s1> <p,s1> → <p,ε> s: 2 3 4 7 8 5 6 9 1 main: m: else go_up else return call s call s go_up call m go_right call m go_down go_down return else call m return

Propositions and Formula Atomic proposition “up” is true of configurations <p,m7w> and <p,s2w> Atomic proposition “down” is true of configurations <p,m9w> and <p,s5w> Atomic proposition “right” is true of configuration <p,m4w> G (up → (¬down U right)) and G (down→ (¬up U right))

Intersection of PDA and FSA Computes the intersection of the security model (represented as FSA) with the program PDA by taking their parallel composition, which creates a new PDA (called the composite PDA), whose states come from the FSA and whose input symbols and stack symbols come from the PDA, by the following algorithm: Input symbols in the composite PDA are dropped because we only care about its state reachability, not about its acceptable languages. The initial configuration of the composite PDA is (s0, p0) where s0 is the initial state of the security model and p0 is the entry point of the program (usually the entry point of the function main).

Formal Framework S = set of security-relevant (SR) operations. B = all sequences of security operations that violate the security property. A feasible trace t is a sequence of SR operations executed along a path p through the program. T = set of all feasible traces. Problem: Find all t in T that also belong to B

Formal Framework – contd. B is a regular language. This implies there exists a FSA M such that B = L(M). T is a context free language. This implies that there exists a PDA P such that T = L(P). Problem: Is C = L(M) ∩ L(P) empty ? Observations: C is a CFL that is accepted by the intersection of M and P there are efficient algorithms to compute the intersection of a PDA and an FSA and to determine if the language accepted by a PDA is empty

Formal Framework – contd. L(M) ∩ L(P) empty implies that L(M) ∩ T is also empty because T is contained in L(P). Thus, the approach is sound but it can produce false positives.

Demonstration Using Example 3 S = {execl(), seteuid(0), seteuid(!0)}. The FSA M is as shown earlier. T = {[seteuid(!0), execl()], [execl()]}. L(M) ∩ T = [execl()] This indicates the presence of a security vulnerability.

Results wu-ftp 2.4 beta 11 wu-ftp 2.4 beta 12 Sendmail 8.12.0 Performance 110 sec in Parsing and 95 sec in Model checking

Conclusion since it is fully interprocedural, it is especially useful in finding interprocedural bugs, which are more likely to elude manual audit since it is sound (modulo mild assumptions), it can reliably catch all bugs of the specified types thanks to our novel compaction algorithm, MOPS is efficient and scales to handle large programs. They are investigating how much data flow analysis they can incorporate into MOPS without affecting its scalability

References Hao Chen, David Wagner, “MOPS: An Infrastructure for Examining Security Properties of Software”, In Proceedings of the 9th ACM Conference on Computer and Communications Security (CCS) , pages 235--244, Washington, DC, November 2002. Dawson Engler, Madanlal Musuvathi, “Static Analysis versus Software Model Checking for Bug Finding” In Verification, Model Checking and Abstract Interpretation (VMCAI), pages 191-210, Venice, January 2004. Hao Chen, Drew Dean, David Wagner, “Model Checking One Million Lines of C Code”, In Proceedings of the 11th Annual Network and Distributed System Security Symposium (NDSS), San Diego, CA, February 2004. Hao Chen, David Wagner, Drew Dean, “Setuid Demystified”, In Proceedings of the 11th USENIX Security Symposium, pages 171--190, San Francisco, CA, August 2002. Javier Esparza , David Hansel , Peter Rossmanith , Stefan Schwoon, “Efficient Algorithms for Model Checking Pushdown Systems”, Proceedings of the 12th International Conference on Computer Aided Verification, p.232-247, July 15-19, 2000. Peter Linz “An Introduction to Formal Languages and Automata” Jones & Bartlett Publishers, 3rd edition, 2000. J. Hopcroft and J. Ullman. “Introduction to automata theory, languages, and computation”, Addison-Wesley, 1979.

Thanks for Your Attention