1 Model Checking One Million Lines of C Code Hao Chen Drew Dean (SRI International) David Wagner with David Schultz, Geoff Morrison, Ben Schwarz Jacob West, Jeremy Lin

2 Outline Overview of MOPS Engineering efforts Make MOPS work on lots of real world applications Experience Security properties and findings

3 MOPS (MOdel checking Programs for Security properties) A static analysis tool that checks source programs for temporal safety properties Main features –Pushdown model checking –Inter-procedural analysis –Control flow centric

4 Algorithm: Pushdown Model Checking  : set of security operations –e.g.  = { seteuid(!0), seteuid(0), execl() } B  *: sequences of security operations that violate the property –B is a regular language –The user describes B by an FSA T  *: sequences of security operations from feasible traces in the program –T is a context-free language –MOPS automatically builds T from the program Question: Is B  T =  ? T B

5 The MOPS Process Parser Model Checker Program Safety Property CFG FSA Program OK Error Traces (they may violate the property) FSA: Finite State Automaton CFG: Control Flow Graph

6 Integrate MOPS into the Build Process Goal: build a CFG for each target executable in the source package Low-level programs –mops_cc1: parse each foo.c into foo.cfg –mops_ld: link all *.cfg together How to use these programs to build CFGs for each package conveniently?

7 First Attempt: Edit Makefile Feasible for packages with simple Makefiles –OpenSSH Problems –Some Makefiles are automatically generated – they will be overwritten when the package is rebuilt –Too many Makefiles to edit – some packages generate one Makefile in each sub-directory –Some Makefiles are very complicated

8 Second Attempt: Interpose on the Build Process to Generate CFGs Set the env variable GCC_EXEC_PREFIX to let gcc use –mops_cc1 for cc1 –mops_ld for ld Therefore –Each source file foo.c is parsed into a CFG file foo.o –Each executable file a.out contains a linked CFG –Note that machine code is not generated

9 Second Attempt: Advantages and Problems Advantage: works even if the build process –Moves files into other directories (mv) –Renames files (mv) –Creates libraries (ar) Problem –Sometimes machine code is needed By autoconf to test the functionality of the compiler By the build process to generate C header files –But we let gcc create CFGs instead of machine code

10 Third Attempt Build Both Machine Code and CFGs Solution –Build both machine code and CFGs. For each foo.c, build both foo.o and foo.cfg Problem –Machine code and its corresponding CFG may be separated if the build process Moves and renames files (mv) Creates and uses libraries (ar) –We need to “bind” machine code with its CFG

11 Final Attempt Place both machine code and its CFG in the same object file –The ELF object format allows user-defined sections –For each source file foo.c gcc’s cc1 generates machine code foo.o mops_cc1 generates CFG foo.cfg Run objcopy to place foo.cfg into foo.o

12 Now, as Easy as This mops –m property.mfsa -- gcc foo.c bar.c mops –b rpm –m property.mfsa -- rh9/*.src.rpm

13 Experiment: Checking Security Properties on Large Packages Properties –Drop privilege before making unsafe system calls –Avoid race condition in file system access –Create root-jail safely –Create temporary files safely –Avoid stderr vulnerability Programs –7 programs, total one millions LOC

14 Property: Drop Privilege before Executing Untrusted Programs Setuid-root programs need to drop root privilege before executing untrusted programs –exec...(), system(), popen() Otherwise, the untrusted program will run with the root privilege

15 Property: Avoid Race Condition in File System Access It is suspicious to make two system calls in a program path that use the same filename E.g. –access(f) followed by open(f) –stat(f) followed by create(f) Vulnerability –Symbolic links

16 Property: Create root jail securely Rule: chroot() must be followed by chdir() immediately A vulnerable program The adversary sends in../../etc/password from the network // cwd==/var/ftp chroot(“/var/ftp/pub”); filename = read_from_network(); fd = open(filename, O_RDONLY);

17 Property: Avoid Race Condition in Creating Temporary Files victim.c: filename = mktemp(template); fd = open(filename, …); But an adversary can create a file with the same name between the two statements Then, victim.c will –Open the adversary’s file, or –Fail to create the temporary file (with the O_EXCL flag)

18 Rules for Avoiding Race Condition in Creating Temporary Files Never use the unsafe functions: mktemp(), tmpnam(), tempname(), tmpfile() Never reuse the parameter f in mkstemp(f) in any other function call, such as stat(), chmod(), etc –Because the same name may refer to a different file (tmpwatch)

19 Property: Avoid stderr Vulnerability int main() { // fd 0 and 1 are open, but fd 2 is closed … f = open(“/etc/passwd”, O_RDWR); // now /etc/passwd is opened to fd 2 fprintf(stderr, “%s”, user_input); // oops, have written to /etc/passwd }

20 Rule: Avoid stderr Vulnerability Property –The first three file descriptors should not be opened by any file except /dev/null and /dev/zero

21 Property: stderr vulnerability PackageLOCRunning Time # Error Traces Real BugsTotal Sendmail221K13m32s02 Postfix97K58.2s02 OpenSSH58K3m1s37 Apache240K53.0s22 BIND280K3m13s11 At5.1K7.7s22 Cron3.7K5.0s22

22 Property: create temporary files safely PackageLOCRunning Time # Error Traces Real BugsTotal Sendmail221K43.9s00 Postfix97K28.8s00 OpenSSH58K49.0s01 Apache240K47.4s11 BIND280K67.3s11 At5.1K5.3s00 Cron3.7K4.4s00

23 Experiment: Build CFGs for All RPM Packages in Redhat 9.0 Linux Number of packages: 840 Time: 3 days 14 hours on 1.5G Pentium Preliminary results: –Successful: 514 packages –Failed: 326 packages Reasons for failure –Lack of prerequisite RPM packages (the major reason) –Bugs/limitations in MOPS’s parser –The package was not written in C

24 Summary Engineering efforts in making MOPS work on lots of real world applications Experience in checking security properties on large applications