1. Inferring and checking system rules by static analysis William R Wright

2. CS 297: Security and Programming Languages 2 Material reviewed… Checking System Rules Using System-Specific, Programmer- Written Compiler Extensions. Dawson Engler, et al. OSDI 2000. RacerX: Effective, Static Detection of Race Conditions and Deadlocks. Dawson Engler, et al. SOSP 2003. Bugs as Deviant Behavior: A General Approach to Inferring Errors in Systems Code. OSDI 2000.

3. CS 297: Security and Programming Languages 3 System Rules Systems follow their own unique set of design based “correctness” rules Such rules go beyond “no dereferencing of NULL pointers

4. CS 297: Security and Programming Languages 4 Example – System rule Temporal ordering: b must always follow a

5. CS 297: Security and Programming Languages 5 Checking of such rules These rules are often unchecked. For example, suppose I am required to issue conn.close() but forget to do so. Code compiles even though I broke a system “rule”.

6. CS 297: Security and Programming Languages 6 How to check system wide rules? Although compilers observe general language semantics, they are ignorant of the rules unique to systems. Static analysis can apply systems rules.

7. CS 297: Security and Programming Languages 7 Checking systems Rules ESC enables some checking via annotations (even automating annotations via Houdini). Vault – very manual. SLAM – labeled “promising” - next presentation!

8. CS 297: Security and Programming Languages 8 Proposed method The proposed method complements those efforts. Goal is to extract ad hoc rules from source code, requiring minimal effort. Also, to provide an extensible framework to define and check systems rules.

9. CS 297: Security and Programming Languages 9 Meta-Level compilation (MC) One may define a systems rule via metal, a “high level, state-machine” language.

10. CS 297: Security and Programming Languages 10 Details about metal A rule defined in metal is called a State Machine (SM). Once so defined, we compile the rule(s) with mcc, a metal compiler, and dynamically link the result into xgcc (based on GNU gcc).

11. CS 297: Security and Programming Languages 11 Details about metal (cont’d) When compiling the code be analyzed, xgcc outputs errors based on deviations from the metal rules. Notice that modifications to source are unnecessary. If one there is a bug fix, one can easily recompile with the compiler of choice.

12. CS 297: Security and Programming Languages 12 Sample Metal rule templates With metal one may define systems rules such as: – “Never/always do X” – “Always do X before/after Y” – “In situation X, do (not do) Y” – “In situation X, do Y rather than Z”

13. CS 297: Security and Programming Languages 13 Example – metal rule (from Deviant paper)

14. CS 297: Security and Programming Languages 14 Example: …more problem code (Extensions paper)

15. CS 297: Security and Programming Languages 15 Example (cont’d) metal rule can find the bug - 6 lines of code – (Extensions, Fig. 1, Section 3.1) – Finds deviations by looking fo Functions to look for – Disable interrupts:cli() – Re-enable interrupts:sti() or restore_flags(flags) [restores to original interrupt status when paired with save_flags(flags)]

16. CS 297: Security and Programming Languages 16 Inferring deviant behavior Suppose you were just hired to get uptime from 98% to 99.999% on an airline reservation system with 5,000,000 lines of code. You know little about the system. Those who do ask you to help with the debugging. The newspaper reports that “Software problems” caused your employer’s latest disaster.

17. CS 297: Security and Programming Languages 17 Inferring deviant behavior (cont’d) The programming team has already used their knowledge of the system and as much static analysis as they could come up with. What if you could automate an examination of the source that results in a set of Metal rules that reflect the ad hoc (undocumented) behavior that the system should follow?

18. CS 297: Security and Programming Languages 18 Inference Method Assume that action taken by code is there to accomplish something. Divide actions into inferences about programmer “beliefs”.

19. CS 297: Security and Programming Languages 19 MUST beliefs Beliefs then go in two categories: – MUST beliefs: if (p==null){ System.out.println( “The pointer is” + p + “.”); } Programmer expresses belief that p is null inside the block, then contradicts the belief.

20. CS 297: Security and Programming Languages 20 MAY beliefs – MAY beliefs: back to original example – Since we see stmt.close after stmt.executeQuery, maybe this is a system rule.

21. CS 297: Security and Programming Languages 21 Beliefs – discovering One may derive possible MAY beliefs by: – Traversing the program, observing all actions that happen in tandem. – Assuming that they MUST happen in that way. – In a second pass, applying those assumptions. – Initiate a statistical analysis of the results (errors).

22. CS 297: Security and Programming Languages 22 Statistical analysis One may quickly rule out many MAY beliefs when finding that they are rarely or infrequently followed. These are “coincidences”, not MUST beliefs.

23. CS 297: Security and Programming Languages 23 Statistical analysis - detail Sort the errors by the z statistic: Which essentially measures the degree to which a MAY belief is supported by its incidence (contradictions accounted for) in the code sample. One examines errors from the most likely to the least, stopping when the effort becomes counterproductive.

24. CS 297: Security and Programming Languages 24 Results Coverity, commercialized these methods Look at http://scan.coverity.com/http://scan.coverity.com/ “Number of defects Fixed (since 3/6/2006): 6131. Targeted gcc, Samba, Linux-2.6, Perl, PHP, OpenSSL, and apparently plenty of “private” code.