1. Checking System Rules Using System-Specific Programmer Written Compiler Extensions Dawson Engler, Benjamin Chelf, Andy Chou and Seth Hallem Presented by: Erez Louidor

2. Traditional Methods for Correctness Checking Formal verification –Models are difficult and costly to construct. –Models need maintenance. –Sometimes suffer from over-simplifications. Can be used to rule out deviant behaviors.

3. Traditional Methods for Correctness Checking (cont.) Testing Dynamic - The number of execution paths grows exponentially with code size: -Thorough testing requires writing many tests -Thorough testing costs a lot of time. - Requires running the tested code Can be used to check properties that are very difficult to check by other (static) means. Real-time constraints.

4. Traditional Methods for Correctness Checking (cont.) Manual inspection (code review) –Error prone. –Impractical to perform thoroughly in large systems. Usually done only on “Critical” sub-systems. Specifying the property to check is usually easy.

5. Static Analysis with Meta Compilation Compilers can be used to check general restrictions statically: –“A function can be called only with the parameter types it was declared.” –“A function cannot change a ‘const’ object.” Programming languages are usually too general to express system-specific restrictions: –“A function that returns with an error must free the resources it acquired.” –“Check every user-supplied pointer for validity before dereferencing it.” –“A blocking function must not be called when interrupts are disabled.”

6. Static Analysis with Meta Compilation (cont.) Solution: extend the compiler, to check system-specific rules. –Use a meta-language to write compiler extension, and a meta-compiler to compile the extension.

7. Static Analysis with Meta Compilation (cont.) Compilers work with the code itself: No need to construct and maintain a model. Static: Can examine all execution paths. Does not require running the code. -Some properties are very hard/impossible to check. Scales well to large systems. – Can be used to find bugs, not to guarantee their absence. – Can produce many false warnings.

8. A Meta Compilation System Extensions are written in Metal –A high-level language based on the state-machine abstraction. Metal extensions are compiled with the metal compiler. The compiled extensions are dynamically linked into xg++, a C++ compiler built on top of g++. The system is “still under development” and is not publicly available.

9. Example 1: A Metal Interrupt Checker. sm check_interrupts { // Variables used in patterns decl { unsigned } flags; // Patterns to specify enable/disable functions. pat enable = { sti(); } | { restore_flags(flags); }; pat disable = { cli(); }; // States. The first state is the initial state. is_enabled: disable  is_disabled | enable  { err(“double enable”}; }; is_disabled: enable  is_enabled | disable  { err(“double disable”}; } $end_of_path$  { err(“exiting with interrupts enabled!”);}; }

10. Metal Overview A Metal program describes an extended state- machine by transition rules: –Each transition rule specifies: A source state. A pattern. An optional action - specified as C code. An optional destination state. –If any code matches the pattern, and the current state is the source state, metal executes the action, and updates the current state By default, this state-machine is applied down every execution path.

11. Example 1: A Metal Interrupt Checker (cont.). /* From Linux kernel 2.3.99 drivers/block/raid5.c */ static struct buffer_head* get_free_buffer(struct stripe_head* sh, int b_size) { stuct buffer_head* bh; unsigned long flags; save_flags(flags); cli(); if ((bh=sh->buffer_pool)==NULL) return NULL; sh->buffer_pool = bh->b_next; bh->b_size = b_size; restore_flags(flags); return bh; } An extended version of this checker found 82 bugs in the Linux kernel code.

12. Example 2: A static memory allocation checker sm null_checker { decl {scalar} sz; decl {const int} retv; decl {any_ptr} v1; state decl { any_ptr } v; start, v.all: {((v=(any)malloc(sz)) == 0)  true=v.null, false=v.not_null | {((v=(any)malloc(sz)) != 0)  true=v.not_null, false=v.null | {((v =(any)malloc(sz)}  v.unknown; v.unknown, v.null, v.not_null: { (v==0) }  true=v.null, false=v.not_null | { (v != 0) }  true=v.not_null, false=v.null; v.unknown, v.not_null: {return retv; }  {if (mgk_int_cst(retv) < 0) err(“Error path leak!”);};

13. Example 2: A static memory allocation checker (cont.) v.null, v.unknown: {*(any *)v }  {err(“Using pointer illegally!”); }; v.unknown, v.null, v.not_null: { free(v); }  v.freed; v.freed: { free(v) }  { err(“Dup free!”); } | { v }  { err(“Use-after-free!”); }; v.all: { v=v1 }  v.stop; }

14. Example 2: A static memory allocation checker (cont.) An extended version of this checker found 132 errors in the Linux kernel code: 61 of which were false warnings: The checker does not handle variable copies. It does not detect when a clean-up routine would free the memory. /* Drivers/isdn/pcbit:pcbit_init_dev */ … free(dev); iounmap((unsigned char*)dev->sh_mem); release_mem_region(dev->ph_mem, 4096); …

15. The Analysis Algorithm The algorithm traverses all paths in the source code flow graph. The algorithm’s state is a list of state-values: –A global state-value ‘start’ in the previous example. –A set of variable-instance state-values: Each variable instance is bound to a different program-object, e.g. there may be several v’s, each bound to a different pointer. Binding a state-value to a variable, adds an instance-state to the set. (e.g. setting v’s state-value to ‘not_null’). Binding the special state-value ‘stop’ to a variable-instance, removes that instance from the set of instance-variable states, and stops tracking that program object.

16. The Analysis Algorithm (cont.) At each node in the graph, and for each transition rule, the algorithm iterates on all the state-values in the current state, looking for an applicable transition. For each node, the algorithm keeps a list of the states in which it visited that node. Upon reaching a node with an already visited state the algorithm back-tracks. –Deals with loops (assuming the SM has a finite number of states). –Speeds up the analysis by pruning redundant paths. –Assumes the SM is deterministic.

17. Example 3: Assertions Metal extensions can also operate in a linear flow- insensitive mode. sm Assert flow_insensitive { decl {any} expr, x, y, z; decl {any_call} any_fcall; decl {any_args} args; // Find all assert calls. Then apply SM to “expr” in state “in_assert” start: { assert(expr); }  {mgk_expr_recurse(expr, in_assert); }; // Find all side-effects. A function call is considered a side-effect. in_assert: { any_fcall(args) }  {err(“function call”); } | { x = y }  { err(“assignment”); } // ‘=‘ also matches ‘+=‘, ‘-=‘, etc. | {z++}  { err(“post-increment”); } | {--z}  { err (“pre-decrement”); }; // We should also check for ++z, z--. }

18. Example 3: Assertions (cont.) The authors also wrote a flow sensitive metal static assertion checker, that tries to find false assertions in compile-time. –Hindered by the primitive dataflow analysis of xg++. –Found 5 errors in the FLASH cache-coherence code.

19. Possible Restrictions Using similar checkers it is possible to check restrictions of the following forms: –Never/Always do X Never use floating-point in the kernel. Never allocate more than xxx bytes on the stack. Always allocate as much storage as an object needs –Never/Always do X before/after Y Check user pointers before using them in the kernel. –In situation X do (not do) Y If interrupts are disabled, do not block.

20. Global Analysis So far, every checker used local analysis: –The analysis was confined to the function scope. Many system rules, are context dependent, and apply globally across function boundaries: –“A blocking function must not be called when interrupts are disabled.”

21. Global Analysis (cont.) The xg++ system described in the paper does not support global analysis well. To check the above rule in the Linux kernel, the authors used xg++ to compute a list of all possible blocking functions: –A function is possibly-blocking if it either blocks directly or starts a call-chain containing a function that blocks directly. They then applied a metal local-analysis extension to check for every function that it doesn’t call a possibly blocking function in an interrupt-disabled mode. This method found 123 bugs, and produced 8 false warnings.

22. Optimizing with Meta- Compilation Domain-specific static-analysis can be used for finding optimization opportunities: –The optimizers built into compilers are inherently general, and therefore may be too conservative for a specific system. –Some optimizations are system-specific. The authors used metal extensions to find hundreds of optimization opportunities in the FLASH machine cache coherence code.