1. An Overview on Static Program Analysis Mooly Sagiv

2. Subjects u What is static analysis u Usage in compilers –Where does it fit? –Prospective –Challenges u Other clients u Why is it called ``abstract interpretation''? u Undecidability u Handling Undecidability u Soundness of abstract interpretation –Definition –Violation of soundness –Abstract interpretation cannot be always homomorphic u Relation to program verification u Origins u Complementary approaches u Tentative schedule

3. Static Analysis u Automatic derivation of static properties which hold on every execution leading to a program location

4. Example Static Analysis Problem u Find variables with constant value at a given program location u Example program int p(int x){ return x *x ; void main()} { int z; if (getc()) z = p(6) + 8; else z = p(-7) -5; printf (z); } 44

5. Recursive Program int x void p(a) { read (c); if c > 0 { a = a -2; p(a); a = a + 2; } x = -2 * a + 5; print (x); } void main { p(7); print(x); }

6. Iterative Approximation z =3 while (x>0) if (x=1) y =7y =z+4 assert y==7 [x  ?, y  ?, z  ? ] [x  ?, y  ?, z  3 ] [x  1, y  ?, z  3 ] [x  1, y  7, z  3 ] [x  ?, y  7, z  3 ] [x  ?, y  ?, z  3 ]

7. List reverse(Element  head) { List rev, n; rev = NULL; while (head != NULL) { n = head  next; head  next = rev; head = n; rev = head; } return rev; } Memory Leakage potential leakage of address pointed to by head

8. Memory Leakage Element  reverse(Element  head) { Element  rev,  n; rev = NULL; while (head != NULL) { n = head  next; head  next = rev; rev = head; head = n; } return rev; }  No memory leaks

9. A Simple Example void foo(char *s ) { while ( *s != ‘ ‘ ) s++; *s = 0; } Potential buffer overrun: offset(s)  alloc(base(s))

10. A Simple Example void foo(char *s) @require string(s) { while ( *s != ‘ ‘&& *s != 0) s++; *s = 0; }  No buffer overruns

11. Example Static Analysis Problem u Find variables which are live at a given program location u Used before set on some execution paths from the current program point

12. A Simple Example /* c */ L0: a := 0 /* ac */ L1:b := a + 1 /* bc */ c := c + b /* bc */ a := b * 2 /* ac */ if c < N goto L1 /* c */ return c ab c

13. Compiler Scheme String Scanner Parser Semantic Analysis Code Generator Static analysis Transformations Tokens AST LIR source-program tokens AST IR IR +information

14. Other Example Program Analyses u Reaching definitions u Expressions that are ``available'' u Dead code u Pointer variables never point into the same location u Points in the program in which it is safe to free an object u An invocation of virtual method whose address is unique u Statements that can be executed in parallel u An access to a variable which must be in cache u Integer intervals

15. The Need for Static Analysis u Compilers –Advanced computer architectures –High level programming languages (functional, OO, dynamic) u Software Productivity Tools –Compile time debugging »Stronger type Checking for C »Array bound violations »Identify dangling pointers »Generate test cases »Generate certification proofs u Program Understanding

16. Challenges in Static Analysis u Non-trivial u Correctness u Precision u Efficiency of the analysis u Scaling

17. C Compilers u The language was designed to reduce the need for optimizations and static analysis u The programmer has control over performance (order of evaluation, storage, registers) u C compilers nowadays spend most of the compilation time in static analysis u Sometimes C compilers have to work harder!

18. Software Quality Tools u Detecting hazards (lint) –Uninitialized variables a = malloc() ; b = a; cfree (a); c = malloc (); if (b == c) printf(“unexpected equality”); u References outside array bounds u Memory leaks (occurs even in Java!)

19. Foundation of Static Analysis u Static analysis can be viewed as interpreting the program over an “abstract domain” u Execute the program over larger set of execution paths u Guarantee sound results –Every identified constant is indeed a constant –But not every constant is identified as such

20. Example Abstract Interpretation Casting Out Nines u Check soundness of arithmetic using 9 values 0, 1, 2, 3, 4, 5, 6, 7, 8 u Whenever an intermediate result exceeds 8, replace by the sum of its digits (recursively) u Report an error if the values do not match u Example query “123 * 457 + 76543 = 132654$?” –Left 123*457 + 76543= 6 * 7 + 7 =6 + 7 = 4 –Right 3 –Report an error u Soundness (10a + b) mod 9 = (a + b) mod 9 (a+b) mod 9 = (a mod 9) + (b mod 9) (a*b) mod 9 = (a mod 9) * (b mod 9)

21. Even/Odd Abstract Interpretation u Determine if an integer variable is even or odd at a given program point

22. Example Program while (x !=1) do { if (x %2) == 0 { x := x / 2; } else { x := x * 3 + 1; assert (x %2 ==0); } } {0, 1, 2, … } {0, 2, 3, … } {0, 2, 4, …} {3, 5, 7, …} {0, 1, 2, … } {10, 16, 22, …} {1 }

23. Example Program while (x !=1) do { if (x %2) == 0 { x := x / 2; } else { x := x * 3 + 1; assert (x %2 ==0); } } ? ? E O ? E O

24.   Abstract Abstract Interpretation Concrete  Sets of stores Descriptors of sets of stores 

25. Odd/Even Abstract Interpretation  {-2, 1, 5} {0,2} {2}{0}  EO ? All concrete states   {x: x  Even}

26. Odd/Even Abstract Interpretation  {-2, 1, 5} {0,2} {2}{0}  EO ?    All concrete states   {x: x  Even}

27. Odd/Even Abstract Interpretation  {-2, 1, 5} {0,2} {2}{0}  EO ?  All concrete states {x: x  Even}  

28. Example Program while (x !=1) do { if (x %2) == 0 { x := x / 2; } else { x := x * 3 + 1; assert (x %2 ==0); } } OE

29. (Best) Abstract Transformer Concrete Representation ConcretizationAbstraction Operational Semantics St Abstract Representation Abstract Semantics St

30. Concrete and Abstract Interpretation

31. Example Program [x !=1] 1 [x := x /2] 3 [exit] 5 [x %2=0] 2 [x := x * 3 +1] 4 x 1 = x’ 3  x’ 4  x' 1 = x 1 x 2 = x’ 1   x‘ 2 = x 2 x 3 = x’ 2  E x' 3 =  x 4 = x’ 2  O x‘ 4 = x 4 * # O + # O x 5 = x’ 1  O x‘ 5 = x 5 F T T F

32. Runtime vs. Static Testing RuntimeAbstract EffectivenessMissed ErrorsFalse alarms Locate rare errors CostProportional to program’s execution Proportional to program’s size

33. Abstract (Conservative) interpretation abstract representation Set of states concretization Abstract semantics statement s abstract representation abstraction Operational semantics statement s Set of states

34. Example rule of signs u Safely identify the sign of variables at every program location u Abstract representation {P, N, ?} u Abstract (conservative) semantics of * P N ?

35. Abstract (conservative) interpretation {…,,…} concretization Abstract semantics x := x*#y abstraction Operational semantics x := x*y {…, …}

36. Example rule of signs (cont) u Safely identify the sign of variables at every program location u Abstract representation {P, N, ?} u  (C) = if all elements in C are positive then return P else if all elements in C are negative then return N else return ? u  (a) = if (a==P) then return{0, 1, 2, … } else if (a==N) return {-1, -2, -3, …, } else return Z

37. Example Constant Propagation u Abstract representation set of integer values and and extra value “?” denoting variables not known to be constants u Conservative interpretation of +

38. Example Constant Propagation(Cont) u Conservative interpretation of *

39. Example Program {,, …,,, …} x = 5; {,, … } y = 7; {,, …} if (getc()) y = x + 2; {,, …} z = x +y ; { }

40. Example Program x = 5; y = 7; if (getc()) y = x + 2; z = x +y ;

41. Example Program (2) {,, …,,, …} if (getc()) x= 3 ; y = 2; {,, …} else x =2; y = 3; {,, …} z = x +y; {, }

42. Undecidability Issues u It is undecidable if a program point is reachable in some execution u Some static analysis problems are undecidable even if the program conditions are ignored u It may be undecidable to compute the best transformers u Computing the least abstract value may be undecidable

43. The Constant Propagation Example while (getc()) { if (getc()) x 1 = x 1 + 1; if (getc()) x 2 = x 2 + 1;... if (getc()) x n = x n + 1; } y = truncate (1/ (1 + p 2 (x 1, x 2,..., x n )) /* Is y=0 here? */

44. Coping with undecidabilty u Loop free programs u Simple static properties u Interactive solutions u Conservative estimations –Every enabled transformation cannot change the meaning of the code but some transformations are no enabled –Non optimal code –Every potential error is caught but some “false alarms” may be issued

45. Analogies with Numerical Analysis u Approximate the exact semantics u More precision can be obtained at greater u computational costs

46. Violation of soundness u Loop invariant code motion u Dead code elimination u Overflow ((x+y)+z) != (x + (y+z)) u Quality checking tools may decide to ignore certain kinds of errors

47. Abstract interpretation cannot be always homomorphic (rules of signs) abstraction abstraction Operational semantics x := x+y Abstract semantics x := x+#y

48. Local Soundness of Abstract Interpretation abstraction Operational semantics statement Abstract semantics statement# 

49. Optimality Criteria u Precise (with respect to a subset of the programs) u Precise under the assumption that all paths are executable (statically exact) u Relatively optimal with respect to the chosen abstract domain u Good enough

50. Relation to Program Verification u Fully automatic u Applicable to a programming language u Can be very imprecise u May yield false alarms u Requires specification and loop invariants u Program specific u Relative complete u Provide counter examples u Provide useful documentation u Can be mechanized using theorem provers Program AnalysisProgram Verification

51. Origins of Abstract Interpretation u [Naur 1965] The Gier Algol compiler “A process which combines the operators and operands of the source text in the manner in which an actual evaluation would have to do it, but which operates on descriptions of the operands, not their value” u [Reynolds 1969] Interesting analysis which includes infinite domains (context free grammars) u [Syntzoff 1972] Well foudedness of programs and termination u [Cousot and Cousot 1976,77,79,…] The foundation u [Graham and Wegman, 1975: Kamm and Ullman, Kildall 1977] Algorithmic foundations u [Tarjan 1981] Reductions to semi-ring problems u [Sharir and Pnueli 1981] Foundation of the interprocedural case u [Allen, Kennedy, Cock, Jones, Muchnick and Schwartz]

52. Complementary Approaches u Better programming language design u Type checking u Just in time and dynamic compilation u Profiling u Sophisticated hardware u Runtime tests u Concolic testing

53. Summary u Understanding concretization is essential –For dataflow analysis –For type inference/checking –For logicians u Abstract interpretation is not limited to a particular style of programming