1. Program Analysis and Verification 0368-4479 http://www.cs.tau.ac.il/~maon/teaching/2013-2014/paav/paav1314b.html http://www.cs.tau.ac.il/~maon/teaching/2013-2014/paav/paav1314b.html Noam Rinetzky Lecture 2: Program Semantics Slides credit: Roman Manevich, Mooly Sagiv, Eran Yahav

2. Good manners Mobiles 2

3. Admin Grades – 4 Assignments (30%) 1 involves programming – 1 Lesson summary (10%) – Toar I: Final exam (60%) Must pass – Toar II: Project (60%) Scribes (this week) ?Scribes (nextweek) 3

4. Today What does semantics mean? – Why do we need it? – How is it related to analysis/verification? Operational semantics – Natural operational semantics – Structural operational semantics 4

5. Motivation: Verifying absence of bugs static OSStatus SSLVerifySignedServerKeyExchange(SSLContext *ctx, bool isRsa, SSLBuffer signedParams, uint8_t *signature, UInt16 signatureLen) { OSStatus err;... if ((err = SSLHashSHA1.update(&hashCtx, &serverRandom)) != 0) goto fail; if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0) goto fail;... fail: SSLFreeBuffer(&signedHashes); SSLFreeBuffer(&hashCtx); return err; } 5

6. What can we do about it? Monitoring Testing Static analysis Formal verification Specification Run time Design Time 6

7. What can we do about it? Monitoring Testing Static analysis Formal verification Specification Run time Design Time 7

8. Program analysis & verification y = ? x = ? if (x > 0) { y = 42; } else { y = 73; foo(); } assert (y == 42);  Is assertion true? No/?/Yes 8

9. Program analysis & verification  Can we prove this? Automatically?  Bad news: problem is generally undecidable Yes/?/No y = ? x = ? x = y * 2 if (x % 2 == 0) { y = 42; } else { y = 73; foo(); } assert (y == 42); 9

10. universe Main idea: use over-approximation Exact set of configurations/ Behaviors/stat es Over Approximation 10

11. Main idea: find (properties of) all reachable states* initial states bad states reachable states *or of something else … 11

12. Technique: find (properties of) more than all reachable states* initial states bad states reachable states *or of something else … 12

13. Program analysis & verification y = ?; x = ?; x = y * 2 if (x % 2 == 0) { y = 42; } else { y = 73; foo(); } assert (y == 42); ? 13

14. What does P do? y = ?; x = ?; x = y * 2 if (x % 2 == 0) { y = 42; } else { y = 73; foo(); } assert (y == 42); ? 14

15. What does P mean? y = ?; x = ?; x = y * 2 if (x % 2 == 0) { y = 42; } else { y = 73; foo(); } assert (y == 42); … syntax semantics 15

16. “Standard” semantics y = ?; x = y * 2 if (x % 2 == 0) { y = 42; } else { y = 73; foo(); } assert (y == 42); …-1,0,1, … yx 16

17. “Standard” semantics (“state transformer”) y = ?; x = y * 2 if (x % 2 == 0) { y = 42; } else { y = 73; foo(); } assert (y == 42); …-1,0,1, … yx 17

18. “Standard” semantics (“state transformer”) y = ?; y=3, x=9 x = y * 2 if (x % 2 == 0) { y = 42; } else { y = 73; foo(); } assert (y == 42); …-1,0,1, … yx 18

19. “Standard” semantics (“state transformer”) y = ?; y=3, x=9 x = y * 2y=3, x=6 if (x % 2 == 0) {y=3, x=6 y = 42;y=42, x=6 } else { y = 73;… foo();… } assert (y == 42); y=42, x=6 …-1,0,1, … yx 19

20. “State transformer” semantics initial states bad states reachable states y=3,x=9 y=3,x=6 20

21. “State transformer” semantics initial states bad states reachable states y=4,x=1 y=4,x=8 21

22. “State transformer” semantics initial states bad states reachable states y=4…,x=… 22

23. initial states bad states reachable states y=3,x=9 y=3,x=6 y=4,x=1 y=4,x=8 y=3,x=6 y=4,x=8 “State transformer” semantics Main idea: find (properties of) all reachable states* y=4…,x=… 23

24. “Standard” (collecting) semantics (“sets-of states-transformer”) y = ?; x = ?;{(y,x) | y,x ∈ Nat} x = y * 2 if (x % 2 == 0) { y = 42; } else { y = 73; foo(); } assert (y == 42); 24

25. “Standard” (collecting) semantics (“sets-of states-transformer”) y = ?; {(y=3, x=9),(y=4,x=1),(y=…, x=…)} x = y * 2 {(y=3, x=6),(y=4,x=8),(y=…, x=…)} if (x % 2 == 0) { {(y=3, x=6),(y=4,x=8),(y=…, x=…)} y = 42; {(y=42, x=6),(y=42,x=8),(y=42, x=…)} } else { y = 73; { } foo(); { } } assert (y == 42); {(y=42, x=6),(y=42,x=8),(y=42, x=…)} Yes 25

26. “Set-of-states transformer” semantics initial states bad states reachable states y=3,x=9 y=3,x=6 y=4,x=1 y=3,x=6 y=4,x=1 26

27. “Abstract-state transformer” semantics y = ?; y=T, x=T x = y * 2 if (x % 2 == 0) { y = 42; } else { y = 73; foo(); } assert (y == 42); T OE T yx T OE T (y=E,x=E)={(0,0), (0,2), (-4,10),…} 27

28. “Abstract-state transformer” semantics y = ?; y=T, x=T x = y * 2y=T, x=E if (x % 2 == 0) {y=T, x=E y = 42;y=T, x=E } else { y = 73;… foo();… } assert (y == 42); y=E, x=E Yes/?/No T OE T yx T OE T (y=E,x=E)={(0,0), (0,2), (-4,10),…} 28

29. “Abstract-state transformer” semantics y = ?; y=T, x=T x = y * 2y=T, x=E if (x % 2 == 0) {y=T, x=E y = 42;y=T, x=E } else { y = 73;… foo();… } assert (y == 42); y=E, x=E Yes/?/No T OE T yx T OE T (y=E,x=E)={(0,0), (0,2), (-4,10),…} 29

30. “Abstract-state transformer” semantics y = ?; y=T, x=T x = y * 2y=T, x=E if (x % 2 == 0) {y=T, x=E y = 42;y=E, x=E } else { y = 73;… foo();… } assert (y%2 == 0) y=E, x=E ? T OE T yx T OE T (y=E,x=E)={(0,0), (0,2), (-4,10),…} 30

31. “Abstract-state transformer” semantics initial states bad states 31 reachable states

32. “Abstract-state transformer” semantics initial states bad states 32 reachable states

33. “Abstract-state transformer” semantics initial states bad states 33 reachable states

34. “Abstract-state transformer” semantics initial states bad states 34 reachable states

35. Technique: explore abstract states initial states bad states 35 reachable states

36. Sound: cover all reachable states 36 initial states bad states reachable states

37. 37 Imprecise abstraction initial states bad states 37 reachable states False alarms

38. What does P mean? y = ?; x = ?; x = y * 2 if (x % 2 == 0) { y = 42; } else { y = 73; foo(); } assert (y == 42); … syntax semantics Abs 38

39. Programming Languages Syntax “how do I write a program?” – BNF – “Parsing” Semantics “What does my program mean?” – … 39

40. Program semantics State-transformer – Set-of-states transformer – Trace transformer Predicate-transformer Functions 40

41. Program semantics State-transformer – Set-of-states transformer – Trace transformer Predicate-transformer Functions Cat-transformer 41

42. What semantics do we want? Captures the aspects of computations we care about – “adequate” Hides irrelevant details – “fully abstract” Compositional 42

43. Operational Semantics

44. Recap

45. Program semantics State-transformer – Set-of-states transformer – Trace transformer Predicate-transformer Functions Cat-transformer

46. What semantics do we want? Captures the aspects of computations we care about – “adequate” Hides irrelevant details – “fully abstract” Compositional

47. Formal semantics 47 “Formal semantics is concerned with rigorously specifying the meaning, or behavior, of programs, pieces of hardware, etc.” / page 1

48. Formal semantics 48 “This theory allows a program to be manipulated like a formula – that is to say, its properties can be calculated.” Gérard Huet & Philippe Flajolet homage to Gilles Kahn

49. Why formal semantics? Implementation-independent definition of a programming language Automatically generating interpreters (and some day maybe full fledged compilers) Verification and debugging – if you don’t know what it does, how do you know its incorrect? 49

50. Levels of abstractions and applications 50 Program Semantics Assembly-level Semantics (Small-step) Static Analysis (abstract semantics)  

51. Semantic description methods Operational semantics – Natural semantics (big step) [G. Kahn] – Structural semantics (small step) [G. Plotkin] Trace semantics Collecting semantics [Instrumented semantics] Denotational semantics [D. Scott, C. Strachy] Axiomatic semantics [C. A. R. Hoare, R. Floyd] 51

52. 52 http://www.daimi.au.dk/~bra8130/Wiley_book/wiley.html

53. Operational Semantics

54. A simple imperative language: While Abstract syntax: a ::= n | x | a 1 + a 2 | a 1  a 2 | a 1 – a 2 b ::= true | false | a 1 = a 2 | a 1  a 2 |  b | b 1  b 2 S ::= x := a | skip | S 1 ; S 2 | if b then S 1 else S 2 | while b do S 54

55. Concrete Syntax vs. Abstract Syntax 55 a y := x S a z y Sa x z S ; S ; S a x z S a y x S ; S ; S a z y S z:=x; x:=y; y:=z z:=x; (x:=y; y:=z)(z:=x; x:=y); y:=z

56. Exercise: draw an AST 56 SS ; S y:=1; while  (x=1) do (y:=y*x; x:=x-1)

57. Syntactic categories n  Numnumerals x  Varprogram variables a  Aexparithmetic expressions b  Bexpboolean expressions S  Stmstatements 57

58. Semantic categories Z Integers {0, 1, -1, 2, -2, …} T Truth values { ff, tt } State Var  Z Example state:s=[ x  5, y  7, z  0] Lookup: s x = 5 Update: s[ x  6] = [ x  6, y  7, z  0] 58

59. Example state manipulations [ x  1, y  7, z  16] y = [ x  1, y  7, z  16] t = [ x  1, y  7, z  16][ x  5] = [ x  1, y  7, z  16][ x  5] x = [ x  1, y  7, z  16][ x  5] y = 59

60. Semantics of arithmetic expressions Arithmetic expressions are side-effect free Semantic function A  Aexp  : State  Z Defined by induction on the syntax tree A  n  s = n A  x  s = s x A  a 1 + a 2  s = A  a 1  s + A  a 2  s A  a 1 - a 2  s = A  a 1  s - A  a 2  s A  a 1 * a 2  s = A  a 1  s  A  a 2  s A  (a 1 )  s = A  a 1  s --- not needed A  - a  s = 0 - A  a 1  s Compositional Properties can be proved by structural induction 60

61. Arithmetic expression exercise Suppose s x = 3 Evaluate A  x+1  s 61

62. Semantics of boolean expressions Boolean expressions are side-effect free Semantic function B  Bexp  : State  T Defined by induction on the syntax tree B  true  s = tt B  false  s = ff B  a 1 = a 2  s = B  a 1  a 2  s = B  b 1  b 2  s = B   b  s = 62

63. Operational semantics Concerned with how to execute programs – How statements modify state – Define transition relation between configurations Two flavors – Natural semantics: describes how the overall results of executions are obtained So-called “big-step” semantics – Structural operational semantics: describes how the individual steps of a computations take place So-called “small-step” semantics 63

64. Natural operating semantics (NS) 64

65. Natural operating semantics (NS) aka “Large-step semantics” 65  S, s   s’ all steps

66. Natural operating semantics Developed by Gilles Kahn [STACS 1987]STACS 1987 Configurations  S, s  Statement S is about to execute on state s sTerminal (final) state Transitions  S, s   s’Execution of S from s will terminate with the result state s’ – Ignores non-terminating computations 66

67. Natural operating semantics  defined by rules of the form The meaning of compound statements is defined using the meaning immediate constituent statements 67  S 1, s 1   s 1 ’, …,  S n, s n   s n ’  S, s   s’ if… premise conclusion side condition

68. Natural semantics for While 68  x := a, s   s[x  A  a  s] [ass ns ]  skip, s   s [skip ns ]  S 1, s   s’,  S 2, s’   s’’  S 1 ; S 2, s   s’’ [comp ns ]  S 1, s   s’  if b then S 1 else S 2, s   s’ if B  b  s = tt [if tt ns ]  S 2, s   s’  if b then S 1 else S 2, s   s’ if B  b  s = ff [if ff ns ] axioms

69. Natural semantics for While 69  S, s   s’,  while b do S, s’   s’’  while b do S, s   s’’ if B  b  s = tt [while tt ns ]  while b do S, s   s if B  b  s = ff [while ff ns ] Non-compositional

70. Example Let s 0 be the state which assigns zero to all program variables 70  x:=x+1, s 0    skip, s 0   s 0 s 0 [x  1]  skip, s 0   s 0,  x:=x+1, s 0   s 0 [x  1]  skip ; x:=x+1, s 0   s 0 [x  1]  x:=x+1, s 0   s 0 [x  1]  if x=0 then x:=x+1 else skip, s 0   s 0 [x  1]

71. Derivation trees Using axioms and rules to derive a transition  S, s   s’ gives a derivation tree – Root:  S, s   s’ – Leaves: axioms – Internal nodes: conclusions of rules Immediate children: matching rule premises 71

72. Derivation tree example 1 Assumes 0 =[x  5, y  7, z  0] s 1 =[x  5, y  7, z  5] s 2 =[x  7, y  7, z  5] s 3 =[x  7, y  5, z  5] 72  ( z:=x; x:=y); y:=z, s 0   s 3  ( z:=x; x:=y), s 0   s 2  y:=z, s 2   s 3  z:=x, s 0   s 1  x:=y, s 1   s 2 [ass ns ] [comp ns ]

73. Derivation tree example 1 Assumes 0 =[x  5, y  7, z  0] s 1 =[x  5, y  7, z  5] s 2 =[x  7, y  7, z  5] s 3 =[x  7, y  5, z  5] 73  ( z:=x; x:=y); y:=z, s 0   s 3  ( z:=x; x:=y), s 0   s 2  y:=z, s 2   s 3  z:=x, s 0   s 1  x:=y, s 1   s 2 [ass ns ] [comp ns ]

74. Top-down evaluation via derivation trees Given a statement S and an input state s find an output state s’ such that  S, s   s’ Start with the root and repeatedly apply rules until the axioms are reached – Inspect different alternatives in order In While s’ and the derivation tree is unique 74

75. Top-down evaluation example Factorial program with s x = 2 Shorthand: W= while  (x=1) do (y:=y*x; x:=x-1) 75  y:=1; while  (x=1) do (y:=y*x; x:=x-1), s   s[y  2][x  1]  y:=1, s   s[y  1]  W, s[y  1]   s[y  2, x  1]  y:=y*x; x:=x-1, s[y  1]   s[y  2][x  1]  W, s[y  2][x  1]   s[y  2, x  1]  y:=y*x, s[y  1]   s[y  2]  x:=x-1, s[y  2]   s[y  2][x  1] [ass ns ] [comp ns ] [ass ns ] [comp ns ] [while ff ns ] [while tt ns ] [ass ns ]

76. Program termination Given a statement S and input s – S terminates on s if there exists a state s’ such that  S, s   s’ – S loops on s if there is no state s’ such that  S, s   s’ Given a statement S – S always terminates if for every input state s, S terminates on s – S always loops if for every input state s, S loops on s 76

77. Semantic equivalence S 1 and S 2 are semantically equivalent if for all s and s’  S 1, s   s’ if and only if  S 2, s   s’ Simple example while b do S is semantically equivalent to: if b then (S; while b do S) else skip – Read proof in pages 26-27 77

78. Properties of natural semantics Equivalence of program constructs – skip; skip is semantically equivalent to skip – ((S 1 ; S 2 ); S 3 ) is semantically equivalent to (S 1 ; (S 2 ; S 3 )) – (x:=5; y:=x*8) is semantically equivalent to (x:=5; y:=40) 78

79. Equivalence of (S 1 ; S 2 ); S 3 and S 1 ; (S 2 ; S 3 ) 79

80. Equivalence of (S 1 ; S 2 ); S 3 and S 1 ; (S 2 ; S 3 ) 80  (S 1 ; S 2 ), s   s 12,  S 3, s 12   s’  (S 1 ; S 2 ); S 3, s   s’  S 1, s   s 1,  S 2, s 1   s 12  S 1, s   s 1,  (S 2 ; S 3 ), s 12   s’  (S 1 ; S 2 ); S 3, s   s’  S 2, s 1   s 12,  S 3, s 12   s’ Assume  (S 1 ; S 2 ); S 3, s   s’ then the following unique derivation tree exists: Using the rule applications above, we can construct the following derivation tree: And vice versa.

81. Deterministic semantics for While Theorem: for all statements S and states s 1, s 2 if  S, s   s 1 and  S, s   s 2 then s 1 =s 2 The proof uses induction on the shape of derivation trees (pages 29-30) – Prove that the property holds for all simple derivation trees by showing it holds for axioms – Prove that the property holds for all composite trees: For each rule assume that the property holds for its premises (induction hypothesis) and prove it holds for the conclusion of the rule 81 single node #nodes>1

82. The semantic function S ns The meaning of a statement S is defined as a partial function from State to State S ns : Stm  (State  State) Examples: S ns  skip  s = s S ns  x:=1  s = s [x  1] S ns  while true do skip  s = undefined 82 S ns  S  s = s’ if  S, s   s’ undefined else

83. Structural operating semantics (SOS) aka “Small-step semantics” 83  S, s    S’, s’  first step

84. Structural operational semantics Developed by Gordon Plotkin Configurations:  has one of two forms:  S, s  Statement S is about to execute on state s sTerminal (final) state Transitions  S, s     =  S’, s’  Execution of S from s is not completed and remaining computation proceeds from intermediate configuration   = s’Execution of S from s has terminated and the final state is s’  S, s  is stuck if there is no  such that  S, s    84 first step

85. Structural semantics for While 85  x:=a, s   s[x  A  a  s] [ass sos ]  skip, s   s [skip sos ]  S 1, s    S 1 ’, s’   S 1 ; S 2, s    S 1 ’; S 2, s’  [comp 1 sos ]  if b then S 1 else S 2, s    S 1, s  if B  b  s = tt [if tt sos ]  if b then S 1 else S 2, s    S 2, s  if B  b  s = ff [if ff sos ]  S 1, s   s’  S 1 ; S 2, s    S 2, s’  [comp 2 sos ] When does this happen?

86. Structural semantics for While 86  while b do S, s    if b then S; while b do S ) else skip, s  [while sos ]

87. Derivation sequences A derivation sequence of a statement S starting in state s is either A finite sequence  0,  1,  2 …,  k such that 1.  0 =  S, s  2.  i   i+1 3.  k is either stuck configuration or a final state An infinite sequence  0,  1,  2, … such that 1.  0 =  S, s  2.  i   i+1 Notations: –  0  k  k  0 derives  k in k steps –  0  *   0 derives  in a finite number of steps For each step there is a corresponding derivation tree 87

88. Derivation sequence example Assume s 0 =[x  5, y  7, z  0] 88  (z:=x; x:=y); y:=z, s 0    x:=y; y:=z, s 0 [z  5]    y:=z, (s 0 [z  5])[x  7]   ((s 0 [z  5])[x  7])[y  5]  (z:=x; x:=y); y:=z, s 0    x:=y; y:=z, s 0 [z  5]   z:=x; x:=y, s 0    x:=y, s 0 [z  5]   z:=x, s 0   s 0 [z  5] Derivation tree for first step:

89. Evaluation via derivation sequences For any While statement S and state s it is always possible to find at least one derivation sequence from  S, s  – Apply axioms and rules forever or until a terminal or stuck configuration is reached Proposition: there are no stuck configurations in While 89

90. Factorial (n!) example Input state s such that s x = 3 y := 1; while  (x=1) do (y := y * x; x := x – 1) 90  y :=1 ; W, s    W, s[y  1]    if  (x =1) then ((y := y * x; x := x – 1); W else skip), s[y  1]    ((y := y * x; x := x – 1); W), s[y  1]    (x := x – 1; W), s[y  3]    W, s[y  3][x  2]    if  (x =1) then ((y := y * x; x := x – 1); W else skip), s[y  3][x  2]    ((y := y * x; x := x – 1); W), s[y  3] [x  2]    (x := x – 1; W), s[y  6] [x  2]    W, s[y  6][x  1]    if  (x =1) then ((y := y * x; x := x – 1); W else skip, s[y  6][x  1]    skip, s[y  6][x  1]   s[y  6][x  1]

91. Program termination Given a statement S and input s – S terminates on s if there exists a finite derivation sequence starting at  S, s  – S terminates successfully on s if there exists a finite derivation sequence starting at  S, s  leading to a final state – S loops on s if there exists an infinite derivation sequence starting at  S, s  91

92. Properties of structural operational semantics S 1 and S 2 are semantically equivalent if: – for all s and  which is either final or stuck,  S 1, s   *  if and only if  S 2, s   *  – for all s, there is an infinite derivation sequence starting at  S 1, s  if and only if there is an infinite derivation sequence starting at  S 2, s  Theorem: While is deterministic: – If  S, s   * s 1 and  S, s   * s 2 then s 1 =s 2 92

93. Sequential composition Lemma: If  S 1 ; S 2, s   k s’’ then there exists s’ and k=m+n such that  S 1, s   m s’ and  S 2, s’   n s’’ The proof (pages 37-38) uses induction on the length of derivation sequences – Prove that the property holds for all derivation sequences of length 0 – Prove that the property holds for all other derivation sequences: Show that the property holds for sequences of length k+1 using the fact it holds on all sequences of length k (induction hypothesis) 93

94. The semantic function S sos The meaning of a statement S is defined as a partial function from State to State S sos : Stm  (State  State) Examples: S sos  skip  s = s S sos  x:=1  s = s [x  1] S sos  while true do skip  s = undefined 94 S sos  S  s = s’ if  S, s   * s’ undefined else

95. An equivalence result For every statement in While S ns  S  = S sos  S  Proof in pages 40-43 95

96. The End