1. Program Analysis and Verification 0368-4479
Noam Rinetzky
Lecture 11: Interprocedural Shape Analysis
Slides credit: Roman Manevich, Mooly Sagiv, Eran Yahav

2. Canonical Abstraction
Top u1 u1 u2 u2 u3 u3
[Sagiv, Reps, Wilhelm, TOPLAS02]

3. Canonical Abstraction
Top u1 u1 u2 u2 u3 u3
Top

4. Canonical Abstraction ()
Merge all nodes with the same unary predicate values into a single summary node
Join predicate values  ’(u’1 ,..., u’k) =  { (u1 ,..., uk) | f(u1)=u’1 ,..., f(uk)=u’k }
Converts a state of arbitrary size into a 3-valued abstract state of bounded size
(C) =  { (c) | c  C }

5. Information Loss
Top n
Canonical abstraction
Top
Top n
Top
Top
Top n

6. Instrumentation Predicates
Record additional derived information via predicates
rx(v) = v1: x(v1)  n*(v1,v)
c(v) = v1: n(v1, v)  n*(v, v1)
Canonical abstraction
Top n
Top c
Top n c
Top

7. Embedding Theorem: Conservatively Observing Properties
Top
rTop
rTop
No Cycles
No cycles (derived)
1/2 1
v1,v2: n(v1, v2)  n*(v2, v1)
v:c(v)

8. Operational Semantics
void push (int v) { Node x = malloc(sizeof(Node)); x->d = v; x->n = Top; Top = x; }
Top n u1 u2 u3
int pop() {
if (Top == NULL) return EMPTY;
Node *s = Top->n;
int r = Top->d;
Top = s;
return r; }
s’(v) = v1: Top(v1)  n(v1,v)
s = Top->n
Top n u1 u2 u3 s

9. ? Abstract Semantics s = Topn s’(v) = v1: Top(v1)  n(v1,v) Top Top
rTop s
Top
rTop
rTop
s’(v) = v1: Top(v1)  n(v1,v)
s = Top->n

10. Best Transformer (s = Topn)
rTop
Top
rTop
Concrete Semantics
Top s
rTop
Top
rTop
s’(v) = v1: Top(v1)  n(v1,v) . .
Canonical Abstraction  ?
Top s
rTop
Abstract Semantics
Top s
rTop
Top
rTop
rTop

11. Semantic Reduction
Improve the precision of the analysis by recovering properties of the program semantics
A Galois connection (C, , , A)
An operation op:AA is a semantic reduction when
lL2 op(l)l and
(op(l)) = (l)  l op  C A

12. The Focus Operation Focus: Formula((3-Struct)  (3-Struct))
Generalizes materialization
For every formula 
Focus()(X) yields structure in which  evaluates to a definite values in all assignments
Only maximal in terms of embedding
Focus() is a semantic reduction
But Focus()(X) may be undefined for some X

13. Partial Concretization Based on Transformer (s=Topn)
rTop
Top
rTop
Abstract Semantics
Top
rTop
Top s
rTop
s’(v) = v1: Top(v1)  n(v1,v)
Canonical Abstraction
Focus (Topn)
Partial
Concretization
u: top(u) n(u, v)
Top s
rTop
Abstract Semantics
Top
rTop
Top
rTop s

14. Partial Concretization
Locally refine the abstract domain per statement
Soundness is immediate
Employed in other shape analysis algorithms [Distefano et.al., TACAS’06, Evan et.al., SAS’07, POPL’08]

15. The Coercion Principle
Another Semantic Reduction
Can be applied after Focus or after Update or both
Increase precision by exploiting some structural properties possessed by all stores (Global invariants)
Structural properties captured by constraints
Apply a constraint solver

16. Apply Constraint Solver
Top
rTop
Top
rTop x
Top
rTop x x rx
rx,ry n y x rx
rx,ry n y

17. Sources of Constraints
Properties of the operational semantics
Domain specific knowledge
Instrumentation predicates
User supplied

18. Example Constraints x(v1) x(v2)eq(v1, v2)
n(v, v1) n(v,v2)eq(v1, v2)
n(v1, v) n(v2,v)eq(v1, v2)is(v)
n*(v1, v2)t[n](v1, v2)

19. Abstract Transformers: Summary
Kleene evaluation yields sound solution
Focus is a statement-specific partial concretization
Coerce applies global constraints

20. DEMO

21. What about procedures?

22. A Semantics for Procedure Local Heaps and its Abstractions
Noam Rinetzky Tel Aviv University
Jörg Bauer Universität des Saarlandes
Thomas Reps University of Wisconsin
Mooly Sagiv Tel Aviv University
Reinhard Wilhelm Universität des Saarlandes

23. Motivation Interprocedural shape analysis Challenge
Conservative static pointer analysis
Heap intensive programs
Imperative programs with procedures
Recursive data structures
Challenge
Destructive update
Localized effect of procedures

24. Main idea
Local heaps x x x y t g x
call p(x); y g t

25. Main idea
Local heaps
Cutpoints x x x y t g x
call p(x); y g t

26. Main Results Concrete operational semantics Abstractions Large step
Functional analysis
Storeless
Shape abstractions
Local heap
Observationally equivalent to “standard” semantics
Java and “clean” C
Abstractions
Shape analysis [Sagiv, Reps, Wilhelm, TOPLAS ‘02]
May-alias [Deutsch, PLDI ‘94] …

27. Outline Motivating example Why semantics
Local heaps
Cutpoints
Why semantics
Local heap storeless semantics
Shape abstraction

28. Example … static void main() { } static List reverse(List t) { } n t nq n q
List x = reverse(p); n p x t r n t r n
List y = reverse(q);
List z = reverse(x);
return r;

29. Example static void main() { } static List reverse(List t) { }
List x = reverse(p); n t n p x n p x n t n n q
List y = reverse(q); n t r n t r n q y
List z = reverse(x);
return r;

30. Example n t n static void main() { } static List reverse(List t) { }
List x = reverse(p); n t
List y = reverse(q); n p x n p x n t q y n q y n
List z = reverse(x); p n x z t r n t r n
return r;

31. Cutpoints
Separating objects
Not pointed-to by a parameter

32. Cutpoints proc(x) Separating objects Not pointed-to by a parameter
Stack sharing

33. Cutpoints proc(x) proc(x) Separating objects
Not pointed-to by a parameter
proc(x)
proc(x) n n n n n n n n n n x p x n n y
Stack sharing
Heap sharing

34. Cutpoints proc(x) proc(x) Separating objects
Not pointed-to by a parameter
Capture external sharing patterns
proc(x)
proc(x) n n n n n n n n n n x p x n n y
Stack sharing
Heap sharing

35. Example static void main() { } static List reverse(List t) { }
List x = reverse(p);
List y = reverse(q); n p x n t n n n p x q y n q y n
List z = reverse(x); p n z x r t n r t n
return r;

36. Outline Motivating example Why semantics
Local heap storeless semantics
Shape abstraction

37. Abstract Interpretation [Cousot and Cousot, POPL ’77]
Operational semantics  
Abstract transformer

38. Introducing local heap semantics
Operational semantics ~
Local heap Operational semantics ’ ’
Abstract transformer

39. Outline Motivating example Why semantics
Local heap storeless semantics
Shape abstraction

40. Programming model Single threaded Procedures Heap Value parameters
Recursion
Heap
Recursive data structures
Destructive update
No explicit addressing (&)
No pointer arithmetic

41. Simplifying assumptions
No primitive values (only references)
No globals
Formals not modified

42. Storeless semantics No addresses Memory state: y=x Alias analysis
Object: 2Access paths
Heap: 2Object
Alias analysis x n x
x.n
x.n.n
y=x x n y
x.n
y.n
x.n.n
y.n.n
x=null y n
y.n
y.n.n

43. Example p? static void main() { } static List reverse(List t) {
return r; }
List x = reverse(p); t n
List y = reverse(q);
t.n.n.n
t.n.n
t.n t
t.n.n n
t.n.n.n t
t.n n n n p
x.n.n.n p
x.n.n
x.n x x
y.n.n q n y
y.n
y.n.n q n y
y.n
List z = reverse(x);
z.n n z x
z.n.n.n
z.n.n z x
r.n n r t
r.n.n.n
r.n.n
r.n n r t
r.n.n.n
r.n.n p?

44. Example static void main() { } static List reverse(List t) { return r;
List x = reverse(p); L t n
List y = reverse(q);
t.n.n.n L
t.n.n
t.n t
t.n.n n
t.n.n.n L t
t.n n n n p
x.n.n.n p
x.n.n
x.n x x
y.n.n q n y
y.n
y.n.n q n y
y.n
List z = reverse(x);
p.n
z.n n p z
x p.n.n.n
z.n.n.n
p.n.n
z.n.n p z x
p.n p
p.n.n
p.n.n.n
L.n
r.n n L r
t L.n.n.n
r.n.n.n
L.n.n
r.n.n t
L.n
r.n n L r
t L.n.n.n
r.n.n.n
L.n.n
r.n.n t

45. Cutpoint labels Relate pre-state with post-state Additional roots
Mark cutpoints at and throughout an invocation

46. Cutpoint labels L  {t.n.n.n} t L
Cutpoint label: the set of access paths that point to a cutpoint
when the invoked procedure starts
t.n.n.n L
t.n.n
t.n t L t
L  {t.n.n.n}

47. Sharing patterns L  {t.n.n.n} Cutpoint labels encode sharing patterns
w.n w w p
Stack sharing
Heap sharing
L  {t.n.n.n}

48. Observational equivalence
L  L (Local-heap Storeless Semantics)
G  G (Global-heap Store-based Semantics)
L and G observationally equivalent
when for every access paths AP1, AP2
 AP1 = AP2 (L)   AP1 = AP2 (G)

49. Main theorem: semantic equivalence
L  L (Local-heap Storeless Semantics)
G  G (Global-heap Store-based Semantics)
L and G observationally equivalent
st, L  ’L st, G  ’G
LSL
GSB
’L and ’G are observationally equivalent

50. Corollaries Preservation of invariants Detection of memory leaks
Assertions: AP1 = AP2
Detection of memory leaks

51. Applications Develop new static analyses
Shape analysis
Justify soundness of existing analyses

52. Related work Storeless semantics Jonkers, Algorithmic Languages ‘81
Deutsch, ICCL ‘92

53. Shape abstraction Shape descriptors represent unbounded memory states
Conservatively
In a bounded way
Two dimensions
Local heap (objects)
Sharing pattern (cutpoint labels)

54. A Shape abstraction L={t.n.n.n} r n n n t L r L r.n L.n r.n.n L.n.n
t, r.n.n.n
L.n.n.n t L

55. A Shape abstraction L=* r n n n t L r L r.n L.n r.n.n L.n.n t, r.n.n.n
L.n.n.n t L

56. A Shape abstraction L t L=* n L=* r n n n t L r t, r.n L.n r.n r L r.n

57. A Shape abstraction L r
t, r.n
L.n
r.n t
L=* n

58. A Shape abstraction L={t.n.n.n} r n n n t L L=* n t L r L r.n L.n
L.n.n
t, r.n.n.n
L.n.n.n t L L r
t, r.n
L.n
r.n t
L=* n

59. A Shape abstraction L1={t.n.n.n} L2={g.n.n.n} d n n n g L2 r n n n t
L2.n
d.n.n
L2.n.n
g, d.n.n.n
L2.n.n.n g L2 r n n n r L1
r.n
L1.n
r.n.n
L1.n.n
t, r.n.n.n
L1.n.n.n t L1
L=* n d n n d L
d.n
L.n
t, d.n
L.n t n L n n r L
r.n
L.n
t, r.n
L.n t r

60. Cutpoint-Freedom

61. How to tabulate procedures?
Procedure  input/output relation
Not reachable  Not effected
proc: local (reachable) heap  local heap
main() {
append(y,z); } p q
append(List p, List q) { … } p q x n t x n t n y z y n z p q n n

62. How to handle sharing? External sharing may break the functional view
main() {
append(y,z); } p q n
append(List p, List q) { … } p q n n n y t t x z y n z p q n x n

63. What’s the difference? n n n n x y append(y,z); append(y,z); t t y z x
1st Example
2nd Example
append(y,z);
append(y,z); x n t n n n y y t z x z

64. Cutpoints An object is a cutpoint for an invocation
Reachable from actual parameters
Not pointed to by an actual parameter
Reachable without going through a parameter
append(y,z)
append(y,z) n n n n y y n n t t x z z

65. Cutpoint freedom Cutpoint-free Invocation: has no cutpoints
Execution: every invocation is cutpoint-free
Program: every execution is cutpoint-free
append(y,z)
append(y,z) n n n n y x t x t y z z

66. Interprocedural shape analysis for cutpoint-free programs using 3-Valued Shape Analysis

67. Memory states: 2-Valued Logical Structure
A memory state encodes a local heap
Local variables of the current procedure invocation
Relevant part of the heap
Relevant  Reachable
main
append p q n n x t y z

68. Memory states Represented by first-order logical structures Predicate
Meaning
x(v)
Variable x points to v
n(v1,v2)
Field n of object v1 points to v2

69. Memory states Represented by first-order logical structures Predicate
Meaning
x(v)
Variable x points to v
n(v1,v2)
Field n of object v1 points to v2 p u1 1 u2 q u1 u2 1 n u1 u2 1 p q n u1 u2

70. Operational semantics
Statements modify values of predicates
Specified by predicate-update formulae
Formulae in FO-TC

71. Procedure calls append(p,q) append body append(y,z) y z x n p q
Verify cutpoint
freedom
Compute input
… Execute callee …
3 Combine output
append(y,z)
append body p n q y z x n

72. Procedure call: 1. Verifying cutpoint-freedom
An object is a cutpoint for an invocation
Reachable from actual parameters
Not pointed to by an actual parameter
Reachable without going through a parameter
append(y,z)
append(y,z) n n n n y y x n z x z n t t
Cutpoint free
Not Cutpoint free

73. Procedure call: 1. Verifying cutpoint-freedom
Invoking append(y,z) in main
R{y,z}(v)=v1:y(v1)n*(v1,v)  v1:z(v1)n*(v1,v)
isCPmain,{y,z}(v)= R{y,z}(v)  (y(v)z(v1)) 
( x(v)  t(v)  v1: R{y,z}(v1)n(v1,v))
(main’s locals: x,y,z,t) n n n n y y x n z x z n t t
Cutpoint free
Not Cutpoint free

74. Procedure call: 2. Computing the input local heap
Retain only reachable objects
Bind formal parameters
Call state p n q
Input state n n y x z n t

75. Procedure body: append(p,q)
Input state
Output state p n q n n p q

76. Procedure call: 3. Combine output
Call state
Output state n n p n q n y x z n t

77. Procedure call: 3. Combine output
Call state
Output state n n n n n y p x z q n t
Auxiliary predicates y n z x t
inUc(v)
inUx(v)

78. Observational equivalence
CPF  CPF (Cutpoint free semantics)
GSB  GSB (Standard semantics)
CPF and GSB observationally equivalent
when for every access paths AP1, AP2
 AP1 = AP2 (CPF)   AP1 = AP2 (GSB)

79. Observational equivalence
For cutpoint free programs:
CPF  CPF (Cutpoint free semantics)
GSB  GSB (Standard semantics)
CPF and GSB observationally equivalent
It holds that
st, CPF  ’CPF  st, GSB  ’GSB
’CPF and ’GSB are observationally equivalent

80. Introducing local heap semantics
Operational semantics ~
Local heap Operational semantics ’ ’
Abstract transformer

81. Shape abstraction
Abstract memory states represent unbounded concrete memory states
Conservatively
In a bounded way
Using 3-valued logical structures

82. 3-Valued logic 1 = true 0 = false 1/2 = unknown
A join semi-lattice, 0  1 = 1/2

83. Canonical abstractiony z n n n n n x n n t y z x n t

84. Instrumentation predicates
Record derived properties
Refine the abstraction
Instrumentation principle [SRW, TOPLAS’02]
Reachability is central!
Predicate
Meaning
rx(v)
v is reachable from variable x
robj(v1,v2)
v2 is reachable from v1
ils(v)
v is heap-shared
c(v)
v resides on a cycle

85. Abstract memory states (with reachability)z n n n n n x rx rx rx rx rx rx
rx,ry
rx,ry rz rz rz rz rz rz n n t rt rt rt rt rt rt y rx
rx,ry n z rz x rt t

86. The importance of reachability: Call append(y,z)x rx rx rx
rx,ry rz rz rz n n t rt rt rt y z n y z x n t n n n n x rx rx
rx,ry rz rz n n t rt rt

87. Abstract semantics
Conservatively apply statements on abstract memory states
Same formulae as in concrete semantics
Soundness guaranteed [SRW, TOPLAS’02]

88. Procedure calls append(p,q) append body append(y,z) y z x n p q
Verify cutpoint
freedom
Compute input
… Execute callee …
3 Combine output
append(y,z)
append body p n q y z x n

89. Conservative verification of cutpoint-freedom
Invoking append(y,z) in main
R{y,z}(v)=v1:y(v1)n*(v1,v)  v1:z(v1)n*(v1,v)
isCPmain,{y,z}(v)= R{y,z}(v)  (y(v)z(v1)) 
( x(v)  t(v)  v1: R{y,z}(v1)n(v1,v)) n n n n n y ry ry rz y ry ry ry rx rz n n n z n x z rt rt t rt rt t
Cutpoint free
Not Cutpoint free

90. Interprocedural shape analysis
Tabulation exits p p
call f(x) y x x y

91. Interprocedural shape analysis
Analyze f p p
Tabulation exits p p
call f(x) y x x y

92. Interprocedural shape analysis
Procedure  input/output relation
Input
Output q q rq rq p q p q n rp rq rp rq rp p n n … q p q n n n rp rp rq rp rp rp rq

93. Interprocedural shape analysis
Reusable procedure summaries
Heap modularity q p rp rq n y n z x
append(y,z) rx ry rz y x z n rx ry rz
append(y,z) g h i k n rg rh ri rk
append(h,i)

94. Plan Cutpoint freedom Non-standard concrete semantics
Interprocedural shape analysis
Prototype implementation

95. Prototype implementation
TVLA based analyzer
Soot-based Java front-end
Parametric abstraction
Data structure
Verified properties
Singly linked list
Cleanness, acyclicity
Sorting (of SLL)
+ Sortedness
Unshared binary trees
Cleaness, tree-ness

96. Iterative vs. Recursive (SLL)
585

97. Inline vs. Procedural abstraction
// Allocates a list of
// length 3
List create3(){ … }
main() {
List x1 = create3();
List x2 = create3();
List x3 = create3();
List x4 = create3();

98. Call string vs. Relational vs
Call string vs. Relational vs. CPF [Rinetzky and Sagiv, CC’01] [Jeannet et al., SAS’04]

99. Summary Cutpoint freedom Non-standard operational semantics
Interprocedural shape analysis
Partial correctness of quicksort
Prototype implementation

100. Application Properties proved Recursive  Iterative
Absence of null dereferences
Listness preservation
API conformance
Recursive  Iterative
Procedural abstraction

101. Related Work Interprocedural shape analysis Local Reasoning
Rinetzky and Sagiv, CC ’01
Chong and Rugina, SAS ’03
Jeannet et al., SAS ’04
Hackett and Rugina, POPL ’05
Rinetzky et al., POPL ‘05
Local Reasoning
Ishtiaq and O’Hearn, POPL ‘01
Reynolds, LICS ’02
Encapsulation
Noble et al. IWACO ’03
...

102. Summary Operational semantics Applications Storeless Local heap
Cutpoints
Equivalence theorem
Applications
Shape analysis
May-alias analysis