1. Interprocedural shape analysis for cutpoint-free programs Noam Rinetzky Tel Aviv University Joint work with Mooly Sagiv Tel Aviv University Eran Yahav IBM Watson

2. Motivation Interprocedural shape analysis  Conservative static pointer analysis  Heap intensive programs Imperative programs with (recursive) procedures Linked data structures Challenge  Destructive update  Localized effect of procedures

3. y t g x Main idea Local heaps y t g call p(x); x x x x x

4. y t g x Main idea Local heaps Cutpoint freedom y t g call p(x); x x x x ? POPL ’05 SAS ’05

5. Cutpoints An object is a cutpoint for an invocation  Reachable from actual parameters  Not pointed to by an actual parameter  Reachable without going through a parameter call p(y,z) y x n n t z y n n t z n n

6. Cutpoint freedom Cutpoint-free  Invocation: has no cutpoints  Execution: every invocation is cutpoint-free  Program: every execution is cutpoint-free x y n n t z y n n t z x call p(y,z) n n

7. call p(y,z); Cutpoint freedom: benefits n n t z y x call p(y,z); n x n t y z Restricted aliasing Procedure ~ function  Input / output relation Not Cutpoint free Cutpoint free

8. Main results Cutpoint freedom Non-standard concrete semantics  Verifies that an execution is cutpoint-free  Local heaps Interprocedural shape analysis  Conservatively verifies program is cutpoint free Desired properties  Partial correctness of quicksort  Procedure summaries Prototype implementation

9. Plan Cutpoint freedom Non-standard concrete semantics Interprocedural shape analysis Prototype implementation Related work

10. Programming model Single threaded Procedures Value parameters Formal parameters not modified Recursion Heap Recursive data structures Destructive update  No explicit addressing (&)  No pointer arithmetic

11. Memory states A memory state encodes a local heap  Local variables of the current procedure invocation  Relevant part of the heap Relevant  Reachable x y n n t z p q main append

12. Memory states Represented by first-order logical structures PredicateMeaning x(v)x(v)Variable x points to v n(v1,v2)n(v1,v2)Field n of object v 1 points to v 2

13. Memory states Represented by first-order logical structures p q p u1u1 1 u2u2 0 u1u1 u2u2 q u1u1 0 u2u2 1 nu1u1 u2u2 u1u1 01 u2u2 00 n PredicateMeaning x(v)x(v)Variable x points to v n(v1,v2)n(v1,v2)Field n of object v 1 points to v 2

14. Operational semantics Statements modify values of predicates Specified by predicate-update formulae  Formulae in FO-TC

15. Large step semantics Procedure ~ input/output relation Procedure call rule

16. main() { append(y,z); } Large step semantics Procedure ~ input/output relation Procedure call rule append(List p, List q) { … } n x n t y n z n p q p q n y z p q n x n t 1. Verify cutpoint freedom 2 Compute input … Execute callee … 3 Combine output

17. Procedure call: 1. Verifying cutpoint-freedom y x n t z y n t z n n Not Cutpoint free n n x Cutpoint free An object is a cutpoint for an invocation  Reachable from actual parameters  Not pointed to by an actual parameter  Reachable without going through a parameter append(y,z)

18. Procedure call: 1. Verifying cutpoint-freedom y x n t z y n t z n n (main’s locals: x,y,z,t) Cutpoint free Not Cutpoint free Invoking append(y,z) in main  R {y,z} (v)=  v 1 :y(v 1 )  n*(v 1,v)   v 1 :z(v 1 )  n*(v 1,v)  isCP main,{y,z} (v)= R {y,z} (v)  (  y(v)  z(v 1 ))  ( x(v)  t(v)   v 1 :  R {y,z} (v 1 )  n(v 1,v)) n n x

19. Procedure call: 2. Computing the input local heap Retain only reachable objects Bind formal parameters y n t z n n x Call state p n q n Input state

20. Procedure body: append(p,q) Input state p n q n Output state p n q n n

21. Procedure call: 3. Combine output Call state Output state y n t z n n p n q n n x

22. Procedure call: 3. Combine output y n z n p n q n inUc(v) inUx(v) Auxiliary predicates n t n x y n z n n x t n Call state Output state

23. Observational equivalence  CPF   CPF (Cutpoint free semantics)  GSB   GSB (Standard semantics)  CPF and  GSB observationally equivalent when for every access paths AP 1, AP 2  AP 1 = AP 2  (  CPF )   AP 1 = AP 2  (  GSB )

24. Observational equivalence For cutpoint free programs:   CPF   CPF (Cutpoint free semantics)   GSB   GSB (Standard semantics)   CPF and  GSB observationally equivalent It holds that   st,  CPF    ’ CPF   st,  GSB    ’ GSB   ’ CPF and  ’ GSB are observationally equivalent

25. Introducing local heap semantics Operational semantics Abstract transformer Local heap Operational semantics ~ ’’ ’’

26. Plan Cutpoint freedom Non-standard concrete semantics Interprocedural shape analysis Prototype implementation Related work

27. Shape abstraction Abstract memory states represent unbounded concrete memory states  Conservatively  In a bounded way  Using 3-valued logical structures

28. 3-Valued logic 1 = true 0 = false 1/2 = unknown A join semi-lattice, 0  1 = 1/2

29. Canonical abstraction y z x t n n n n n n y z x n n t n n n n

30. y z x t n n n n n n y z x n n t n n n n n

31. Instrumentation predicates Record derived properties Refine the abstraction  Instrumentation principle [SRW, TOPLAS’02] Reachability is crucial! PredicateMeaning rx(v)rx(v)v is reachable from variable x r obj (v 1,v 2 )v 2 is reachable from v 1 ils(v)v is heap-shared c(v)v resides on a cycle

32. Abstract memory states (with reachability) y r x r x,r y r x n n z rzrz rzrz x n n rtrt t rtrt n rtrt n r x,r y r x rzrz rzrz rtrt rtrt rtrt rzrz n y r x,r y n n z rzrz rzrz x r x n n rtrt t rtrt n n n rzrz

33. The importance of reachability: Call append(y,z) y z r x r x,r y n n r z x r x n n r t t n n y z x n n t n n n r x r x,r y r x n n rzrz rzrz x n n rtrt t rtrt n rtrt n rzrz n y z n

34. Abstract semantics Conservatively apply statements on abstract memory states  Same formulae as in concrete semantics  Soundness guaranteed [SRW, TOPLAS’02]

35. Procedure calls 1. Verify cutpoint freedom 2 Compute input … Analyze callee … 3 Combine output

36. Conservative verification of cutpoint-freedom y t z y n t n n n ryry ryry ryry ryry ryry rxrx rtrt rtrt rtrt rtrt rzrz rzrz n n n n n z x Cutpoint free Not Cutpoint free Invoking append(y,z) in main  R {y,z} (v)=  v 1 :y(v 1 )  n*(v 1,v)   v 1 :z(v 1 )  n*(v 1,v)  isCP main,{y,z} (v)= R {y,z} (v)  (  y(v)  z(v 1 ))  ( x(v)  t(v)   v 1 :  R {y,z} (v 1 )  n(v 1,v))

37. Interprocedural shape analysis

38. Tabulation exits y Interprocedural shape analysis call f(x) p x y x p

39. p y Interprocedural shape analysis call f(x) x y p p Tabulation exits Analyze f p x

40. Interprocedural shape analysis Procedure  input/output relation p q n n rqrq rprp rprp q p n n rprp rqrq n rprp q q rqrq rqrq q p rprp q p rprp rqrq n rprp rqrq … Input Output rprp

41. g h i k n n g h i k n rgrg rgrg rgrg rhrh rhrh riri rkrk rhrh rgrg riri rkrk append(h,i) y x z n n nn rxrx ryry rzrz y x z n n n rzrz rxrx ryry rxrx rxrx rxrx ryry rxrx rxrx append(y,z) y n z x y z x rxrx ryry rzrz ryry rxrx ryry Interprocedural shape analysis Reusable procedure summaries  Heap modularity q p rprp rqrq n rprp q p rprp rqrq

42. Plan Cutpoint freedom Non-standard concrete semantics Interprocedural shape analysis Prototype implementation Related work

43. Prototype implementation TVLA based analyzer Soot-based Java front-end Parametric abstraction Data structureVerified properties Singly linked listCleanness, acyclicity Sorting (of SLL)+ Sortedness Unshared binary treesCleaness, tree-ness

44. Iterative vs. Recursive (SLL) 585

45. Inline vs. Procedural abstraction // Allocates a list of // length 3 List create3(){ … } main() { List x1 = create3(); List x2 = create3(); List x3 = create3(); List x4 = create3(); … }

46. Call string vs. Relational vs. CPF [Rinetzky and Sagiv, CC’01] [Jeannet et al., SAS’04]

47. Plan Cutpoint freedom Non-standard concrete semantics Interprocedural shape analysis Prototype implementation Related work

48. Interprocedural shape analysis  Rinetzky and Sagiv, CC ‘01  Chong and Rugina, SAS ‘03  Jeannet et al., SAS ‘04  Hackett and Rugina, POPL ‘05  Rinetzky et al., POPL ‘05 Local Reasoning  Ishtiaq and O’Hearn, POPL ‘01  Reynolds, LICS ‘02 Encapsulation  Hogg, OOPSLA ‘91 ...

49. Related work SAS’ 05 Local heaps  Cutpoint: forbidden Simple call rule Automatically detects cutpoint freedom New shape analysis sorting (quicksort) Prototype POPL’ 05 Local heaps Cutpoints: allowed  Complicated call rule Cutpoints may hurt precision Justify existing analysis New shape analysis Abstract objects  Abstract cutpoints

50. Cutpoint freedom vs. Encapsulation(?) Restricted local heap sharing  Parameters dominate local heap  Unrestricted intraprocedural sharing Dynamic domination Technique  Abstract interpretation  Cutpoint-free programs  Hard to scale Automatic Restricted heap sharing  Owner(s) dominate heap references  Unrestricted stack sharing Static domination Technique  Type systems  Type-correct programs Scalable  User annotation

51. Future work False cutpoints  Liveness analysis Cutpoint profiler  Guide abstraction design y n t n n ryry ryry ryry rxrx rtrt rtrt rzrz n n z x append(y,z); x = null;

52. Summary Cutpoint freedom Non-standard operational semantics Interprocedural shape analysis  Partial correctness of quicksort Prototype implementation

53. End www.cs.tau.ac.il/~maon Interprocedural shape analysis for cutpoint-free programs Noam Rinetzky, Mooly Sagiv, and Eran Yahav SAS, 2005 (To appear) A Semantics for procedure local heaps and its abstraction Noam Rinetzky, Jörg Bauer, Thomas Reps, Mooly Sagiv, and Reinhard Wilhelm POPL, 2005

55. quickSort(List p, List q) 5 7 8 6 tl p 5 2 6 8 1 3 7 4 2 1 4 3 p hd 1 5 2 3 4 p low 6 8 7 tl high p 1 6 8 7 5 2 3 4 hd low tl high

56. Quicksort List quickSort(List p, List q) { If(p==q || q == null) return p; List h = partition(p,q); List x = p.n; p.n = null; List low = quickSort(h, p); List high = quickSort(x,null); p.n = high; Return low; } ptl hd 2 1 4 3 5 7 6 ptl 8

57. Quicksort List quickSort(List p, List q) { If(p==q || q == null) return p; List h = partition(p,q); List x = p.n; p.n = null; List low = quickSort(h, p); List high = quickSort(x,null); p.n = high; Return low; } ptl hd 2 1 4 3 5 7 6 ptl 8 p hd low 2 1 4 3 5 7 6 ptl 8 hd

58. Quicksort ptl hd 2 1 4 3 5 7 6 ptl 8 p hd low 2 1 4 3 5 7 6 ptl 8 hd 8 6 … 9 … 8 9 … Lev Ami et. al. ISSTA’00

59. Backup

60. Intraprocedural statements Specified by predicate-update formulae Formulae in FO-TC  Example: p.n=q Assert  v: p(v) n’(v 1,v 2 )  n(v 1,v 2 )  p(v 1 )  q(v 2 ) p q p n q p.n=q

61. POPL’ 05: Procedure call rule

62. SAS ’05: Procedure call rule