Interprocedural shape analysis for cutpoint-free programs Noam Rinetzky Tel Aviv University Joint work with Mooly Sagiv Tel Aviv University Eran Yahav IBM Watson

Motivation Interprocedural shape analysis  Conservative static pointer analysis  Heap intensive programs Imperative programs with (recursive) procedures Linked data structures Challenge  Destructive update  Localized effect of procedures

y t g x Main idea Local heaps y t g call p(x); x x x x x

y t g x Main idea Local heaps Cutpoint freedom y t g call p(x); x x x x ? POPL ’05 SAS ’05

Cutpoints An object is a cutpoint for an invocation  Reachable from actual parameters  Not pointed to by an actual parameter  Reachable without going through a parameter call p(y,z) y x n n t z y n n t z n n

Cutpoint freedom Cutpoint-free  Invocation: has no cutpoints  Execution: every invocation is cutpoint-free  Program: every execution is cutpoint-free x y n n t z y n n t z x call p(y,z) n n

call p(y,z); Cutpoint freedom: benefits n n t z y x call p(y,z); n x n t y z Restricted aliasing Procedure ~ function  Input / output relation Not Cutpoint free Cutpoint free

Main results Cutpoint freedom Non-standard concrete semantics  Verifies that an execution is cutpoint-free  Local heaps Interprocedural shape analysis  Conservatively verifies program is cutpoint free Desired properties  Partial correctness of quicksort  Procedure summaries Prototype implementation

Plan Cutpoint freedom Non-standard concrete semantics Interprocedural shape analysis Prototype implementation Related work

Programming model Single threaded Procedures Value parameters Formal parameters not modified Recursion Heap Recursive data structures Destructive update  No explicit addressing (&)  No pointer arithmetic

Memory states A memory state encodes a local heap  Local variables of the current procedure invocation  Relevant part of the heap Relevant  Reachable x y n n t z p q main append

Memory states Represented by first-order logical structures PredicateMeaning x(v)x(v)Variable x points to v n(v1,v2)n(v1,v2)Field n of object v 1 points to v 2

Memory states Represented by first-order logical structures p q p u1u1 1 u2u2 0 u1u1 u2u2 q u1u1 0 u2u2 1 nu1u1 u2u2 u1u1 01 u2u2 00 n PredicateMeaning x(v)x(v)Variable x points to v n(v1,v2)n(v1,v2)Field n of object v 1 points to v 2

Operational semantics Statements modify values of predicates Specified by predicate-update formulae  Formulae in FO-TC

Large step semantics Procedure ~ input/output relation Procedure call rule

main() { append(y,z); } Large step semantics Procedure ~ input/output relation Procedure call rule append(List p, List q) { … } n x n t y n z n p q p q n y z p q n x n t 1. Verify cutpoint freedom 2 Compute input … Execute callee … 3 Combine output

Procedure call: 1. Verifying cutpoint-freedom y x n t z y n t z n n Not Cutpoint free n n x Cutpoint free An object is a cutpoint for an invocation  Reachable from actual parameters  Not pointed to by an actual parameter  Reachable without going through a parameter append(y,z)

Procedure call: 1. Verifying cutpoint-freedom y x n t z y n t z n n (main’s locals: x,y,z,t) Cutpoint free Not Cutpoint free Invoking append(y,z) in main  R {y,z} (v)=  v 1 :y(v 1 )  n*(v 1,v)   v 1 :z(v 1 )  n*(v 1,v)  isCP main,{y,z} (v)= R {y,z} (v)  (  y(v)  z(v 1 ))  ( x(v)  t(v)   v 1 :  R {y,z} (v 1 )  n(v 1,v)) n n x

Procedure call: 2. Computing the input local heap Retain only reachable objects Bind formal parameters y n t z n n x Call state p n q n Input state

Procedure body: append(p,q) Input state p n q n Output state p n q n n

Procedure call: 3. Combine output Call state Output state y n t z n n p n q n n x

Procedure call: 3. Combine output y n z n p n q n inUc(v) inUx(v) Auxiliary predicates n t n x y n z n n x t n Call state Output state

Observational equivalence  CPF   CPF (Cutpoint free semantics)  GSB   GSB (Standard semantics)  CPF and  GSB observationally equivalent when for every access paths AP 1, AP 2  AP 1 = AP 2  (  CPF )   AP 1 = AP 2  (  GSB )

Observational equivalence For cutpoint free programs:   CPF   CPF (Cutpoint free semantics)   GSB   GSB (Standard semantics)   CPF and  GSB observationally equivalent It holds that   st,  CPF    ’ CPF   st,  GSB    ’ GSB   ’ CPF and  ’ GSB are observationally equivalent

Introducing local heap semantics Operational semantics Abstract transformer Local heap Operational semantics ~ ’’ ’’

Plan Cutpoint freedom Non-standard concrete semantics Interprocedural shape analysis Prototype implementation Related work

Shape abstraction Abstract memory states represent unbounded concrete memory states  Conservatively  In a bounded way  Using 3-valued logical structures

3-Valued logic 1 = true 0 = false 1/2 = unknown A join semi-lattice, 0  1 = 1/2

Canonical abstraction y z x t n n n n n n y z x n n t n n n n

y z x t n n n n n n y z x n n t n n n n n

Instrumentation predicates Record derived properties Refine the abstraction  Instrumentation principle [SRW, TOPLAS’02] Reachability is crucial! PredicateMeaning rx(v)rx(v)v is reachable from variable x r obj (v 1,v 2 )v 2 is reachable from v 1 ils(v)v is heap-shared c(v)v resides on a cycle

Abstract memory states (with reachability) y r x r x,r y r x n n z rzrz rzrz x n n rtrt t rtrt n rtrt n r x,r y r x rzrz rzrz rtrt rtrt rtrt rzrz n y r x,r y n n z rzrz rzrz x r x n n rtrt t rtrt n n n rzrz

The importance of reachability: Call append(y,z) y z r x r x,r y n n r z x r x n n r t t n n y z x n n t n n n r x r x,r y r x n n rzrz rzrz x n n rtrt t rtrt n rtrt n rzrz n y z n

Abstract semantics Conservatively apply statements on abstract memory states  Same formulae as in concrete semantics  Soundness guaranteed [SRW, TOPLAS’02]

Procedure calls 1. Verify cutpoint freedom 2 Compute input … Analyze callee … 3 Combine output

Conservative verification of cutpoint-freedom y t z y n t n n n ryry ryry ryry ryry ryry rxrx rtrt rtrt rtrt rtrt rzrz rzrz n n n n n z x Cutpoint free Not Cutpoint free Invoking append(y,z) in main  R {y,z} (v)=  v 1 :y(v 1 )  n*(v 1,v)   v 1 :z(v 1 )  n*(v 1,v)  isCP main,{y,z} (v)= R {y,z} (v)  (  y(v)  z(v 1 ))  ( x(v)  t(v)   v 1 :  R {y,z} (v 1 )  n(v 1,v))

Interprocedural shape analysis

Tabulation exits y Interprocedural shape analysis call f(x) p x y x p

p y Interprocedural shape analysis call f(x) x y p p Tabulation exits Analyze f p x

Interprocedural shape analysis Procedure  input/output relation p q n n rqrq rprp rprp q p n n rprp rqrq n rprp q q rqrq rqrq q p rprp q p rprp rqrq n rprp rqrq … Input Output rprp

g h i k n n g h i k n rgrg rgrg rgrg rhrh rhrh riri rkrk rhrh rgrg riri rkrk append(h,i) y x z n n nn rxrx ryry rzrz y x z n n n rzrz rxrx ryry rxrx rxrx rxrx ryry rxrx rxrx append(y,z) y n z x y z x rxrx ryry rzrz ryry rxrx ryry Interprocedural shape analysis Reusable procedure summaries  Heap modularity q p rprp rqrq n rprp q p rprp rqrq

Plan Cutpoint freedom Non-standard concrete semantics Interprocedural shape analysis Prototype implementation Related work

Prototype implementation TVLA based analyzer Soot-based Java front-end Parametric abstraction Data structureVerified properties Singly linked listCleanness, acyclicity Sorting (of SLL)+ Sortedness Unshared binary treesCleaness, tree-ness

Iterative vs. Recursive (SLL) 585

Inline vs. Procedural abstraction // Allocates a list of // length 3 List create3(){ … } main() { List x1 = create3(); List x2 = create3(); List x3 = create3(); List x4 = create3(); … }

Call string vs. Relational vs. CPF [Rinetzky and Sagiv, CC’01] [Jeannet et al., SAS’04]

Plan Cutpoint freedom Non-standard concrete semantics Interprocedural shape analysis Prototype implementation Related work

Interprocedural shape analysis  Rinetzky and Sagiv, CC ‘01  Chong and Rugina, SAS ‘03  Jeannet et al., SAS ‘04  Hackett and Rugina, POPL ‘05  Rinetzky et al., POPL ‘05 Local Reasoning  Ishtiaq and O’Hearn, POPL ‘01  Reynolds, LICS ‘02 Encapsulation  Hogg, OOPSLA ‘91 ...

Related work SAS’ 05 Local heaps  Cutpoint: forbidden Simple call rule Automatically detects cutpoint freedom New shape analysis sorting (quicksort) Prototype POPL’ 05 Local heaps Cutpoints: allowed  Complicated call rule Cutpoints may hurt precision Justify existing analysis New shape analysis Abstract objects  Abstract cutpoints

Cutpoint freedom vs. Encapsulation(?) Restricted local heap sharing  Parameters dominate local heap  Unrestricted intraprocedural sharing Dynamic domination Technique  Abstract interpretation  Cutpoint-free programs  Hard to scale Automatic Restricted heap sharing  Owner(s) dominate heap references  Unrestricted stack sharing Static domination Technique  Type systems  Type-correct programs Scalable  User annotation

Future work False cutpoints  Liveness analysis Cutpoint profiler  Guide abstraction design y n t n n ryry ryry ryry rxrx rtrt rtrt rzrz n n z x append(y,z); x = null;

Summary Cutpoint freedom Non-standard operational semantics Interprocedural shape analysis  Partial correctness of quicksort Prototype implementation

End Interprocedural shape analysis for cutpoint-free programs Noam Rinetzky, Mooly Sagiv, and Eran Yahav SAS, 2005 (To appear) A Semantics for procedure local heaps and its abstraction Noam Rinetzky, Jörg Bauer, Thomas Reps, Mooly Sagiv, and Reinhard Wilhelm POPL, 2005

quickSort(List p, List q) tl p p hd p low tl high p hd low tl high

Quicksort List quickSort(List p, List q) { If(p==q || q == null) return p; List h = partition(p,q); List x = p.n; p.n = null; List low = quickSort(h, p); List high = quickSort(x,null); p.n = high; Return low; } ptl hd ptl 8

Quicksort List quickSort(List p, List q) { If(p==q || q == null) return p; List h = partition(p,q); List x = p.n; p.n = null; List low = quickSort(h, p); List high = quickSort(x,null); p.n = high; Return low; } ptl hd ptl 8 p hd low ptl 8 hd

Quicksort ptl hd ptl 8 p hd low ptl 8 hd 8 6 … 9 … 8 9 … Lev Ami et. al. ISSTA’00

Backup

Intraprocedural statements Specified by predicate-update formulae Formulae in FO-TC  Example: p.n=q Assert  v: p(v) n’(v 1,v 2 )  n(v 1,v 2 )  p(v 1 )  q(v 2 ) p q p n q p.n=q

POPL’ 05: Procedure call rule

SAS ’05: Procedure call rule