Type-Based Verification of Assembly Language for Compiler Debugging Bor-Yuh Evan ChangAdam Chlipala George C. NeculaRobert R. Schneck University of California, Berkeley TLDI 2005 Long Beach, California January 10, 2005

2 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Problem and Motivation assembly code existingType check assembly code produced by existing compilers for a Java-like language Validate that certifying compilation aids compiler debugging –using undergraduate compilers course as a test bed Introduce undergraduates to using compiler technology for software quality

3 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Problem and Motivation Starting Point: Bytecode verification –Effective, simple, and very accessible –But does not work for assembly code Why assembly code? –Native code compilers are more complex –Also debug JITs Why existing compilers? –Force to make the verifier flexible –Better accessibility to students (i.e., require little or no annotations) –More test cases to validate “certified compilation for debugging”

4 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Debugging Compilers with Verification Run Compile Run Compiler Test Case Compiled Program Test Inputs Stressed Compilers Student Compile Verify Compiler Test Case Relaxed Compilers Student Ð^K^V^H^P^L^V^H0^L^V^ Hp^L^V^H°^L^V^Hð^L^V^ H^P^M^V^H0^M^V^Hp^M^ V^H^Ð^M^V^Hð^M^V^ Line 2147: Type error – argument -8(SP0) : unknown, which is not <= nonnull String.

5 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Challenges class P { int f; int m() { … } } class C extends P { int m() { … } } … P p = new P(); P c = new C(); c.m(); … invokevirtual P.m() branch (= r c 0) L abort r tmp := m[r c + 8] r tmp := m[r tmp + 12] r arg0 := r c r ra := & L ret jump [r tmp ] L ret : invokevirtual P.m()

6 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Challenges class P { int f; int m() { … } } class C extends P { int m() { … } } … P p = new P(); P c = new C(); c.m(); … invokevirtual P.m() branch (= r c 0) L abort r tmp := m[r c + 8] r tmp := m[r tmp + 12] r arg0 := r c r ra := & L ret jump [r tmp ] L ret : h r c : P, … i h r c : nonnull P, … i h r tmp : disp(P), … i h r tmp : meth(P,12), … i h r arg0 : P, … i h r rv : int, … i

7 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Challenges class P { int f; int m() { … } } class C extends P { int m() { … } } … P p = new P(); P c = new C(); c.m(); … invokevirtual P.m() branch (= r c 0) L abort r tmp := m[r c + 8] r tmp := m[r tmp + 12] r p r arg0 := r p r ra := & L ret jump [r tmp ] L ret : h r c : P, … i h r c : nonnull P, … i h r tmp : disp(P), … i h r tmp : meth(P,12), … i P h r arg0 : P, … i h r rv : int, … iunsound

8 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Challenges class P { int f; int m() { … } } class C extends P { int m() { … } } … P p = new P(); P c = new C(); int f = p.f; p.m(); x = f + 1; branch (= r p 0) L abort r tmp := r p + 12 r f := m[r tmp ] branch (= r p 0) L abort r tmp := m[r p + 8] r tmp := m[r tmp + 12] r arg0 := r p r ra := & L ret jump [r tmp ] L ret : r x := r f + 1 reordering and optimization

9 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Challenges branch (= r p 0) L abort r tmp := r p + 12 r f := m[r tmp ] r x := r f + 1 branch (= r p 0) L abort r tmp := m[r tmp - 4] r tmp := m[r tmp + 12] r arg0 := r p r ra := & L ret jump [r tmp ] L ret : h r tmp : int ptr, … i “funny” pointer arithmetic class P { int f; int m() { … } } class C extends P { int m() { … } } … P p = new P(); P c = new C(); int f = p.f; p.m(); x = f + 1;

10 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Outline Overview Abstract State –Types –Join algorithm Verification Procedure Educational Experience Concluding Remarks

11 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Overview To maximize accessibility (i.e., minimize annotations) –infer types at intermediate program points using abstract interpretation –willingness to have a limited amount of specialization to a compilation strategy To handle assembly code –use (simple) dependent types To be less sensitive to the compilation strategy –assign types to intermediate values lazily

12 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Abstract State Keep intermediate expressions in symbolic form Symbolic values Symbolic values abstract irrelevant expressions keeping only type information Intermediate expressions track many dependencies between registers

13 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Why Symbolic Values? Used to track dependencies between registers Value state form convenient for abstract interpretation r 1 = r 2 Æ r 1 = r r 1 =  + 4, r 2 =  + 4, r 3 =   r 1 := r 2  [r 1 !  (r 2 )]

14 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Values Define a normalization of expressions to values Values give the expressions of interest –For our case, address computation and comparisons with constants –Would vary depending on the source language or compiler of interest –Not all equal expressions normalize to the same value, so value forms must be chosen carefully Registers are equal if they map to the same values (after normalization)

15 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Types Primitive Types and Object References Types for Run-time Structures Tracks that the value is the dispatch table of object 

16 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Join h r 1 =  +4, r 2 =  +4, r 3 =  D  : disp(  ),  : nonnull exactly C i h r 1 =  +4, r 2 =  +8, r 3 =  D  : disp(  ),  : nonnull C i h r 1 =  +4, r 2 = , r 3 =  D  : disp(  ),  : nonnull C,  : >i ( ,  ) !  ( ,  ) ! 

17 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Join Joining of type state basically given by subtyping –subtyping lattice still fairly simple – dictated mostly by the class inheritance hierachy Values are (fancy) labels for equivalence classes of registers –lattice of conjunctions of equalities on registers –first normalize expressions to values for join –each distinct pair of values in the input state gives an equivalence class in the result state

18 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Outline Overview Abstract State –Types –Join algorithm Verification Procedure Educational Experience Concluding Remarks

19 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Verification Example class P { int f; int m() { … } } class C extends P { … } branch (= r p 0) L abort r tmp := r p + 12 r f := m[r tmp ] r x := r f + 1 r tmp := m[r tmp - 4] r tmp := m[r tmp + 12] r arg0 := r p r ra := & L ret jump [r tmp ] L ret : Typing Rule h r tmp =  + 12 D … i h r p =  D  : P i h … D  : nonnull P i h r f =  D  : int i h r x =  + 1 D  : int i

20 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Verification Example class P { int f; int m() { … } } class C extends P { … } branch (= r p 0) L abort r tmp := r p + 12 r f := m[r tmp ] r x := r f + 1 r tmp := m[r tmp - 4] r tmp := m[r tmp + 12] r arg0 := r p r ra := & L ret jump [r tmp ] L ret : h r tmp =  + 12 D … i h r p =  D  : P i h … D  : nonnull P i Typing Rule h r f =  D  : int i h r x =  + 1 D  : int i h r tmp =  D  : disp(  ) i h r tmp =  D  : meth( ,12) i h r arg0 =  D … i h r rv =  D  : int i Calling meth( ,12) checks r arg0 =   + 12 – 4   + 8

21 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Outline Overview Abstract State –Types –Join algorithm Verification Procedure Educational Experience Concluding Remarks

22 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Comparing Compiler Development Good compilations increase: 53% to 75% Without Coolaid With Coolaid

23 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Comparing Compiler Development Type errors decrease: 44% to 19% –even 29% to 15% if only consider those that are visible in testing Without Coolaid With Coolaid

24 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Comparing Compiler Development False alarms: 6% to 1% –price for ease of use Without Coolaid With Coolaid

25 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Student Perspective 0:“counter-productive” 6:“can’t imagine being without it” A favorite comment: “I would be totally lost without Coolaid. I learn best when I am using it hands-on …. I was able to really understand stack conventions and optimizations and to appreciate them.” Traditional testing procedure –49 test cases / inputs Mean scores: –33 (67%) in 2002 –34 (69%) in 2003 –39 (79%) in 2004

26 1/10/2005Chang et al.: Type-Based Verification of Assembly Language for Compiler Debugging Conclusion Extended data-flow based type inference of intermediate languages to assembly code –Able to retrofit existing compilers –Minimal annotations for accessibility Provided experimental confirmation that certifying compilation helps early compiler debugging Introduced undergraduates to the idea of improving software quality with program analysis