1. Comparing Semantic and Syntactic Methods in Mechanized Proof Frameworks C.J. Bell, Robert Dockins, Aquinas Hobor, Andrew W. Appel, David Walker 1

2. In the last decade, dozens of researchers have been investigating proof-carrying code (PCC) These researchers have split into two camps: – those using syntactic proof methods – those using semantic proof methods 2

3. We want to be able to investigate different proof methodologies, such as syntactic and semantic type systems The list-machine benchmark is – assembly language – operational semantics – type system specification – two implementaions of a type system This benchmark is – simple, so that it is easy to understand – modular, so that it is flexible – publically available at http://www.cs.princeton.edu/~appel/listmachine/2.0 3 List-Machine Benchmark

4. Changes to the List-Machine Benchmark for 2.0 Implemented only in Coq Added a semantic type system Reorganized the framework 4

5. Outline Introduction Organization of the List-Machine framework Extend the List Machine with fault tolerance Semantic and syntactic methods in large systems 5

6. Machine Specification 6

7. Modules 7

8. 8 Typechecking Algorithm Typechecker Soundness Proof Type System Type System Specification Typechecking Algorithm check(Π,Ψ) = true Typechecker Soundness Proof check(Π,Ψ) = true → Π  ⊢ blocks Ψ Type System Proves: Π⊢ blocks Ψ → safe Ψ Type System Specification type operators definitions of typing rules statement of safety Π⊢ blocks Ψ → safe Ψ

9. 9 Typechecking Algorithm check(Π,Ψ) = true Typechecker Soundness Proof check(Π,Ψ) = true → Π  ⊢ blocks Ψ Type System Proves: Π⊢ blocks Ψ → safe Ψ Type System Specification type operators definitions of typing rules statement of safety Π⊢ blocks Ψ → safe Ψ

10. Syntactic Type System Type operators defined inductively Typing rules defined inductively The type system is proven sound using metatheorems (progress & preservation) using induction over definitions. 10 Type System Specification Syntactic Soundness Proof Π⊢ blocks Ψ → safe Ψ

11. Semantic Type System 11 reusable Type System Specification Semantic Soundness Proof Π⊢ blocks Ψ → safe Ψ List Machine Hoare Logic Π⊢ blocks Ψ Π;Ψ⊢ block ι:P Π;Ψ⊢ instr P{ι}Q Modal Specification Logic Modal Model Library

12. Outline Introduction Organization of the List-Machine framework Extend the List Machine with fault tolerance Semantic and syntactic methods in large systems 12

13. Fault Tolerance Extend the List-Machine framework to provide fault tolerance – Requires non-trivial modifications to the framework – Demonstrates the flexibility of the framework 13

14. Simple List-Machine Example (without faults) 14

15. Fault Model Single Event Upset – assume a fault will occur at most once A fault may change just one register’s value to any other value.

16. Simple List-Machine Example (with faults) 16

17. Fault-Tolerant Modified Machine Specification 17

18. Fault-Tolerant Example 18

19. Incorrect Fault-Tolerant Example 19

20. Is the modified code fault-tolerant? Fault tolerance becomes part of the safety property Type system ensures proper use of colors Model possible occurrences of faults 20

21. Modify the Operational Semantics 21

22. Modify the Operational Semantics Branch instructions require green and blue computations to agree 22

23. FT Summary 23 Semantic Syntactic Machine syntax Operational semantics Typechecker Type systems Definition of “safe” to include fault states Safety (colors, no faults) Model faults Safety in the presence of faults

24. Outline Introduction Organization of the List-Machine framework Extend the List Machine with fault tolerance Semantic and syntactic methods in large systems 24

25. How Semantic and Syntactic Methods Scale Princeton Foundational Proof-Carrying Code (FPCC) Vs. Carnegie Mellon ConCert project FPCC :: SemanticConCert :: Syntactic 25

26. Common Traits Include a TAL for ML compiled to machine code Goal: guarantee a memory property for untrusted code Written in Twelf Industrial-strength TALs Large systems 26

27. Composition Trusted Computing Base T + L + M << P 27 M achine – SPARC or x86 definitions L ogic – example: definition of modular arithmatic T heorems – statement of the safety property P roof Checker – theorem checker for FPCC and a metatheorem checker for ConCert

28. Token count of TCB components 28

29. Token count of TCB components The TCBs are equivalent in size except for the Checker 29

30. Interface Safety Requires updating the policy moving the type system from P roof to T heorem – now part of the TCB Should the type system be semantic or syntactic? 30

31. Scaling Law Semantic: new definition per type constructor Syntactic: new definition per expression constructor Toy systems have few expression constructors… 31

32. Real systems have more expression constructors than type constructors.  semantic methods require fewer definitions Is the average type definition larger than the average typing rule? 32

33. In toy systems, typing rules are simple... |- stmt_prim_lbladd_ADD_imm: judge_stmt (e_prim A (p_lbladd V1 (val_diff L0 Lab I2))) Prog L CCEnv AENV KL Ps Phi L' CCEnv AENV KL Ps' Phi' <- regbind A At Prog <- targetreg At Ar <- regbind_val Prog V1 Vt <- realreg Vt Vr <- diff_value Prog (val_diff L0 Lab I2) Vc <- imm13 Vc (c Vimm13) <- valueTy Prog KL Phi V1 (offset I1 (int pi= (addr Lab))) <- valueTy Prog KL Phi (val_diff L0 Lab I2) (offset I2 (diff L0 Lab)) <- check_lbladd_offset I1 I2 <- num_add I1 I2 I1+I2 <- venv_add\ Prog A (offset I1+I2 (int pi= (addr L0))) Phi Phi' <- decode_list L L' Ps Ps' (instr_ADD Vr (inject_imode Vimm13) Ar) =... 33

34. How does this balance in FPCC & ConCert? 34 FPCC’s semantic definitions are half the size of syntactic definitions for FPCC This will become even more pronounced according to the scaling law if the compiler wishes to generate more instructions.

35. Conclusion Introduction Organization of the List-Machine framework Extend the List Machine with fault tolerance Semantic and syntactic methods in large systems 35

36. Appendix 36

37. Modified Typing Rules 37

38. Modified Operational Semantics w = (n,ρ,a) 38 w = (n,ρ,a,ρ’,κ) ρ’ – FT register store κ – color store (and equivalent for the syntactic system)

39. 39 Modified Semantic Type System 39

40. List-Machine Benchmark 2.0 Easily extended Facilitates small scale comparisons between many proof methods (semantic and syntactic). 40

41. Compare how type systems scale between semantic and syntactic proof methods 41 Princeton’s Foundational Proof Carrying Code (FPCC) vs Carnegie Mellon’s ConCert

42. Modules 42

43. 43 Type System Specification Typechecking Algorithm Typechecker Soundness Proof Type System Typechecking Algorithm check(Π,Ψ) = true Typechecker Soundness Proof check(Π,Ψ) = true → Π  ⊢ blocks Ψ Type System Π⊢ blocks Ψ → safe Ψ Type System Specification

44. 44 Typechecking Algorithm check(Π,Ψ) = true Typechecker Soundness Proof check(Π,Ψ) = true → Π  ⊢ blocks Ψ Type System Proves: Π⊢ blocks Ψ → safe Ψ Type System Specification type operators definitions of typing rules statement of safety Π⊢ blocks Ψ → safe Ψ

45. Modules 45