Comparing Semantic and Syntactic Methods in Mechanized Proof Frameworks C.J. Bell, Robert Dockins, Aquinas Hobor, Andrew W. Appel, David Walker 1

In the last decade, dozens of researchers have been investigating proof-carrying code (PCC) These researchers have split into two camps: – those using syntactic proof methods – those using semantic proof methods 2

We want to be able to investigate different proof methodologies, such as syntactic and semantic type systems The list-machine benchmark is – assembly language – operational semantics – type system specification – two implementaions of a type system This benchmark is – simple, so that it is easy to understand – modular, so that it is flexible – publically available at 3 List-Machine Benchmark

Changes to the List-Machine Benchmark for 2.0 Implemented only in Coq Added a semantic type system Reorganized the framework 4

Outline Introduction Organization of the List-Machine framework Extend the List Machine with fault tolerance Semantic and syntactic methods in large systems 5

Machine Specification 6

Modules 7

8 Typechecking Algorithm Typechecker Soundness Proof Type System Type System Specification Typechecking Algorithm check(Π,Ψ) = true Typechecker Soundness Proof check(Π,Ψ) = true → Π  ⊢ blocks Ψ Type System Proves: Π⊢ blocks Ψ → safe Ψ Type System Specification type operators definitions of typing rules statement of safety Π⊢ blocks Ψ → safe Ψ

9 Typechecking Algorithm check(Π,Ψ) = true Typechecker Soundness Proof check(Π,Ψ) = true → Π  ⊢ blocks Ψ Type System Proves: Π⊢ blocks Ψ → safe Ψ Type System Specification type operators definitions of typing rules statement of safety Π⊢ blocks Ψ → safe Ψ

Syntactic Type System Type operators defined inductively Typing rules defined inductively The type system is proven sound using metatheorems (progress & preservation) using induction over definitions. 10 Type System Specification Syntactic Soundness Proof Π⊢ blocks Ψ → safe Ψ

Semantic Type System 11 reusable Type System Specification Semantic Soundness Proof Π⊢ blocks Ψ → safe Ψ List Machine Hoare Logic Π⊢ blocks Ψ Π;Ψ⊢ block ι:P Π;Ψ⊢ instr P{ι}Q Modal Specification Logic Modal Model Library

Outline Introduction Organization of the List-Machine framework Extend the List Machine with fault tolerance Semantic and syntactic methods in large systems 12

Fault Tolerance Extend the List-Machine framework to provide fault tolerance – Requires non-trivial modifications to the framework – Demonstrates the flexibility of the framework 13

Simple List-Machine Example (without faults) 14

Fault Model Single Event Upset – assume a fault will occur at most once A fault may change just one register’s value to any other value.

Simple List-Machine Example (with faults) 16

Fault-Tolerant Modified Machine Specification 17

Fault-Tolerant Example 18

Incorrect Fault-Tolerant Example 19

Is the modified code fault-tolerant? Fault tolerance becomes part of the safety property Type system ensures proper use of colors Model possible occurrences of faults 20

Modify the Operational Semantics 21

Modify the Operational Semantics Branch instructions require green and blue computations to agree 22

FT Summary 23 Semantic Syntactic Machine syntax Operational semantics Typechecker Type systems Definition of “safe” to include fault states Safety (colors, no faults) Model faults Safety in the presence of faults

Outline Introduction Organization of the List-Machine framework Extend the List Machine with fault tolerance Semantic and syntactic methods in large systems 24

How Semantic and Syntactic Methods Scale Princeton Foundational Proof-Carrying Code (FPCC) Vs. Carnegie Mellon ConCert project FPCC :: SemanticConCert :: Syntactic 25

Common Traits Include a TAL for ML compiled to machine code Goal: guarantee a memory property for untrusted code Written in Twelf Industrial-strength TALs Large systems 26

Composition Trusted Computing Base T + L + M << P 27 M achine – SPARC or x86 definitions L ogic – example: definition of modular arithmatic T heorems – statement of the safety property P roof Checker – theorem checker for FPCC and a metatheorem checker for ConCert

Token count of TCB components 28

Token count of TCB components The TCBs are equivalent in size except for the Checker 29

Interface Safety Requires updating the policy moving the type system from P roof to T heorem – now part of the TCB Should the type system be semantic or syntactic? 30

Scaling Law Semantic: new definition per type constructor Syntactic: new definition per expression constructor Toy systems have few expression constructors… 31

Real systems have more expression constructors than type constructors.  semantic methods require fewer definitions Is the average type definition larger than the average typing rule? 32

In toy systems, typing rules are simple... |- stmt_prim_lbladd_ADD_imm: judge_stmt (e_prim A (p_lbladd V1 (val_diff L0 Lab I2))) Prog L CCEnv AENV KL Ps Phi L' CCEnv AENV KL Ps' Phi' <- regbind A At Prog <- targetreg At Ar <- regbind_val Prog V1 Vt <- realreg Vt Vr <- diff_value Prog (val_diff L0 Lab I2) Vc <- imm13 Vc (c Vimm13) <- valueTy Prog KL Phi V1 (offset I1 (int pi= (addr Lab))) <- valueTy Prog KL Phi (val_diff L0 Lab I2) (offset I2 (diff L0 Lab)) <- check_lbladd_offset I1 I2 <- num_add I1 I2 I1+I2 <- venv_add\ Prog A (offset I1+I2 (int pi= (addr L0))) Phi Phi' <- decode_list L L' Ps Ps' (instr_ADD Vr (inject_imode Vimm13) Ar) =... 33

How does this balance in FPCC & ConCert? 34 FPCC’s semantic definitions are half the size of syntactic definitions for FPCC This will become even more pronounced according to the scaling law if the compiler wishes to generate more instructions.

Conclusion Introduction Organization of the List-Machine framework Extend the List Machine with fault tolerance Semantic and syntactic methods in large systems 35

Appendix 36

Modified Typing Rules 37

Modified Operational Semantics w = (n,ρ,a) 38 w = (n,ρ,a,ρ’,κ) ρ’ – FT register store κ – color store (and equivalent for the syntactic system)

39 Modified Semantic Type System 39

List-Machine Benchmark 2.0 Easily extended Facilitates small scale comparisons between many proof methods (semantic and syntactic). 40

Compare how type systems scale between semantic and syntactic proof methods 41 Princeton’s Foundational Proof Carrying Code (FPCC) vs Carnegie Mellon’s ConCert

Modules 42

43 Type System Specification Typechecking Algorithm Typechecker Soundness Proof Type System Typechecking Algorithm check(Π,Ψ) = true Typechecker Soundness Proof check(Π,Ψ) = true → Π  ⊢ blocks Ψ Type System Π⊢ blocks Ψ → safe Ψ Type System Specification

44 Typechecking Algorithm check(Π,Ψ) = true Typechecker Soundness Proof check(Π,Ψ) = true → Π  ⊢ blocks Ψ Type System Proves: Π⊢ blocks Ψ → safe Ψ Type System Specification type operators definitions of typing rules statement of safety Π⊢ blocks Ψ → safe Ψ

Modules 45