An Open Framework for Certified System Software Xinyu Feng Yale University

Why Certified Software? Ariane 5 Mars Polar Lander Mars climate orbiter

Something More Relevant … “A typical modern car contains around 20 built-in microcontrollers. Luxury models can have as many as 80. Such microcontrollers, …, are in constant communication with one another. In extreme cases, a single programming error in one of the control elements can mean life or death.” -- Fraunhofer Magazine, 2004 (2) Fig. from:

Toyota recalled its 160,000 Prius cars in Oct 2005, because of bugs in the software controlling the hybrid gas-electric engine system… Photo from edmunds.com

How to Guarantee Software Quality? Hardware We need a “certified” computing platform! Buggy? Bootloader + OS + Device DriverCertified Bootloader + OS + Device Driver Buggy? Runtime Services & LibrariesCertified Runtime Services & Libraries Buggy? Security InfrastructureCertified Security Infrastructure Uncertified legacy code becomes second-class citizen! We need firm control of the lowest-level software! [King et al. S&P'06]

Certified Software: Problem Definition Hardware –processors, memory, storage, devices, … Software –bootloader, device drivers, OS, runtime, applications, … Need a mathematical proof showing that as long as the hardware works, the software always work according to its specification specification S binary code C formal proof P

OS timer KBD scheduler spawn, yield, exit, lock, monitors, … … ctxt interrupts … threads A Mini-OS bootloader 1300-line 16bit x86 code, Bootable! But how to certify the code?

Low-level code: C/Assembly Certifying the Mini-OS Many challenges: Interrupts Certified the whole system Many different features Different abstraction levels Device drivers / IO 1300 lines of code Code loading Concurrency bootloader scheduler timer int. handler keyboard driver keyboard int. handler thread lib: spawn, exit, yield, … sync. lib: locks and monitors …

My Contributions Specialized program logics –SCAP: stack-based control abstractions –SAGL: modular concurrency verification –CMAP: dynamic thread creation –concurrency with relaxed memory model [ongoing work] An open framework for certified systems –OCAP: embedding and interoperation between different verification systems [TLDI’07] –interoperability based on semantic models [ongoing work] [PLDI’06] [ICFP’05] [ESOP’07]

Outline of This Talk The OCAP Framework SAGL: modular concurrency verification SCAP: stack-based control abstractions Embedding and linking in OCAP

Modules at Different Levels All concurrency verification assumes built-in concurrency Context switching, scheduler –Too low-level to be certified in these logics Threads & schedulers have never been certified in a single logic! Example: how to certify multi- threaded software?

L1L1 L2L2 L3L3 L4L4 Building Fully Certified Systems One logic for all code –Consider all possible interactions. –Very difficult! Reality –Only limited combinations of features are used. –It’s simpler to use a specialized logic for each combination. –Interoperability between logics

OCAP Our Solution LnLn … L1L1 Mechanized Meta-Logic (CiC) Modeling of the machine … C1C1 CnCn C1C1 C1C1 CnCn … OS CnCn … C1C1 CnCn TCB

The Machine I1I1 f1:f1: I2I2 f2:f2: I3I3 f3:f3: … (code heap) C 0 r1r1 12… r2r2 r3r3 …rnrn (data heap) H (register file) R (state) S addu … lw … sw … … j f (instr. seq.) I (program) P::=(C,S,pc) ::=(H,R) ::={f  I} * pc

Program Specifications I1I1 f1:f1: I2I2 f2:f2: I3I3 f3:f3: … (code heap) C 0 r1r1 12… r2r2 r3r3 …rnrn (data heap) H (register file) R (state) S addu … lw … sw … … j f (instr. seq.) I (program) P::=(C,S,pc) ::=(H,R) ::={f  I} * pc 11 22 33 (spec)  ::= {f   } * 

Invariant-Based Verification Initial condition: Inv  (P 0 ) P0P0 c1c1 P1P1 c2c2 P2P2 c3c3 … cncn PnPn Progress: if Inv  (P), then  P’. P  c P’. Preservation: if Inv  (P) and P  c P’, then Inv  (P’).

Mechanized Meta-Logic (CiC) OCAP Rules LnLn … “Domain specific” logics Modeling of the machine L1L1 … C1C1 CnCn may use different  How to link modules?

How to Link Modules {r 1 :  1, …, r n :  n } {P}_{Q} call f ( _ ) t a ( _ )h( _ )h a' ………… f: …………

{r 1 :  1, …, r n :  n } {P}_{Q} ( _ ) t a ( _ )h( _ )h a' How to Link Modules How to define interpretation? Encode the invariant enforced in our invariant-based proof methodology. a should be expressive enough to encode Inv.

An Open Framework A set of Hoare-logic rules as the foundational layer XCAP [Ni&Shao'06] approach a  (S) = … used and generalized in OCAP [TLDI'07] use “a” as the assertion language supports first-class code pointers, mutable references, polymorphisms, recursive types, … Indexed approach simpler model for weak-ref, partial correctness, concurrency…, than existing work on indexed model [Appel&McAllester'01] [Amed'04] [Appel et al.'07] a i (S) = … [ongoing work with Cai, Shao & Tan]

OCAP Rules The OCAP Framework [TLDI'07] LnLn … L1L1 … C1C1 CnCn ( ) L1 ( ) Ln Sound OCAP Soundness Mechanized Meta-Logic (CiC) Modeling of the machine XCAPSCAPTAL … SAGL

OCAP Outline of This Talk The OCAP Framework SAGL: modular concurrency verification SCAP: stack-based control abstractions Embedding and linking in OCAP LnLn … Mechanized Meta-Logic (CiC) … C1C1 Modeling of the machine L1L1 CnCn SAGL SCAP

Certifying Concurrent Programs How to guarantee non-interference… in a modular way? Existing work is not modular/general…

Certifying Concurrent Programs Assume-Guarantee (A-G) reasoning [Misra&Chandy’81, Jones’83] thread modularity, general  spec of A&G requires global data invariants Concurrent Separation Logic (CSL) [O’Hearn, Brookes 2004] thread modularity + local reasoning  restrictive synchronization pattern  shared resources can be accessed only inside critical regions SAGL: extend A-G with local reasoning [ESOP'07] improved modularity without loss of generality

Assume-Guarantee Reasoning Thread T and its environment –Environment: the collection of all other threads except T A: assumption about environment’s transition G: guarantee to the environment a: precondition

A-G Reasoning To certify each thread:  S, S'. a S  A S S'  a S' Non-Interference of threads: stability of precondition: G S Next c (S) transitions satisfy the guarantee:  i,j. G i  A j ( i  j )

A-G Reasoning a1a2a1a2 a 1  a 2 a1a2a1a2 G2G2 A1A1 A2A2 G1G1 a2a2 a2a2 a1a1 a1a1 Requires global invariants!

SAGL: Overview Partition of resources: shared & private p1p1 a1a2a1a2 p2p2 Threads specs: (a 1, p 1 ), (a 2, p 2 ), … Partition is conceptual: (p 1  p 2 )  (a 1  a 2 )

SAGL: Memory Access p1p1 a1a2a1a2 p2p2 Threads have exclusive access to their private resources. All threads can access shared part. Needs to guarantee non-interference. Follows assume/guarantee reasoning. Partitions can be dynamically adjusted.

p1p1 a1a2'a1a2' p2p2 p1p1 a1a2a1a2 p2p2 SAGL – Access Shared Resource A1A1 G2G2 a1a1 a1a1 A-G reasoning: a special case where p 1 and p 2 are emp.

p1p1 a1a2a1a2 p2'p2' p1p1 a1a2a1a2 p2p2 SAGL – Access Private Resource

p1p1 a 1  a 2  p 2  p1p1 a1a2a1a2 p2p2 p1p1 a1a2'a1a2' p2p2 SAGL - Redistribution A1A1 G2G2 A1A1 G2G2 lockunlock a1a1 a1a1 a1a1 a1a1

getNode(): l.acq(); … l.rel(); -{(a inv, emp)} Example: List -{(emp, emp  List(x))}; -{(emp, Node(y)  List(x))} -{(List(x), Node(y))} a inv = free( l )  List( x )   free( l )  emp x … x … y a inv

SAGL [ESOP’07] p1p1 a1a2a1a2 p2p2 Threads have exclusive access to their private resources. All threads can access shared part. Needs to guarantee non-interference. Follows assume/guarantee reasoning. Partitions can be dynamically adjusted. A-G reasoning CSL

OCAP Outline of this talk The OCAP Framework SAGL: modular concurrency verification SCAP: stack-based control abstractions Embedding and linking in OCAP LnLn … Mechanized Meta-Logic (CiC) … C1C1 Modeling of the machine L1L1 CnCn SAGLSCAP

Certifying C & Assembly Code? How to specify/verify control abstractions? Stack-based control abstractions call/return, tail call, exceptions (stack cutting/ stack unwinding), coroutines/threads context switching How to formulate the stack invariants?

Problems – call/return void f(){ void h(){ h(); return; return; } } Stacks are hidden!

f:... sw $ra, -4($fp) h: jal h ;; $ra contains ct ct: lw $ra, -4($fp) jr $ra... jr $ra fp stack ?? ct Problems – call/return void f(){ void h(){ h(); return; return; } } Does f use the right return addr.? pc ra R

void cmp1(int x,jmp_buf env){ if (x == 0) longjmp(env, 1); else return; } Problems – setjmp/longjmp int rev(int x){ if (setjmp(env) == 0){ cmp0(x, env); return 0; }else{ return 1; } void cmp0(int x,jmp_buf env){ cmp1(x, env); } jmp_buf env = …; pc f0f0 … sp env f0f0 pc env cannot outlive the stack frame of rev ! … …

Stack-Based Control Abstractions A simple system (SCAP) for modular verification of (1) compiled C code & (2) manually-written assembly code weak-continuation Tail calls Exceptions: Stack-cutting Exceptions: Stack-unwinding Threads context switching Coroutines (w. function call) setjmp/longjmp Function call/return PLDI'06 PLDI’06 TLDI’07 YALEU/DCS/TR-1336

SCAP specifications: (p, g) –p: State  Prop –g: State  State  Prop Specifications f:... sw $ra, -4($fp) jal h ct: lw $ra, -4($fp)... jr $ra {(p 0, g 0 )} {(p 1, g 1 )} g0g0 g1g1 g 0 S S’  S’.$ra = S.$ra … Challenges –f uses the “right” return addr.? –Hoare triple {p} f {q}? In different basic blocks! {$ra = n …}

Program Spec. and Code Pointers jal f jal h jr $ra g0g0 g4g4 p0p0 p4p4 g1g1 p1p1 g2g2 p2p2 g3g3 p3p3 … Program Specification  ::= {f 1  (p 1,g 1 ), …,f n  (p n,g n )} “safe” to return ( jr $ra ): –$ra  dom(  )   ($ra)=(p,g) –p holds at the time of return

SCAP : Stack Invariant p0p0 g0g0 p1p1 g1g1 p2p2 g2g2 p3p3 g3g3 g 0 S 0 S 1  S 1.$ra     (S 1.$ra))=(p 1, g 1 )  p 1 S 1 g 0 S 0 S 1  g 1 S 1 S 2  S 2.$ra     (S 2.$ra)=(p 2, g 2 )  p 2 S 2 g 0 S 0 S 1  g 1 S 1 S 2  g 2 S 2 S 3  S 3.$ra     (S 3.$ra)=(p 3, g 3 )  p 3 S 3 jr $ra Logical control stack Always safe to return? S0S0 S1S1 S2S2 S3S3 …

SCAP : Stack Invariant WFST(n, g 0, S 0,  )   S 1. g 0 S 0 S 1   p 1,g 1.  (S 1.$ra)=(p 1, g 1 )  p 1 S 1  WFST(n-1, g 1, S 1,  ) WFST(0, g 0, S 0,  )    S 1. g 0 S 0 S 1 Invariant: p S   n.WFST(n, g, S,  ) p0p0 g0g0 p1p1 g1g1 p2p2 g2g2 p3p3 g3g3 jr $ra Logical control stack S0S0 S1S1 S2S2 S3S3

SCAP : Invariant Preservation Inv(S): p S   n.WFST(n, g, S,  ) c S’ p S   n.WFST(n,g,S,  ) S p’ S’   n.WFST(n,g’,S’,  )  p’,g’

SCAP: call p0p0 g0g0 p1p1 g1g1 jr $ra p0p0 g0g0 p g jal f p S  WFST(n, g, S,  )p 0 S 0  WFST(n+1, g 0, S 0,  ) S S0S0 n+1 n … p S  p 0 S 0 p1p1 g1g1 n … p S  g 0 S 0 S 1  p 1 S 1 p S  g 0 S 0 S 1  g 1 S 1 S 2  g S S 2 g 0 S 0 S 1  S 0.$ra = S 1.$ra S1S1 S1S1 S2S2 S2S2

SCAP: ret p g p1p1 g1g1 p1p1 g1g1 jr $ra p S  WFST(n, g S,  )p 1 S 1  WFST(n-1, g 1 S 1,  ) n n-1 … … p S  g S S 1 S S1S1

Generalization: Stack Unwinding/Cutting g1g1 p1p1 jr ra p g + p1p1 g1g1 p g Multi-ret p1p1 g1g1 jr ra p g Tail-call

Example: setjmp/longjmp int rev(int x){ if (setjmp(env) == 0){ cmp0(x, env); return 0; }else{ return 1; } void cmp0(int x,jmp_buf env){ cmp1(x, env); } void cmp1(int x,jmp_buf env){ if (x == 0) longjmp(env, 1); else return; } jmp_buf env = …;

Applications of SCAP malloc/free [Feng et al. PLDI'06, Xiang et al. QSIC'06] thread scheduler [Feng et al. TLDI'07] garbage collectors [McCreight&Shao PLDI'07]

OCAP Outline of This Talk The OCAP Framework SAGL: modular concurrency verification SCAP: stack-based control abstractions Embedding and linking in OCAP LnLn … Mechanized Meta-Logic (CiC) … C1C1 Modeling of the machine L1L1 CnCn SAGLSCAP

Threads and Scheduler Thread code C 1,…,C n –Certified in SAGL as concurrent code –Do not know about thread queue SCAP SAGL + Scheduler C S –Certified in SCAP as sequential code –Manages thread queue TQ –Do not touch H

How to Specify Scheduler/ yield jal yield yield: jr $ ra yield g S1S1 S2S2 |S 2 | |S 1 | SAGLSCAP

Embedding of SCAP ( (p, g) ) scap = , S. p S   n.WFST(n, g, S,  ) ┝ scap C:  ┝ ocap C :|  | If, then Soundness: Embedding: Supports proof reuse.

Embedding of SAGL ( ((a,p),A,G) ) sagl = , S.  Q={pc 1, …, pc n }.  (pc i ) =((a i,p i ),A i,G i )  InMem(Q)  (p 1  …  p n )  (a 1  …  a n )  NI((A 1,G 1 ), …, (A n,G n ))  stable(a i,A i ) Embedding:

Soundness of SAGL Embedding ┝ sagl C:  y ┝ ocap C:|  |. If, then

Certifying The System ┝ sagl C i :  i ┝ scap C s :  y ┝ ocap C s :  y  y ┝ ocap C i :  i SCAP SAGL the “link” rule ┝ ocap C i  C s :  i   y

More Applications of OCAP TAL [Morrisett'98] malloc lib.GCs [TLDI'07] [Lin et al. TASE'07]

Conclusion Goal: modular verification of system software –modules use different computation features –modules are at different abstraction levels Solution: to certify different modules using different logics –SCAP: stack-based control abstractions –SAGL: modular concurrency verification –CMAP: assume-guarantee with dynamic threads –… –OCAP: interoperation between different verification systems

Conclusion Applications –malloc/free [Feng et al. PLDI'06, Xiang et al. QSIC'06] –garbage collectors [McCreight&Shao PLDI'07] –thread scheduler [Feng et al. TLDI'07] –threads + scheduler [Feng et al. TLDI'07] –TAL + mem. alloc [Feng et al. TLDI'07] –TAL + GC [Lin et al. TASE'07] SCAPOCAP

Ongoing and Future Work Certifying OS –Mini-OS: ongoing work –More realistic OS in the future, e.g. OS for embedded systems Concurrency with relaxed memory models –relaxed memory models and STM General Hoare-logic (next generation of OCAP) –based on a uniform semantic model of features –support concurrency, partial correctness, frame rules … Push verification to high level code –C-like code with inlined assembly –Automated spec. inference and theorem proving

Thank you! Acknowledgments Zhong Shao, Hongxu Cai, Rodrigo Ferreira, Yu Guo, Andrew McCreight, Zhaozhong Ni, Gang Tan, Alex Vaynberg, Sen Xiang

Backup Slides

Certifying the Mini-OS Modeling of x86 machine –Real mode –PIC, keyboard, hard drive –8- and 16-bit integers Verifying the code –Multi-threaded code SAGL with extension of interrupts –Scheduler, thread lib., interrupt handler SCAP with extension of interrupts –Interactions with devices (PIC, PIT, keyboard, HDR) Finite state machine in the specifications

Statistics FIFO scheduler/context switching code –30 line of MIPS code –1400 line of proof scripts malloc/free: –42/50 line of MIPS code –1400/1560 line of proof scripts

Statistics [McCreight et al. PLDI’07]

Statistics – TAL & GC [Lin et al. TASE'07]

SCAP: tail call p0p0 g0g0 jr $ra p g S n … j f p0p0 g0g0 jr $ra S0S0 n … p S  WFST(n, g S,  )p 0 S 0  WFST(n, g 0 S 0,  ) p S  p 0 S 0 p S  g 0 S 0 S 1  g S S 1 S1S1 S1S1

CMAP A-G reasoning –properly nested structures: P1 || P2 low-level code –fork/join based structures –dynamic thread environment CMAP –unbounded dynamic creation/termination –similar to the support of dynamic mem. alloc. –change A and G to approximate the dynamic thread env. …

H1H1 pc 1 pc n … TQ r 31 r0r0 ct Scheduler in SCAP yield: pick one word ( pc i ) from TQ swap pc i and r 31 jr r 31 … jal yield ct: … Thread code: H1H1 pc 1 pc n … TQ r 31 r0r0 ct pc

Scheduler in SCAP yield: (p s, g s ) pick one word ( pc i ) from TQ swap pc i and r 31 jr r 31 gsgs g s  (  r  {r 1,…,r 30 }.[r]=[r]’)   Q, Q ’.  p’.  Q  {[r31]} = Q ’  {[r31]’} WFTQ( Q )  p’ WFTQ( Q ’ )  p’ H1H1 pc 1 pc n … TQ r 31 r0r0 H1H1 pc 1 pc n … TQ r 31 r0r0 ct p s   Q.WFTQ( Q )  True

Example: A-G reasoning … [100] := m; … [101] := n; … G 1 : [101] = [101]' A 1 : [100] = [100]' G 2 : [100] = [100]' A 2 : [101] = [101]'

Example: SAGL reasoning … [100] := m; … [101] := n; … {(emp, 100  _) } -{(emp, 101  _)} G 1 : emp A 1 : emp G 2 : emp A 2 : emp

OCAP: Code pointers codeptr(f,a)   (f) = a (Spec)  ::= {f  a} *  a codeptr Not well-founded! Support of first-class code pointers:

OCAP: Code Pointers A generic specification: (CdSpec)  ::= … (Assert) a  Spec  State  Prop (Spec)  ::= {f   } * (  )  ,S. … No interoperation between multiple systems…  a codeptr a   codeptr  ()()

OCAP: Foreign Languages (LangTy) L ::= CiC Terms  Type (CodeSpec)  ::= CiC Terms  L Inductive TalType : Type := T_int : TalType T_pair: TalType -> TalType-> TalType … Inductive Tal2Type : Type := T2_int : TalType T2_pair: TalType -> TalType-> TalType … L ::= TalType | Tal2Type | …  1 ::= T_int | T_pair  1  1 | …  TalType  2 ::= T2_int | T2_pair  2  2 | …  Tal2Type

OCAP: Specifications (OCdSpec)  ::=   L.( L  Assert )* L (Spec)  ::= {(f 1,  1 ), … (f n,  n )} (Assert) a  Spec  State  Prop (Interp) ( ) L  L  Assert (LangTy) L ::= CiC Terms  Type (CodeSpec)  ::= CiC Terms  L Not well-founded:  ( ) L a

OCAP: Specifications (OCdSpec)  ::= (Spec)  ::= {(f 1,  1 ), … (f n,  n )} (Assert) a  Spec  State  Prop (Interp) ( ) L  L  Assert (LangTy) L ::= CiC Terms  Type (CodeSpec)  ::= CiC Terms  L (LangID)  ::= n  nat (LangDict) D ::= {  1 ,…,  n  }  LangID  (  L. L  Assert)  ( ) L a  