FMCAD A Theory of Mutations with Applications to Vacuity, Coverage, and Fault Tolerance Orna Kupferman 1 Wenchao Li 2 Sanjit A. Seshia 2 1 Hebrew University 2 UC Berkeley TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAA

FMCAD 20082

3 Bob This system is correct even under faults (e.g. flips in latches) Why? Convince me. It satisfies its specification under these faults. Doesn’t this mean the specification coverage is low? Adam So is my specification not good enough or is my system fault-tolerant? Need fault-tolerance! But also need to certify it!

FMCAD Problem  Current mutation-based metrics are inadequate to reason about specification coverage for fault-tolerant circuits in model checking.

FMCAD Preliminaries  Coverage Introduce ∆ to an implementation I and check I’ ² S.  Fault Tolerance I with fault f still satisfies S.  Vacuity Introduce ∆ to a specification S and check I ² S’. All three involve introducing mutations in the verification process!

FMCAD Contributions A theory of mutations:  formally ties together coverage and vacuity in model checking;  enables reasoning coverage for fault-tolerant circuits.

FMCAD Agenda  Related Work Coverage Vacuity  A Theory of Mutations Coverage and Vacuity are dual Aggressiveness amongst mutations  Applications  Conclusion

FMCAD Coverage  Is my specification complete?  Coverage metrics for model checking [HKHZ 99; KGG 99; CKV 01,03] FSM Coverage statepath

FMCAD Coverage  Functional Coverage in BMC [GKD 07]  Detect “forgotten cases” [Claessen 07]  Coverage for fault-tolerant systems [FPFRT 03, DBBDCMF 05] Single stuck-at fault model

FMCAD Vacuity  Is my specification satisfied trivially?  Vacuity detection [KV 99, 03; BBER 01; AFFGP 03; CG 04; BFGKM 05; BK 08] G (req → F grant)G (req → false) Replace a sub-formulae in the most challenging way. Trivially true in a system where req is never sent.

FMCAD Agenda  Related Work Coverage Vacuity  A Theory of Mutations Coverage and Vacuity are dual Aggressiveness amongst mutations  Applications  Conclusion

FMCAD Examples of Mutations  Can mutate inputs, outputs, or latches  Stuck-at  Restricting a signal to a value  Freeing (abstracting) a signal  X old new  Removes behaviors Adds behaviors Modifies behaviors

FMCAD A Theory of Mutations  Properties: Invertability: (C μ ) ν = C Monotonicity: I ² S → I μ ² S μ Duality  Interesting Mutations: Conditional stuck-at Conditional add/remove transitions Permuting events

FMCAD Duality I μ ² S ↔ I ² S ν,where ν and μ are dual mutations. low coveragevacuity

FMCAD Circuit with input = {z}, control signals = {x, y}, output = {x}, described by the state representation on the right. xy x z S simulates I’ and S’ simulates I ,1 I S’ remove behavior I’ add behavior , S

FMCAD Aggressiveness  Mutation  is more aggressive than if applying  makes it harder for the design to satisfy its specification.  I  ² S → I ² S or I ² S  → I ² S  ≥ imp  ≥ spec

FMCAD Some Aggressive Orders  Free(x) ≥ k-SEU(x)  Free(x) ≥ Stuck_at_0(x)  Free(x) ≥ Flip(x)  Delay_k+1 ≥ Delay_k  k-SEU(x) ≥ m-SEU(x) ≥ for k ≥ m  More interesting ones can be found in the paper.

FMCAD Coverage for Fault-tolerance  For a fault-tolerant system I and a set of mutations { j } such that I  j ² S for all 1≤j≤k.  The fault-tolerant system loosely satisfies S if there is a mutation  such that  j ≤ imp  for all 1≤j≤k; I  ² S.

FMCAD Agenda  Related Work Coverage Vacuity  A Theory of Mutations Coverage and Vacuity are dual Aggressiveness amongst mutations  Applications  Conclusion

FMCAD Applications  Useful vacuity information can be obtained for free from coverage checks.  Analyze coverage for fault-tolerant systems.  Improving specifications Catch bugs Strengthen environmental assumptions

FMCAD Vacuity from Coverage  S: G (sp[2..0] = 3’b110 → X (sp[2..0] = 3’b111)  In our experiment, applying the “Flip(x)” mutation to sp[0] still satisfies S.  S’: G (sp[2..0] = 3’b110 → X (sp[2..0] = 3’b110)  S & S’ → G ¬(sp[2..0] = 3’b110)

FMCAD Certifying Fault-Tolerance System behaviors Original low-coverage spec. System behaviors High-coverage spec. certifies system’s target resilience 1-SEU System behaviors 2-SEU

FMCAD Experiments VIS benchmarks, results obtained with Cadence SMV model checker

FMCAD Improving Specifications  Chip Multiprocessor Router [Peh 01]  However, the process still requires some user assistance. Simplied model S: G (ξ → X ¬(grant = 2’b11) S’: G (ξ → X (grant = 2’b10)

FMCAD Conclusion  A theory of mutations that Unifies coverage and vacuity Can be used to certify the correctness of fault-tolerant circuits  A new technique to tighten specifications  The ideas here can be applied to other verification techniques.

FMCAD Q & A Thank you!

FMCAD References