1. FMCAD 20081 A Theory of Mutations with Applications to Vacuity, Coverage, and Fault Tolerance Orna Kupferman 1 Wenchao Li 2 Sanjit A. Seshia 2 1 Hebrew University 2 UC Berkeley TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAA

2. FMCAD 20082

3. 3 Bob This system is correct even under faults (e.g. flips in latches) Why? Convince me. It satisfies its specification under these faults. Doesn’t this mean the specification coverage is low? Adam So is my specification not good enough or is my system fault-tolerant? Need fault-tolerance! But also need to certify it!

4. FMCAD 20084 Problem  Current mutation-based metrics are inadequate to reason about specification coverage for fault-tolerant circuits in model checking.

5. FMCAD 20085 Preliminaries  Coverage Introduce ∆ to an implementation I and check I’ ² S.  Fault Tolerance I with fault f still satisfies S.  Vacuity Introduce ∆ to a specification S and check I ² S’. All three involve introducing mutations in the verification process!

6. FMCAD 20086 Contributions A theory of mutations:  formally ties together coverage and vacuity in model checking;  enables reasoning coverage for fault-tolerant circuits.

7. FMCAD 20087 Agenda  Related Work Coverage Vacuity  A Theory of Mutations Coverage and Vacuity are dual Aggressiveness amongst mutations  Applications  Conclusion

8. FMCAD 20088 Coverage  Is my specification complete?  Coverage metrics for model checking [HKHZ 99; KGG 99; CKV 01,03] FSM Coverage statepath

9. FMCAD 20089 Coverage  Functional Coverage in BMC [GKD 07]  Detect “forgotten cases” [Claessen 07]  Coverage for fault-tolerant systems [FPFRT 03, DBBDCMF 05] Single stuck-at fault model

10. FMCAD 200810 Vacuity  Is my specification satisfied trivially?  Vacuity detection [KV 99, 03; BBER 01; AFFGP 03; CG 04; BFGKM 05; BK 08] G (req → F grant)G (req → false) Replace a sub-formulae in the most challenging way. Trivially true in a system where req is never sent.

11. FMCAD 200811 Agenda  Related Work Coverage Vacuity  A Theory of Mutations Coverage and Vacuity are dual Aggressiveness amongst mutations  Applications  Conclusion

12. FMCAD 200812 Examples of Mutations  Can mutate inputs, outputs, or latches  Stuck-at  Restricting a signal to a value  Freeing (abstracting) a signal 1000 1001 1000  10011000 100X old new  10011000 Removes behaviors Adds behaviors Modifies behaviors

13. FMCAD 200813 A Theory of Mutations  Properties: Invertability: (C μ ) ν = C Monotonicity: I ² S → I μ ² S μ Duality  Interesting Mutations: Conditional stuck-at Conditional add/remove transitions Permuting events

14. FMCAD 200814 Duality I μ ² S ↔ I ² S ν,where ν and μ are dual mutations. low coveragevacuity

15. FMCAD 200815 Circuit with input = {z}, control signals = {x, y}, output = {x}, described by the state representation on the right. xy x z S simulates I’ and S’ simulates I 01 0 0,1 I 01 0 00 0 10 S’ remove behavior I’ add behavior 01 0 11 1 0,1 01 0 00 0 11 1 10 1 S 0 10 1

16. FMCAD 200816 Aggressiveness  Mutation  is more aggressive than if applying  makes it harder for the design to satisfy its specification.  I  ² S → I ² S or I ² S  → I ² S  ≥ imp  ≥ spec

17. FMCAD 200817 Some Aggressive Orders  Free(x) ≥ k-SEU(x)  Free(x) ≥ Stuck_at_0(x)  Free(x) ≥ Flip(x)  Delay_k+1 ≥ Delay_k  k-SEU(x) ≥ m-SEU(x) ≥ for k ≥ m  More interesting ones can be found in the paper.

18. FMCAD 200818 Coverage for Fault-tolerance  For a fault-tolerant system I and a set of mutations { j } such that I  j ² S for all 1≤j≤k.  The fault-tolerant system loosely satisfies S if there is a mutation  such that  j ≤ imp  for all 1≤j≤k; I  ² S.

19. FMCAD 200819 Agenda  Related Work Coverage Vacuity  A Theory of Mutations Coverage and Vacuity are dual Aggressiveness amongst mutations  Applications  Conclusion

20. FMCAD 200820 Applications  Useful vacuity information can be obtained for free from coverage checks.  Analyze coverage for fault-tolerant systems.  Improving specifications Catch bugs Strengthen environmental assumptions

21. FMCAD 200821 Vacuity from Coverage  S: G (sp[2..0] = 3’b110 → X (sp[2..0] = 3’b111)  In our experiment, applying the “Flip(x)” mutation to sp[0] still satisfies S.  S’: G (sp[2..0] = 3’b110 → X (sp[2..0] = 3’b110)  S & S’ → G ¬(sp[2..0] = 3’b110)

22. FMCAD 200822 Certifying Fault-Tolerance System behaviors Original low-coverage spec. System behaviors High-coverage spec. certifies system’s target resilience 1-SEU System behaviors 2-SEU

23. FMCAD 200823 Experiments VIS benchmarks, results obtained with Cadence SMV model checker

24. FMCAD 200824 Improving Specifications  Chip Multiprocessor Router [Peh 01]  However, the process still requires some user assistance. Simplied model S: G (ξ → X ¬(grant = 2’b11) S’: G (ξ → X (grant = 2’b10)

25. FMCAD 200825 Conclusion  A theory of mutations that Unifies coverage and vacuity Can be used to certify the correctness of fault-tolerant circuits  A new technique to tighten specifications  The ideas here can be applied to other verification techniques.

26. FMCAD 200826 Q & A Thank you!

27. FMCAD 200827 References