1. IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007 P ROPERTIES I NCOMPLETENESS E VALUATION BY F UNCTIONAL V ERIFICATION 1

2. M AIN CONTRIBUTION A coverage methodology based on a combination of static and dynamic verification that allows us to reduce the evaluation time with respect to pure formal approaches 2

3. I NTRODUCTION Simulation-based techniques  Lack of exhaustiveness Formal verification  Overcome the exhaustiveness problem  Properties are derived from informal design specifications.  Model checking: prove the presence of bugs, but not their absence 3

4. V ERIFICATION F LOW B ASED ON M ODEL C HECKING 4

5. I NTRODUCTION – M ODEL C HECKING To increase the effectiveness of model checking Vacuity detection: look for properties that hold in a model and can be strengthened without causing them to fail Property coverage: address the question of whether enough properties have been defined How many properties should be defined to completely check the implementation? Coverage metric! 5

6. Mutation-based ACTL, LTL, and CTL State coverage  path coverage  transition-based coverage Implementation-based State explosion problem Cannot precisely reflect the completeness of properties 6 I NTRODUCTION – P REVIOUS W ORK How about use mutation coverage jointly with dynamic verification to address the quality of the model checking process?

7. B ACKGROUND  Kripke structure K = {S, S 0, R, L}  FSM M = {I, O, S, s 0, R}  Product machine M P = M 1 X P M 2  Retroactive network 7 Ι ε

8. M ETHODOLOGY O VERVIEW 8

9. G ENERATION OF F AULTY I MPLEMENTATIONS  The proposed methodology is independent of the adopted fault model  Different fault models can provide different estimations of the property completeness  Functional fault model  Bit coverage  has been proved to be related to design errors  Bit coverage fault model assumptions  Bit failure: stuck-at 0 or stuck-at 1  Condition failure: stuck-at true or stuck-at false  Single fault: A faulty implementation is generated for each fault 9

10. G ENERATION OF F AULTY I MPLEMENTATIONS ( CONT.)  Detectable faults 10

11. G ENERATION OF F AULTY I MPLEMENTATIONS ( CONT.)  A non-optimized algorithm  If fail then f is ε-detectable  Time-consuming and very likely state explosion 11

12. E STIMATION OF G OLDEN M ODEL I NCOMPLETENESS  Ƥ-detectable and Ƥ-det  Property coverage 12

13. E STIMATION OF G OLDEN M ODEL I NCOMPLETENESS ( CONT.)  C P = 1  formal properties are complete w.r.t. a particular fault model  Non-optimized algorithm 13

14. E STIMATION OF G OLDEN M ODEL I NCOMPLETENESS ( CONT.)  Witnesses and counterexamples  Tools can provide witnesses and counterexamples for CTL and LTL properties  Input witness and input counterexample 14

15. W ITNESS C OVERAGE  Property coverage can be estimated by using input witnesses  Under some conditions, C P = C w 15

16. P ROOF OF C P = C W 16  Consider the safety and liveness properties separately

17. P ROOF OF C P = C W ( CONT.) 17

18. P ROOF OF C P = C W ( CONT.) 18

19. I NCREMENTAL P ROPERTY C OVERAGE C OMPUTATION 19

20. C OVERAGE A CCURACY C OMPARISON  Combining static and dynamic verification makes this methodology can deal with real industrial circuits.  The methodology presented in this paper covers faults rather than states.  Can estimate coverage more accurate (compare with previous works) 20

21. E XPERIMENTAL R ESULTS 21

22. I NSPIRE TO IC/CAD C ONTEST  Functional fault model  Estimate coverage by fault instead of properties 22