1. Using Formal Coverage Analyzer for Code Coverage improvement
Yuri Tatarnikov
San Jose, CA, the USA
October 5, MES-2018

2. Agenda
Goal of this job
Design features in terms of coverage constructs
The results
Conclusion

3. What is the goal of this job?
At the end of verification, when you created and debugged all tests from your test plan your block has some level of coverage, but you need to reach ~ 100% coverage.
For it you will work with Designer to:
Add some tests to cover
To exclude some not covered constructs

4. To exclude not covered constructs
Designer makes decision to exclude some part of not covered constructs. It is his or her responsibility. Reasons: lazy, difficult or impossible to create scenario of test, not real situation. You create and support exclusion file for it.
There are some constructs, which in principle can not be covered (UNRs) – to get the list of UNRs for your block – goal of this presentation

5. UNR constructs are determined by Synopsys Formal Coverage Analyzer (FCA)
How does it work? It uses Formal Proof Techniques.
In this particular case FCA takes any code coverage construct (from line, condition, FSM and toggle)
and tries to prove, that it is possible to make it true.
If yes => construct is coverable,
If not => construct is uncoverable.

6. Process of getting UNRs
You (verification guy) – run regression with coverage enabled to get Coverage Data Base
FCA has Coverage Data Base as its input
You provide clock(s), reset(s) and reset durability OR give simulation snapshot, which represents your Design initial state
FCA selects uncovered coverage constructs (for line, toggle, condition, FSM metrics) and tries to generate timing diagram, which will cover them.
If it can not find – this construct is uncoverable.
FCA generates file with all unreachable constructs (UNR)
You put this file in the command, which reports coverage, to exclude UNRs

7. Example –condition coverage
Suppose we have the statement
if (a&b) then …
As the result of simulation we have combination
(a==0) & (b==1) # not covered.
FCA tries to prove, that this combination can be reached by some combination of block inputs , possible by several clock sequence of inputs. Space of possible intermediate states can be huge, which makes the use of this approach in some cases not practical (too much memory, too long time of proof)

8. When is the best time to apply this technique?
For FCA tool - less uncovered constructs - better.
For me(verification engineer) - I finished test development according to test plan. With good, detailed test plan code coverage metrics (line, toggle, condition, FSM) are on the level 70%-80%
For Designer - he/she did not spend any special time yet to try to improve coverage

9. Jobs were done with 2 blocks: A and B Block A
Features in terms of code coverage (pretty small, but functionally not easy):
Lines
Conditions - 473
Signal bits (for toggle) – 27974
FSM states – 61

10. Jobs were done with 2 blocks: block B
Features in terms of code coverage:
Lines
Conditions
Signal bits (for toggle) –
FSM states – 72

11. Block A: Coverage state before FCA
Lines - 123/1835
Conditions - 57/473
Signal bits (toggle) – 315/27974
FSM states – 1/61
Note: xxx/yyy means – uncovered/total

12. Block B: Coverage state before FCA
Lines /43988
Conditions /13470
Signal bits (for toggle) – 9013/672904
FSM states – 3/72
Note: xxx/yyy means – uncovered/total

13. Block A: FCA results - total constructs not covered - 509
- FCA selected as uncoverable ( 18%)
> Line Coverage
- # found : 123
- # covered : 93
- # uncoverable : 30
> Condition Coverage
- # found : 57
- # covered : 42
- # uncoverable : 15
> FSM State Coverage
- # found : 1
- # covered : 1
> FSM Transition Coverage
- # found : 13
- # covered : 13
> Toggle Coverage
- # found : 315
- # covered : 269
- # uncoverable : 46

14. Block B: FCA results - total constructs not covered - 18455
- FCA selected as uncoverable ( 12%)
> Line Coverage
- # found : 7035
- # covered : 7035
> Condition Coverage
- # found : 2377
- # covered : 131
- # uncoverable : 2246
> FSM State Coverage
- # found : 3
- # covered : 3
> FSM Transition Coverage
- # found : 27
- # covered : 23
- # uncoverable : 4
> Toggle Coverage
- # found : 9013
- # covered : 8925
- # uncoverable : 88

15. Blocks code coverage before and after
block_A
SCORE LINE COND TOGGLE FSM
before FCA UNR exclusion
after FCA UNR exclusion
block_B
SCORE LINE COND TOGGLE FSM
before
after exclusion

16. FCA run resourses
one Linux workstation with 8 processors Xeon, each with 4 cores, physical memory 32GB.
FCA job allocated 1 processor, its 4 cores. Each core allocated up to 17GB of virtual memory.
Block A ~ 1 day elapsed time
Block B ~ 5 days

17. Conclusions Preparation for run is very minimal ~ 10 min for me
It is worth to use – remember: found 2338 uncoverable constructs for block B
Preparation for run is very minimal ~ 10 min for me
FCA job is highly paralleled and running it on network can shorten job time
The subject of FCA – blocks only, because they have all logic within
Not using any input constraints – getting minimal array of UNR (UNReachable constructs), but it excludes much of manual work.
This tool became part of the design and verification methodology within our organization following its successful evaluation
Next level of use Formal – to proof /falsify bug assumptions.

18. END