1. Mikhail Chupilko, Alexander Protsenko
Recognition and Explanation of Incorrect Behavior in Simulation-Based Hardware Verification
Mikhail Chupilko,
Alexander Protsenko

2. Outline Development of hardware models Test system architecture
Establishing of conformance
Trace event recombination
Examples
Conclusion

3. Hardware model design
They are developed in Hardware Description Languages, like Verilog or VHDL
The result of development is the program being executed in HDL simulator
The common approach for verification of hardware models is testing of HDL programs
To automatize testing is possible by means of executable models (e.g. in C++)

4. HDL programs input S; CLK output R1, R2; void design() { S
while(true) {
wait(S);
delay(6);
R1 = 1;
delay(1);
R1 = 0;
R2 = 1;
R2 = 0; }
CLK S R1 R2
6 cycles
Parallel assignments

5. Hardware model behavior

6. Reference model-based test oracle
HDL
Test oracle
Reference model
Reference model
reactions
Reaction arbiters
Input interface adapters
Output interface adapters
Stimuli
Reaction
comparators
HDL-model
reactions

7. Typical representation of verification results
Data on so-and-so wire are not correct, the correct data are…
More advanced: data in so-and-so packet generated on so-and-so step are not valid…
The most advanced: data in so-and-so packet should be received on the following cycle, but they were confused with the following packet…

8. Example of the bug found
0xf953e8d83a9b9209  0x19c3827ab2920e58
0x19c3827ab2920e58  0xf953e8d83a9b9209 8

9. Reference model-based test oracle
HDL
Test oracle
Reference model
Reference model
reactions
Reaction arbiters
Input interface adapters
Output interface adapters
Stimuli
Reaction
comparators
HDL-model
reactions
Diagnostics
subsystem

10. Conformance reaction checking
If there is a rearrangement of reference model reactions, where properties of data, iface equality and time restrictions are satisfied

11. If there is a problem…

12. Simplified diagnostics process based on trace of reactions

13. Recombination of reactions with requirement of total equal in data
(null, null) -> remove the pair;
(x, x) -> (null, null);
{(x, y), (y, x)} -> {(x, x), (y, y)};
{(x, null), (null, x)} -> (x, x);
{(x, y), (y, null)} -> {(x, null), (y, y)};
{(x, y), (null, x)} -> {(x, x), (null, y)};
{(x, z), (y, x)} -> {(x, x), (y, z)};

14. Recombination of reactions with requirement of partial equal in data
{(x, y), (y`, x`)} -> {(x, x`), (y`, y)};
{(x, null), (null, x`)} -> (x, x`);
{(x, y`), (y, null)} -> {(x, null), (y, y`)};
{(x, y), (null, x`)} -> {(x, x`), (null, y)};
(x, x`) -> (null, null).

15. Example-1

16. Example-2 Part of trace in text form Table with recombination
[INCORRECT](iface=output_mau cycle=107): Incorrect reaction: Msg{spec=MAU_pha(0xb0418b0f), impl=MAU_pha(0x96e7741c)}, Msg{spec=MAU_dst_pha(0x6), impl=MAU_dst_pha(0)} expected at [107,157]
[INCORRECT](iface=output_mau cycle=105): Incorrect reaction: Msg{spec=MAU_pha(0x96e7741c), impl=MAU_pha(0xb0418b10)}, Msg{spec=MAU_dst_pha(0), impl=MAU_dst_pha(0x6)} expected at [105,155]
[INCORRECT](iface=output_mau cycle=98): Incorrect reaction: Msg{spec=MAU_pha(0x5bf9a01e4), impl=MAU_pha(0xb0418b0f)}, Msg{spec=MAU_dst_pha(0x3), impl=MAU_dst_pha(0x6)} expected at [98,148]
Table with recombination
Explanation based on history of applied rules of recombination

17. C++TESK Testing ToolKit
Web:

18. Conclusion
Better representation of verification results means less efforts for their interpretation
Application of the rule set allow to regroup reaction pairs finding possible hint of what is going bad in the system behavior
The method has been implemented in C++TESK Testing ToolKit and has been successfully used in a number of projects
Future research is connected with showing of history usage in diagnostics and/or giving hints to localization of bugs

19. THANK YOU
Any questions?

20. Example-3

21. Example-4