1. Lecture 9: Testbench and Division
UCSD ECE 111
Prof. Farinaz Koushanfar
Fall 2018
UCSD ECE 111, Prof. Koushanfar, Fall 2018
Some slides are courtesy of Prof. Lin

2. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Testbench
UCSD ECE 111, Prof. Koushanfar, Fall 2018

3. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Testbench
A testbench is an HDL module that is used to test another module, called device under test (DUT).
Testbench contains statements to apply inputs to the DUT and, ideally, to check that the correct outputs are produced.
The input and desired output patterns are called test vectors
UCSD ECE 111, Prof. Koushanfar, Fall 2018

4. UCSD ECE 111, Prof. Koushanfar, Fall 2018
DUT
module sillyfunction ( input logic a, b, c, output logic y); assign y = ~a & ~b & ~c | a & ~b & ~c | a & ~b & c; endmodule
Create a module to be tested
This is a simple module, so we can perform exhaustive testing by applying all eight possible test vectors
UCSD ECE 111, Prof. Koushanfar, Fall 2018

5. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Simple Testbench
module testbench1() logic a, b, c, y; // instantiate device under test sillyfunction dut(a, b, c, y); // apply inputs one at a time intial begin a = 0; b = 0; c = 0;#10; $display("a=%b, b=%b, c=%b, y=%b", a, b, c, y); c = 1; #10; b = 1; c = 0; #10; a = 1; b = 0; c = 0;#10; b = 1; c = 0; #10; end endmodule
First applies the input pattern 000 and waits for 10 time units
Then applies 001 and waits for 10 more units
So forth until all eight possible inputs have been applied
UCSD ECE 111, Prof. Koushanfar, Fall 2018

6. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Simple Testbench
module testbench1()
logic a, b, c, y;
// instantiate device under test
sillyfunction dut(a, b, c, y);
// apply inputs one at a time
intial begin
a = 0; b = 0; c = 0;#10;
$display("a=%b, b=%b, c=%b, y=%b", a, b, c, y);
c = 1; #10;
b = 1; c = 0; #10;
a = 1; b = 0; c = 0;#10;
b = 1; c = 0; #10;
end
endmodule
Instantiates the DUT
Applies the inputs
Blocking assignments and delays are used to apply the inputs in the appropriate order
User must view the results of the simulation and verify by inspection that the correct outputs are produced.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

7. UCSD ECE 111, Prof. Koushanfar, Fall 2018
initial statement
The initial statement executes the statements in its body at the start of simulation.
initial statement should be used only in testbenches for simulation, not in modules intended to be synthesized into actual hardware
Hardware has no way of magically executing a sequence of special steps when it is first turned on
UCSD ECE 111, Prof. Koushanfar, Fall 2018

8. Self-checking Testbench
Checking for correct outputs is tedious and error-prone
Determining the correct outputs is much easier when the design is fresh in your mind
Making minor changes and need to retest weeks later is troublesome
Write a self-checking testbench
UCSD ECE 111, Prof. Koushanfar, Fall 2018

9. Self-checking Testbench
In SystemVerilog, comparison using == or != is effective between signals that do not take on the values of x and z
Testbenches use the === and !== operators for comparisons of equality and inequality, respectively, because these operators work correctly with operands that could be x or z
module testbench2(); logic a, b, c, y; // instantiate device under test sillyfunction dut (a, b, c, y); // apply inputs one at a time and check results initial begin a = 0; b = 0; c = 0; #10; assert (y === 1) else $error(“000 failed.”); c = 1; #10; assert (y === 0) else $error(“001 failed.”); b = 1; c = 0; #10; assert (y === 0) else $error(“010 failed.”); assert (y === 0) else $error(“011 failed.”); a = 1; b = 0; c = 0; #10; assert (y === 1) else $error(“100 failed.”); assert (y === 1) else $error(“101 failed.”); assert (y === 0) else $error(“110 failed.”); assert (y === 0) else $error(“111 failed.”); end endmodule
UCSD ECE 111, Prof. Koushanfar, Fall 2018

10. UCSD ECE 111, Prof. Koushanfar, Fall 2018
assert statement
assert statement checks if a specified condition is true
If not, it executes the else statement
The $error system task in the else statement prints an error message describing the assertion failure
assert is ignored during synthesis
UCSD ECE 111, Prof. Koushanfar, Fall 2018

11. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Test Vectors
Writing code for each test vector also becomes tedious, especially for modules that require a large number of vectors.
Place the test vectors in a separate file
example.tv
000_1
001_0
010_0
011_0
100_1
101_1
110_0
111_0
UCSD ECE 111, Prof. Koushanfar, Fall 2018

12. Testbench with Test Vector
module testbench3(); logic clk, reset; logic a, b, c, y, yexpected; logic [31:0] vectornum, errors; logic [3:0] testvectors [10000:0] // instantiate device under test sillyfunction dut (a, b, c, y); // generate clock always begin clk = 1; #5; clk = 0; #5; end // at start of test, load vectors // and pulse reset initial begin $readmemb("example.tv", testvectors); vectornum = 0; errors = 0; reset = 1; #27; reset = 0; // apply test vectors on rising edge of clk clk) begin #1; (a, b, c, yexpected) = testvectors[vectornum]; // check results on falling edge of clk clk) if (~reset) begin // skip during reset if (y !== yexpected) begin // check result $display("Error: inputs=%b", {a, b, c}); $display(" outputs=%b (%b expected)", y, yexpected); errors = errors + 1; vectornum = vectornum + 1; if (testvectors[vectornum] === 4'bx) begin $display("%d tests completed with %d errors", vectornum, errors); $finish; endmodule
UCSD ECE 111, Prof. Koushanfar, Fall 2018

13. Testbench with Test Vector
// generate clock
always begin
clk = 1; #5;
clk = 0; #5;
end
Generates a clock using an always/process statement with no sensitivity list
Continuously reevaluated
(clock is not needed for this combinational module but included for the sake of understanding)
UCSD ECE 111, Prof. Koushanfar, Fall 2018

14. Testbench with Test Vector
// at start of test, load vectors
// and pulse reset
initial begin
$readmemb("example.tv", testvectors);
vectornum = 0; errors = 0;
reset = 1; #27; reset = 0;
end
Beginning of the simulation, reads the test vectors from a text file and pulses reset for two cycles
$readmemb reads a file of binary numbers into the testvectors array. $readmemh is similar but reads a file of hexadecimal numbers.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

15. Testbench with Test Vector
// at start of test, load vectors
// and pulse reset
initial begin
$readmemb("example.tv", testvectors);
vectornum = 0; errors = 0;
reset = 1; #27; reset = 0;
end
Beginning of the simulation, reads the test vectors from a text file and pulses reset for two cycles
$readmemb reads a file of binary numbers into the testvectors array. $readmemh is similar but reads a file of hexadecimal numbers.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

16. Testbench with Test Vector
// apply test vectors on rising edge of clk
clk) begin
#1; (a, b, c, yexpected) = testvectors[vectornum];
end
Waits 1 time unit after the rising edge of the clock
To avoid any confusion if clock and data change simultaneously
Sets the three inputs (a, b, c) and the expected output, (yexpected) based on the four bits in the current test vector
UCSD ECE 111, Prof. Koushanfar, Fall 2018

17. Testbench with Test Vector
// check results on falling edge of clk
clk)
if (~reset) begin
// skip during reset
if (y !== yexpected) begin
// check result
$display("Error: inputs=%b", {a,b,c});
$display(" outputs=%b (%b expected)", y, yexpected);
errors = errors + 1;
end
vectornum = vectornum + 1;
if (testvectors[vectornum] === 4'bx) begin
$display("%d tests completed with %d errors", vectornum, errors);
$finish;
Compares the generated output, y, with the expected output, yexpected
prints an error if they don’t match
Repeats until no more valid test vectors in the testvectors array.
$finish terminates the simulation
SystemVerilog supports upto 10,001 test vectors
UCSD ECE 111, Prof. Koushanfar, Fall 2018

18. Testbench with Test Vector
Summary
Reads the test vectors from the file
Applies the input test vector to the DUT
Wait, checks that the output values from the DUT match the output vector
Repeat until reaching the end of the test vector file
New inputs are applied on the rising edge of the clock
Output is checked on the falling edge of the clock
Errors are reported as they occur
Total number of test vectors applied and number of errors occurred are printed at the end
UCSD ECE 111, Prof. Koushanfar, Fall 2018

19. Testbench (Verification)
UCSD ECE 111, Prof. Koushanfar, Fall 2018

20. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Introduction
Verification is the process of experimenting our design with possible test scenarios.
Verification contains many phases that includes Testcase generation, coverage, monitor, …, etc.
Before tape-out our design should be verified more than 90% successfully.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

21. Verification Methodology Key Features
The key features associated with the verification methodology are:
Assertion Based Verification
Functional Coverage Driven Verification
Constrained Random Verification
Scoreboard & Checker
UCSD ECE 111, Prof. Koushanfar, Fall 2018

22. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Key Features…
Assertion Based Verification
Assertion Based Verification helps to detect more functional bugs earlier in the verification process and enables short verification cycle times and faster debugging.
Functional Coverage Driven Verification
Functional Coverage Driven Verification is the process in which the result of functional coverage is used to drive the verification to completion. It also gives a more structured approach to verification, instead of writing more testcases.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

23. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Key Features…
Constrained Random Verification
It is well suited for designs that have diverse set of operations and sequences that are difficult to cover completely. Constrained random tests are faster to write and enables faster verification of the design.
Scoreboard & Checker
Scoreboard and Checker is a mechanism used to dynamically predict the response of the design and check the observed response against the predicted response. Usually it refers to the entire dynamic response-checking structure.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

24. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Verification Flow
UCSD ECE 111, Prof. Koushanfar, Fall 2018

25. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Verification Flow
Design Specification
Verification flow starts with understanding the architecture specification of the design under verification. Once the architecture is well understood then comes the verification plan.
Verification Plan
The verification plan comprises the preparation of test case scenarios and testbench environment
UCSD ECE 111, Prof. Koushanfar, Fall 2018

26. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Testcase Scenarios
The testcase scenarios document includes all the possible combinations to test the functionality of the design under test
UCSD ECE 111, Prof. Koushanfar, Fall 2018

27. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Self Checking
Self checking test cases are written in such a way that is tests itself.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

28. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Self Checking…
Corner
Corner cases covers the scenarios which are prone to errors and non-trivial, where there are more possibilities of uncovering bugs.
Directed
Directed cases covers all the scenarios related to the general features and they are straight forward cases which are written in an orderly manner.
Coverage
Coverage cases are based on the coverage report and they contain the scenarios which are missed out in the other normal categories.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

29. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Non-Self Checking
Non - self checking test cases has no self checking code and they are coded such that if the program execution reaches the last line of the test case properly then a success code will be moved into a specific register. After the completion of execution that register can be checked for the success code to ensure the correctness of the design.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

30. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Non-Self Checking…
Random
Random cases cover all the possible combinations of data as well as scenarios in a random manner.
Constrained Random
Constrained Random cases cover the all possible combinations of data as well as scenarios in a random manner, constrained for valid combinations.
Stress
Stress cases exercise the logic with more number of combinations of scenarios as well as data for a long time to ensure the reliability of the design under stress conditions.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

31. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Non-Self Checking
Negative
Negative cases covers the scenarios which are not possible under normal cases. They are used to analyze the behavior of the design under error conditions.
Application Specific
High level cases targeted for a particular application normally written in “C” language.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

32. Testbench Environment
The testbench environment includes different types of environments to be developed for effectively verifying the design under verification. The testbench environments are of two types based on the testing strategies adopted.
Top Level
Block Level
UCSD ECE 111, Prof. Koushanfar, Fall 2018

33. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Top Level
The top level testing includes the testing of the design as a whole and the stimulus forced in this case is the memory image generated from the assembly level or high level test files. Based on the requirement for the verification, there are three types of environments.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

34. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Top Level
Static
Performs functional verification in a static manner using functional model. The results of DUT and model are dumped and compared for equivalence once the execution is over.
Dynamic
Performs timing verification in a dynamic manner using the cycle accurate model. The results of DUT and model are compared for equivalence at each and every cycle dynamically.
Coverage The following are considered :
Line coverage
Branch coverage
FSM coverage
UCSD ECE 111, Prof. Koushanfar, Fall 2018

35. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Block Level
Block level verification includes the verification of the different inner modules of the design for checking the scenarios which cannot be covered using the top level verification. It includes both static and dynamic environments and the stimuli in this case are inputs generated using tasks.
The testbench provides different debugging options and also uses sentinels to ensure the error free flow of data.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

36. SystemVerilog Based Verification Environment
UCSD ECE 111, Prof. Koushanfar, Fall 2018

37. SystemVerilog Based Testbench Development
Sequencer
Sequencer is an object that defines a set of transactions to be executed and controls the execution of other sequences.
Driver
It is the component responsible for executing or processing transactions and provides stimuli to the design-under-test (DUT).
Monitor
This block continuously monitors the DUT signals and bus functions.
Scoreboard
Driver requests are transferred to the scoreboard via monitor block.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

38. Scoreboard Components
Functional Coverage Accumulation
Functional coverage is a measure of which design features have been exercised by the tests.
Dynamic response checking mechanism
The comparison of golden data with DUT data is performed dynamically. This block controls the overall verification environment such as reporting, violations and changing of the stimulus during run-time.
Reporting mechanism
Compiler directives to issue messages throughout the simulation. (Error, Info, Warning)
UCSD ECE 111, Prof. Koushanfar, Fall 2018

39. Integration, Simulation
Once the RTL coding is over, it is hooked up in the testbench environment and then the verification process can be started.
Simulation
Design is simulated using the simulation tool. Any error in the simulation triggers an error message which is dumped into an error log.
TestSuite Regression / Coverage
The Regression Environment is developed using perl and shell scripts, to automate the regression run.
UCSD ECE 111, Prof. Koushanfar, Fall 2018

40. UCSD ECE 111, Prof. Koushanfar, Fall 2018
Bug Report Mechanism
Bug reports are maintained in Google docs for all the projects in the verification phase. Once the bug is encountered it is updated in the bug report with detailed information. Status of the open bugs is updated regularly.
UCSD ECE 111, Prof. Koushanfar, Fall 2018