Prof. John Nestor ECE Department Lafayette College Easton, Pennsylvania ECE VLSI System Design Lecture 5 - Verification and Testbenches February 10, 2003

2/10/03ECE Lecture 52 Announcements  Lab Assistant needed for Dr. Rich’s Lab (Thurs.)  Reading  Handout: “Validating the Intel® Pentium® 4 Microprocessor”, Proceedings of the 39th Design Automation Conference, June  References  J. Bergeron, Writing Testbenches: Functional Verification of HDL Models, 2nd ed., Kluwer Academic Publishers, 2003.

2/10/03ECE Lecture 53 Where we are...  Last Time  A Little More about Delay  System Tasks and Functions  A little more FSM coding  More Verilog features  Recent Developments in Verilog  Today  Discuss Lab 2  Verification and Testbenches Verification Overview Basic Testbench Coding

2/10/03ECE Lecture 54 Lab 2 - Parameterized Counter  Part 1 - Code the Counter  Declare N as parameter  Use synchronous clear  Test for Q==N-1

2/10/03ECE Lecture 55 Review: Counter w/Synchronous Clear module counter(clk, clr, Q, carry); input clk, clr; output [3:0] Q; output carry; reg [3:0] Q; // a signal that is assigned a value assign carry = (Q == 4'b1111); posedge clk ) begin if (clr) Q <= 4'd0; else Q <= (Q + 1) ; end endmodule Q changes on clk edge (usually preferred)

2/10/03ECE Lecture 56 Lab 2 (cont’d) - Verifying the Parameterized Counter  Structural Description  Instantiate counters with N=8, N=10, N=16  Cascade pairs of counters  Test using Verilogger  Synthesize & plot schematic

2/10/03ECE Lecture 57 Verification  Goal of verification:  Demonstrate functional correctness of a design  Attempt to find design errors  Attempt to show that design implements specification  Importance of Verification  Costs of design errors can be high (think “Pentium Floating-Point Error” ~ $300M!)  According to [1], verification consumes about 70% design effort in current systems design [1] J. Bergeron, Writing Testbenches: Functional Verification of HDL Models Kluwer Academic Publishers, 2000.

2/10/03ECE Lecture 58 Verification - Reconvergence Model  Verification checks a “transformation” for correctness  RTL Design and Coding  Synthesis  Physical Design  Reconvergence Model: Initial Specification Transformation Result Transformation

2/10/03ECE Lecture 59 Verification of RTL Design  The Idea: Written Specification RTL Code RTL Coding Verification  How it Really Works: Written Specification RTL Code RTL Coding Verification Interpretation

2/10/03ECE Lecture 510 Redundancy in Verification  Use separate individuals for design, verification Written Specification Verification RTL Code RTL Coding Designer Interpretation Verifier Interpretation

2/10/03ECE Lecture 511 Functional Verification Approaches  Black box  Verify using module I/O ports only  No knowledge of implementation  No access to internals  White box  Verify using module I/O ports and internals  Full knowledge of implementation  Full access to internals during simulation  Gray box  Verify using module I/O ports only  Full knowledge of implementation  No access to internals during simulation

2/10/03ECE Lecture 512 Verification vs. Testing  Verification identifies design errors  Does the design correctly implement the specification?  Does it perform calculations correctly?  Performed before manufacturing  Testing identifies manufacturing faults  Does each chip function properly?  Applied after manufacturing Specification Netlist Design Verification Manufacturing Testing Silicon

2/10/03ECE Lecture 513 Verification Tools  Linting Tools  Simulators  Code Coverage Tools  Formal Verification Tools  Version Control  Issue Tracking

2/10/03ECE Lecture 514 Linting Tools  Key idea: check code for potential problems that are legal HDL but not desirable, e.g.:  Latch inferences  Wire size mismatches  Types of Linting Tools  Commercial tools  Code reviews - peer review  Limitations  Sometimes report non-problems  Can’t look beyond syntax - “spell checker” analogy

2/10/03ECE Lecture 515 Code Coverage Tools  Key idea: check that all code is simulated  Check that all lines of code are exercised in simulation  Check that all paths through conditionals are exercise  Coverage tool function  Insert reporting code into HDL model  Summarize coverage & report to user  Key metric % coverage  Limitations  100% coverage difficult to accomplish  No guarantee of correctness

2/10/03ECE Lecture 516 Simulators  Allow testing of system response to stimulus  Event-driven - including delay models  Cycle-level - one evaluation per clock cycle  Simulation Tools  Waveform viewer  Testbenches - provide stimulus, check response  3rd party models - simulate existing designs Full models Bus-functional models  Limitations of simulation  Can’t be exhaustive for non-trivial designs  Performance bottleneck

2/10/03ECE Lecture 517 Other Verification Tools  Verification languages (e.g. e, Vera)  Used to specify and generate testbenches  Abstraction used to increase productivity  Revision control - used as in software engineering  Formal Verification  Equivalence checking - prove that input, output are equivalent  Model checking - Prove assertions concerning design properties, e.g. Reachability of states Deadlock avoidance Completion of transaction in an interface

2/10/03ECE Lecture 518 Testbenches  A testbench is HDL code to verify a module  Apply input vectors to module inputs  Check module outputs  Report errors to user  Why use a testbench instead of Verilogger?  Portability - testbench will work on any HDL simulator  Automatic checking - don't have to interpret waveform  Expressability - can use the full semantics of HDL to: generate input vectors (possibly from input file) check output vectors control simulation

2/10/03ECE Lecture 519 Design with Testbenches: Typical Approach  Develop a testbench for each module in design  Use for debugging when module design is created  Use to check for errors when module design is changed  Use to check synthesized result  In software, this is known as a "unit test"  Testbenches are essential in large chip design  Design team may include hundreds of people, who work on different subsystems  Testbenches allow semi-automatic checking when different subsystems are changed  Chip design groups do this with "simulation farms"

2/10/03ECE Lecture 520 Instance of M2 Instance of M3 Instance of M1 Definition of Module M4 Testbenches in Hierarchical Design  Example Hierarchy  Module M4 - Top-level module  Modules M1, M2, M3 - used as instances in M4  Create testbenches for all modules M1, M2, M3, M4  What if we change M2?  First run M2 testbench  Next, run M4 testbench

2/10/03ECE Lecture 521 Coding Testbenches in Verilog HDL Module Instance: Device Under Verification (DUV) Testbench Module

2/10/03ECE Lecture 522 Testbench Approaches - Visual Inspection Device under Verification (DUV) Stimulus Generator Testbench File Waveform Viewer OR Text Output

2/10/03ECE Lecture 523 Testbench Approaches - Output Comparison Device under Verification (DUV) Testbench File Reference Model Stimulus Generator Output Comparator Error/Status Messages “Gold” Vectors

2/10/03ECE Lecture 524 Testbench Approaches - Self-Checking Device under Verification (DUV) Stimulus Generator Output Signals Input Signals Testbench File Output Checker Error/Status Messages

2/10/03ECE Lecture 525 Comparing Approaches  Visual inspection  Only practical for small designs  Automatic support: Verilogger timing diagram editor  Output comparison  Effective when a good reference model is available  Used by ASIC foundries - “Gold” vectors are legal definition of a “functional” chip  Output checking  Most difficult to code  Mandatory for large designs

2/10/03ECE Lecture 526 Coding Testbenches  Use simulation features of HDL  Initial blocks  Functions & tasks  System Tasks  Parallel tasks (fork / join)

2/10/03ECE Lecture 527 Verilog Testbench Design  General approach:  Use initial block to apply vectors  Use # delay operator to sequence input changes in time operator to synchronize with clock  Use $display to show output, print messages  Common variations  Write a task (procedure) to do common checking  Use a separate always block to generate the clock

2/10/03ECE Lecture 528 Testbench Example - Comb. Logic  Develop a testbench for a comparator module module compare (a, b, aeqb, agtb, altb); input [7:0] a, b; output aeqb, agtb, altb; assign aeqb = (a == b); assign agtb = (a > b); assign altb = (a < b); endmodule  Do a simple test - no checking (yet)

2/10/03ECE Lecture 529 Testbench for Compare Module module compare_bench; reg [7:0] a, b; wire aeqb, agtb, altb; compare DUV(a,b,aeqb,agtb,altb); initial begin a = 0; b = 0; #10 a = 1; b = 0; #10 a = 255; b = 5; #10 b = 255; #10 a = 127; #10 $stop(); end // initial endmodule

2/10/03ECE Lecture 530 Aside - How Verilogger Works  Timing diagram editor creates a testbench file!  Example: Mux2 timing diagram

2/10/03ECE Lecture 531 Review - Structural Mux module mynand(a, b, nandout); input a, b; output nandout; reg nandout; or b) nandout = ~(a & b); endmodule module myinvert(ia, iout); input ia; output iout assign iout = ~ia; endmodule module mux2(ma, mb, msel, mout); input ma, mb, msel; output mout; wire moutbar, t1, t2; myinvert I1(msel,mselbar); mynand N1(ma,mselbar,t1); mynand N2(mb,msel,t2); mynand N3(t1,t2,mout); endmodule

2/10/03ECE Lecture 532 Verilogger Testbench - Mux2 // Auto-generated test bench created by VeriLogger Pro at Wed Feb 13 13:18: // Timing model is min/max timing. `timescale 1ns / 1ps module syncad_top; wire [1:0] tb_status; reg [1:0] tb_status_driver; assign tb_status = tb_status_diver; reg ma; reg mb; reg msel; wire mout; initial tb_status_driver = 1'b1; //diagram always running in auto-generated file initial begin //SIGNAL ma ma = 1'b1; # // Abs Time: ma = 1'b0; # // Abs Time: ; end initial begin //SIGNAL mb mb = 1'b0; # // Abs Time: mb = 1'b1; # // Abs Time: ; end

2/10/03ECE Lecture 533 Verilogger Testbench - Mux2 (cont’d) initial begin //SIGNAL msel msel = 1'b0; # // Abs Time: msel = 1'b1; # // Abs Time: msel = 1'b0; # // Abs Time: ; end mux2 mux2(.ma ( ma ),.mb ( mb ),.msel ( msel ),.mout ( mout )); initial begin end endmodule

2/10/03ECE Lecture 534 Testbench Example - Sequential Logic  Develop a testbench for this BCD counter: module bcdcounter(clk, reset, enb, Q, carry); input clk, reset, enb; output [3:0] Q; output carry; reg [3:0] Q; // a signal that is assigned a value assign carry = (Q == 9) & enb; posedge clk ) begin if (reset) Q <= 4'd0; else if (enb) begin if (carry) Q <= 0; else Q <= Q + 1; end endmodule

2/10/03ECE Lecture 535 BCD Counter Testbench - Design Ideas  Use a separate always statement to drive clock (must set at beginning of initial block)  Write a task to check values  Use $display system task for error messages

2/10/03ECE Lecture 536 Testbench for BCD Counter module bcdcounter_bench; // signals for connecting the counter reg clk; reg reset; reg enb; wire [3:0] Q; wire carry; // testbench variables; integer i; // counter instance bcdcounter DUT(.clk(clk),.reset(reset),.enb(enb),.Q(Q),.carry(carry));

2/10/03ECE Lecture 537 Testbench for BCD Counter (cont'd) task check; input [3:0] Q, check_Q; input carry, check_carry; begin if (Q != check_Q) $display("Error at time %t: Expected Q=%d, Actual Q=%d", $time, check_Q, Q); if (carry != check_carry) $display("Error at time %t: Expected carry=%d, Actual carry=%d", $time, check_carry, carry); end endtask // note clock drives both counter and bench always #10 clk = ~clk;

2/10/03ECE Lecture 538 Testbench for BCD Counter (cont'd) initial begin clk = 0; // necessary to set it to a known value! reset = 0; enb = clk); // do a reset and check that it worked reset = clk); check(Q,0,carry,0); // now try counting a few cycles #1 reset = 0; #1 enb = 1; for (i=0; i<9; i=i+1) clk); check(Q,i,carry,0); <-- error here! end

2/10/03ECE Lecture 539 Testbench for BCD Counter (cont'd) // now check the carry count should be clk); check(Q,9,carry,1); // now check the clk); check(Q,0,carry,0); // intentional error - count !=2, carry != clk) check(Q,2,carry,0); check(Q,1,carry,1); repeat clk); #1 check(Q,9,carry,1); #5 enb = 0; #2 check(Q,9,carry,0); repeat clk); $stop(); // all done! end // initial endmodule

2/10/03ECE Lecture 540 Testbench Design  How do we verify designs that are too large for exhaustive simulation?  Identify key features How do we exercise? What is correct response?  Identify “Corner cases” Initial conditions “Boundaries” between modes of operation  Ensure code coverage does testbench exercise all of the Verilog code? Commerical coverage tools help this process  Use random inputs to test unanticipated cases

2/10/03ECE Lecture 541 Verification of Large Chips  Create a verification plan which specifies  Features necessary for first-time success Prioritization of features - essential vs. optional Which features should be exercised What the response should be  Testcases to exercise features  Process for reporting and fixing bug  Implement testbenches for each testcase  Report bugs & fix  Big question: when are you done?

2/10/03ECE Lecture 542 Coming Up  Verification and Testbenches  System Design Issues

2/10/03ECE Lecture 543 Lab 3 Overview  Self-checking testbench for generic counter  Identify important features  Create conditions that test these features  Check conditions  Write message when error occurs  “Insert” errors to demonstrate when self-check fails  Test for varying values of N (2, 8, 10, 16)

2/10/03ECE Lecture 544 System Design Issues  ASM Diagrams  Synchronization & Metastability  Handshaking  Working with Multiple Clocks