1. VerificationTechniques for Macro Blocks (IP) Overview Inspection as Verification Adversarial Testing Testbench Design Timing Verification

2. Overview of Macro Verification Overall verification plan Strategies Sub-block simulation Macro simulation Prototyping Limited production

3. Overall Verification Plan Functional verification based on Requirements Specifications Behavioral models Extended functions (for generalization and re-use) Engineering review of verification plan Documentation of verification steps

4. Strategy: Bottom Up Testing Verify individual sublocks Focus on coverage of data values and internal states Verify complete macro (block) Focus on interfaces and timing among sub-blocks Test I/O timing and “corner cases” Perform prototyping Proto-board PC board FPGA Platform based design*

5. Verification vs. Validation In the hardware world: Verification means testing Validation means “formal proof” In the software world: Verification means “formal proof” Validation means testing Don’t be confused…

6. Verification vs. Manufacturing Testing Verification is to confirm that the design is correct It meets the requirements, specifications It meets timing, power, and area goals It meets the re-use goals It assumes a debugging cycle Manufacturing Test Fixed set of tests (vectors from above) Go/NoGo tests

7. Fundamental Ideas Controllability Can you wiggle every wire? Can you set every bit? Observability Can you see the state of every bit? Can you see the value on every wire? During validation, we can do this in the simulator – especially in a bottom-up mode (In manufacturing test this is not so easy)

8. Types of Verification Tests Compliance Testing Against specifications and/or standards Corner Case Testing Try to break the design with complex scenarios “Answer Yes or No [Y]?” > fred Random Testing Use statistical models to generate data patterns Real Code Testing Work with the software team, use real data Regression Testing Keep all tests in a suite and add to them

9. Testing in the Product Development Cycle More complete tests find more subtle bugs Fixes often make design more complex Test time goes up New bugs are introduced by fixes Repeat… time bugs found

10. Verification Tools Simulation functional, cycle, RTL, gate level, circuit level Testbench automation tools Verisity (and others) testing language Code coverage tools Measurements of what got wiggled Hardware Modeling Comparison against / with real hardware Emulation High speed hardware boxes interfaced to simulator Prototyping

11. “Static” Tools Static timing analysis tools “Lint-like” tools Formal verification tools Model checking  Build abstract model of design and environment  Uses “smart” exhaustive methods for safety and liveness tests Liveness: something good will eventually happen Safety: nothing bad will ever happen Theorem proving  Uses automatic theorem provers to verify assertions about the design

12. Inspection as Verification “The fastest, cheapest, and most effective way to detect and remove bugs is by careful inspection of the design and the code” Design reviews Large group review requirements, specifications, assumptions and overall design Code reviews Smaller groups perform line-by-line review of implementation Reviews must be done in a collegial atmosphere with the goal of enhanced quality, not to asses designer performance

13. Prototyping Use Prototyping when: You can afford it  $cost  $time You can not wait for the time it takes to find bugs using available simulation techniques  The prototype is just a fast simulator for the device itself

14. Adversarial Testing The designer’s tests have the goal to show that the system will work… Adversarial Testing uses a verification team of experts in testing all kinds of designs Goal: develop tests to show that the system will fail Report failures, and test cases to design team

15. Sub-Block Simulation 1. Use automated response checking Not appropriate for designer to check waveforms “by eye” How many times will it be done? Use of automated test to build macro block testbench Use assertions in code to test assumptions 2. Test sub-blocks for 100% statement and path coverage

16. Testbench The testbench For each component Is the model of its external world Must generate (all) legal input sequences Must check output sequences Must alert the designer and log discrepancies between expected and actual results The testbench is often more complex than component itself

17. Sub-block Testbench Design Input transaction data types, sequences Output transaction data types, sequences Input Transaction Generator Output Transaction Checker Device Under Test (DUT) Assume simple I/O relationships

18. Input Stimulus Generate “all” legal sequences of inputs Look at functionality of DUT Check corner cases Achieve 100% coverage

19. Output Checking Check for legal sequences of outputs Look at functionality of DUT Check corner cases Check input/output relationships Ideally each sub-block has  Simple Functionality  Simple I/O

20. Getting ready for integration Timing checks Timing plan for each sub-block Clocking, handshaking Timing budgets, per-cycle latency budget Area and power checks Are the sub-blocks within their bugets

21. Concurrent Design, Synthesis & Testbench Development Block Integration Functional Specifications, Documentation, Behavioral Model Validate against Specification and Test Plan / Productization Develop Macro- Test Bench, Test Cases Coordinate TimingSpecify Blocks Code and TestSynthesize Test Full BlockSynthesize Full Block Macro Specifications / Behavioral Model Macro Partitioning and Block Specifications Block Design Test Team Design TeamSynthesis Team

22. Integration Steps Integrating the sub-blocks Top level netlist Final synthesis Final testing Manufacturing testability Scan insertion ATPG Test coverage

23. Timing Verification Post synthesis timing “more real” delays  Delay models based on statistical models  No layout  Not real delays Check for clock/data skew Signal loading, critical path analysis

24. Post Synthesis Test Bench Development Refine the Behavioral Model for testing and verification Wrappers for each sub-block as needed (for signal compatibility before/after synthesis)  After synthesis every wire is a “bit” Document test plan and test vectors

25. Macro Testbench Design is more complex Functional complexity I/O Complexity More people involved Testbench will be delivered with product Testbench will be used to develop scan path and manufacturing tests

26. Generating Testbenches Not simple “do scripts” Complex state machines: Loop  Generate stimulus  Get output  Check results  Log results  Change stimulus parameters

27. Simple Example Simulate Busses Check bus timing Check data ranges Simulate ROM Also, supply data values Check address ranges Register File Also, r/w timing Overwriting data I/O Also, response to asynchronous events Response to out of range data DUT ROM Memory Register File I/O Address Bus Data Bus I/O Bus Testbench Clock/Timing Generator

28. Help! Use existing models for bench components Have a test team, with their own test library Take library components and enhance them with checking/logging functions Use behavioral models for bench Bus functional models  Need to have statistical bounds on input/output timing  Use random generators for stimulus sequences Use C/C++ models and co-simulation for bench Need to have a “master bench controller” to coordinate I/O behaviors on all components

29. Use the Behavioral Model Input Transaction Generator Output Transaction Checker Synthesized Block Behavioral Model

30. Use Commercial Tools Vera - Synopsis Specman Elite – Verisity Both tools use abstract data types to allow user to define data-ranges, states, and scenarios. Tools support statistical models, for stimulus, checking, and logging (beyond the scope of this class)

31. Test, Test, Test Back to Requirements and Specifications Corner Cases Random Cases Regression Testing Code Coverage

32. Types of Coverage Statement Coverage # times each statement executed Branch Coverage Each side of each if/then else executed Condition Coverage Checks each boolean clause in branch condition Path Coverage Paths between blocks (if – if, if – else, else – if) Trigger Coverage Each signal on sensitivity list Toggle Coverage 1/0 transition on every signal

33. Still Not Enough Does not verify that code works right, just that it was run Does not verify that design meets specifications or requirements Does not check results of optimization and synthesis tools