ICS 216 Embedded Systems Validation and Test Instructor: Professor Ian G. Harris Department of Computer Science University of California Irvine

Reading Material No single book. Some Verilog book is useful (some are on reserve at the library) Selected papers Useful book - Writing Testbenches Functional Verification of HDL Models, Janick Bergeron, Second Edition Useful book -Introduction to Formal Hardware Verification, Thomas Kropf

Course Structure Three problem sets: two on simulation-based validation and one on formal verification Final Project - individual or pairs - Final Project Proposal - ~2 pages - Final Project Proposal Presentation - 20 min. - Final Document - Complete description + any code

Embedded Systems “A computer that doesn’t look like a computer” Complex computations hidden behind a simple interface Ex. Cell phone, digital camera, automobile, etc. Design requirements are varied 1.Power, performance, cost, life- critical Design components are varied 1.Digital/analog hardware 2.Systems/application software 3.Mechanical sensors/actuators

Validation is a bottleneck in the design process High cost of design debug (designers, time-to-market) High cost of faulty designs (loss of life, product recall) Hardware and software are often used together Hardware and software are designed separately Covalidation is performed late in the process, necessitating long redesign loops Hardware/Software covalidation problem is more acute Importance of Embedded System Validation

Validation and Test Validation/Verification  Ensuring that the design matches the designer’s intent/specification  Detection of design errors accidentally made by the designer(s)  Usually includes the debugging task as well (Manufacturing) Test  Detection of physical defects in the implementation  Defects appear as a result of manufacturing or of wear mechanisms  Tests the implementation, not the design

Validation vs. Verification Formal verification - Use of proof-based techniques –Model checking, equivalence checking –Time complexity is high –Confidence is high for specified properties Simulation based validation - Use of simulation –Full-chip/full-design validation –Confidence is not well quantified Pentium 4 bugs found by FV (492) vs. Validation (5809) [1] [1] B. Bentley, “Validating the Intel Pentium 4 Microprocessor,” DAC’01.

Embedded System Design Flow Intent Nat. Lang. Spec. Exec. Behavior HW Structural HW BehavioralSW Procedural Machine Code COTS HW/SW HW Design SW DesignPre-Designed Fabrication Embedded System Designer’s Intent: Vague idea of behavior known only to designer. Natural Language Specification: “Complete” behavioral description Written in English (or other). Executable Behavior: A simulatable description of the behavior. Written in a procedural language (SystemVerilog, SystemC, …)

Simulation-Based Validation Intent Nat. Lang. Spec. Exec. Behavior HW Structural HW BehavioralSW Procedural Machine Code COTS HW/SW HW Design SW DesignPre-Designed Fabrication Embedded System Not Simulatable Simulatable Use of simulation to compare two design representations Comparison to natural language spec (or intent) requires manual interaction Comparison to executable behavior requires cosimulation of HW, SW, and pre-designed

Formal Verification, Model Checking Intent Nat. Lang. Spec. Exec. Behavior HW Structural HW BehavioralSW Procedural Machine Code COTS HW/SW HW Design SW DesignPre-Designed Fabrication Embedded System Properties Evaluating design to determine if properties always hold Properties capture some aspect of the designer’s intent - “A traffic light cannot stay RED forever” AF(color!=RED) Strictly limited in design complexity allowable

Formal Verification, Equivalence Checking Proving that two different structural design representations are equivalent Works well for combinational logic. Requires manual interaction for sequential logic. Limited to structural hardware. Infeasible for behavioral descriptions. F1 = a’ + b’ F2 = (ab)’ A/0B/1 C/0D/ A/0B/1 1 0

Elements of Validation Test Generation Simulation Response Analysis Test Sequence Test Responses Writing a good Test Bench Evaluating a Test Bench - Coverage Metrics Using the simulator (vcs) Manual evaluation using the debugger (virsim, CLI) -Viewing waveforms -Insertion of breakpoints -Automatic evaluation -Comparison with known-good results -Assertions -Self-checking code