CONFIDENTIAL1 Biased Random Simulation Guided by Observability-Based Coverage Serdar Tasiran Compaq Systems Research Center, formerly GSRC, UC Berkeley Farzan Fallah Fujitsu Labs of America David G. Chinnery, Scott K. Weber, Kurt Keutzer UC Berkeley Serdar Tasiran Compaq Systems Research Center, formerly GSRC, UC Berkeley Farzan Fallah Fujitsu Labs of America David G. Chinnery, Scott K. Weber, Kurt Keutzer UC Berkeley

2 Simulation-based Functional Validation Simulation Input stimulus generation Design (RTL model) Reference model Monitors, assertions, comparison w/ ref. model Functional Validation

3 Simulation with Coverage Feedback Input stimulus generation Coverage measurement and analysis Diagnosis of unverified portions Simulation Design (RTL model) Reference model Monitors, assertions, comparison w/ ref. model Functional Validation

4 Our Work Simulation Input stimulus generation Coverage measurement and analysis Diagnosis of unverified portions Design (RTL model) Monitors, assertions, comparison w/ ref. model Functional Validation Reference model

5 Our work Simulation Input stimulus generation Coverage measurement and analysis Design (RTL model) Simulation Biased- random input generation 2 Tag coverage analysis (Observability- based coverage) 1 Outline Diagnosis of unverified portions Compute new biases to target non-covered tags 3

6 OutlineOutline Simulation Biased- random input generation Coverage measurement and analysis Compute new biases to target non-covered tags Design under test Simulation Tag coverage analysis (Observability- based coverage) 1

7 ObservabilityObservability  Simulation detects a bug only if –a monitor flags an error, or –design and reference model differ on a variable  Variables checked for functional correctness called observed variables. è Portion of design covered only when 1. it is exercised during simulation (controllability) 2. a discrepancy originating in that portion causes discrepancy in an observed variable (observability)  Low observability  false sense of security –Most of the design is exercised  Looks like high coverage –But most bugs not detected by monitors or reference model –They would have been detected if inputs chosen properly Simulation Design (RTL model) Reference model Monitors, assertions, comparison w/ ref. model Functional Validation

8 Tag Coverage [Devadas, Keutzer, Ghosh ‘96]  HDL code coverage metrics + observability requirement.  Bugs modeled as errors in HDL assignments.  A buggy assignment may be stimulated, but still missed EXAMPLES: –Wrong value generated speculatively, but never used. –Wrong value is computed and stored in memory  Read 1M cycles later, but simulation doesn’t run that long.

9 Tag Coverage [Devadas, Keutzer, Ghosh ‘96]  Generalization of “stuck-at” fault coverage to HDL code  Handles multi-valued variables  Error model: An HDL assignment computes a value –higher (+  or –lower (-  than the intended value. –Example: 3 +  represents values > 3 A = 3 C = F - 2A D  = K * C A +  = 3 ++ A F C DK ++ -- -- ???

10 Tag Coverage  Run simulation vectors  Tag one variable assignment at a time  Use tag calculus  Confirms that –HDL line is activated and –its effect is propagated to an observable variable  Tag Coverage: Subset of tags that propagate to observed variables  Efficient tool: OCCOM [Fallah, et. al.] A = 3 C = F - 2A D  = K * C ++ ++ -- -- ???

11 OutlineOutline Simulation Coverage measurement and analysis Design under test Simulation Tag coverage analysis (Observability- based coverage) Diagnosis of unverified portions Compute new biases to target non-covered tags Biased- random input generation 2  

12 Rationale for Biased-Random Vector Generation  Primary inputs selected according to a probability distribution  Trade-off between –Time to find “good” vectors –Time to simulate vectors  Typically > 50% of simulation is biased random simulation  Improved random vectors  better validation overall  Less intelligence for selecting next step but many more vectors –Can explore deeper into state space Find Simulate 0% 100% Portion of Computation Time

13  Primary inputs at each clock cycle selected according to a probability distribution –Distributions can be functions of circuit state  Distributions ( “weights” ) determined prior to simulation i1i1 i2i2 i3i3 s1s1 s2s2 P(i 1 = 0) = 0.7 P(i 1 = 1) = 0.3 P(i 2 = 0 | s 1 = 1) = 0.6 P(i 2 = 1 | s 1 = 0) = 0.7 Probability Distributions (“Weights”) P(i 3 = 0) = 0.4 P(i 3 = 1) = 0.6 Biased Random Vector Generation

14  Very unlikely to exercise certain cases with uniform random simulation Why Optimize Biases? i1i1 i2i2 i o  P(i 1 = 1) = P(i 2 = 1) = … = P(i 32 = 1) = 0.5   P(o = 1) =   Wunderlich [DAC ‘85, Int’l. Test Conf. ‘88] – –Even for combinational circuits several sets of biases required for good fault coverage   Biases must be picked based on targeted tags.

15 OutlineOutline Simulation Coverage measurement and analysis Design under test Simulation Tag coverage analysis (Observability- based coverage) Diagnosis of unverified portions Compute new biases to target non-covered tags 3   Biased- random input generation  

16  Optimization algorithm determines biases based on –Set of tags targeted –A structural netlist describing the circuit (BLIF-MV)  Intuitive goal –Maximize expected number of tags that will be covered COVER (Circuit,Tags) repeat while (tag coverage rate) > (threshold) Biases = Optimize_Input_Biases(Circuit, Tags) Biased_Random_Simulate(Circuit, Biases), Tags = Tags - Tags_Covered Optimizing Input Biases

17 Modeling Biased Random Simulation  Key subroutine for optimizing input biases: Estimate coverage for given primary input distributions.  Transition probabilities fixed at each state  Model circuit + random generation of inputs as Markov chain.  Long simulation runs  Analyze behavior of circuit at steady state. - Determine tag detection probability at steady state  Huge state space  Approximate analysis s0s0 s1s1 s2s2 s3s3 s4s4 i=0 i=1 i=0 i=1 i=0 i=1 P(s 0 ) = 0 P(s 1 ) = 0 P(s 2 ) = 0.25 P(s 3 ) = 0.25 P(s 4 ) = 0.5 P(i=1) = 0.2

18 Approximation I: Line Probabilities Compute probability distributions of state variables instead of states Prob((s 1,s 2 ) = (a 1,a 2 )) = Prob(s 1 =a 1 ) x Prob(s 2 =a 2 ) –Ignores correlations between latches –Devadas, et. al. [VLSI ’95]  Power estimates within 3% for benchmarks  Individual node distributions correct within 15% èRefinement: Group closely correlated state variables into a single variable. s1s1 s2s2 (s 1,s 2 ) = (a 1, a 2 ) (s’ 1,s’ 2 ) = (a’ 1, a’ 2 )

19 Steady-StateSteady-State Fixed-point prob(ns 1 = v 1 ) = prob( f 1 (i 1, i 2, …, i m, ps 1, ps 2, …, ps n ) = v i ) = prob(ps 1 = v 1 ) … prob(ns i = v j ) = prob( f n (i 1, i 2, …, i m, ps 1, ps 2, …, ps n ) = v j ) = prob(ps i = v i ) s1s1 s2s2 P(ps 1 =v 1 ) P(ps 2 =v 2 ) P(ns 1 =v 1 ) P(ns 2 =v 2 ) Given input probability distributions

20 Computing Latch PDs at Steady-State  Start with initial guess for latch PDs  Given PDs for inputs and latch outputs, compute PDs at latch inputs –Substitute new distributions at latch outputs  Repeat until convergence –Guaranteed under minor restrictions  Key computation: Propagating PDs –Given PDs at the inputs of a combinational circuit, determine PDs at each node. s1s1 s2s2 P(ps 1 =v 1 ) P(ps 2 =v 2 ) P(ns 1 =v 1 ) P(ns 2 =v 2 ) Given input probability distributions

21  Given probability distributions (PDs) at inputs and latch outputs compute PDs of circuit nodes Propagating probability distributions 0 1 i0i0 i1i1 i1i1 i1i1 i2i2 i2i2 P(i 0 = 0) P(i 0 = 2)   Represent each node as a function of primary inputs   Inputs assumed independent  Use recursive algorithm on MDD to compute node PDs

22  While propagating probabilities forward, impose limit on MDD size.  When limit reached, treat intermediate node as primary input  Correlations outside clusters ignored. Approximation II: Clustering 0 1 i0i0 i1i1 i1i1 i1i1 n1n1 n1n1 P(i 0 = 0) P(i 0 = 2) n1n1

23 Estimating Tag Detectability   Given PDs of each circuit node, estimate controllability and observability of tags   Recall  : – –Actual value of node x i is q – –Intended value is p – –q > p Controllability: Probability that x i = p at steady state (Done) Observability: Probability that x i = q causes change in observed variable. – –Function of PDs of other nodes – –May happen along a multi-cycle path p q r s xixi f yjyj

24 Observability Computation Observability: Probability that x i = p q causes change in observed variable.   Propagate observabilities backward, starting from observed variables   Determine observability of x i based on   observability of y j.   PDs of related circuit nodes   Form MDD representing condition for x i = p q to be observed   Compute probability that MDD evaluates to 1.   Many paths: Pick best cluster (y j ) p q r s xixi f yjyj

25 Observability Propagation   Propagate observabilities backward, starting from observed variables   Discrepancies may take multiple cycles to reach observed variable.   Perform several backward passes of observability computation – –Stop at e.g. 10 passes. Analysis too inaccurate for more passes.

26 Optimization Criteria merit cost   Intuitive goal of weight determination algorithm: – –Maximize expected number of tags that biased random simulation will cover   Use merit and cost functions to formalize goal   For a single tag (latch):   Add merit (cost) functions for all targeted tags (latches) Tag detection probability Deviation of latch distribution from uniform

27 Optimizing Input Distributions repeat For each primary input i Select a set of probability distributions p 0, p 1, p 2, …, p n For j=0,…,n Compute merit functions for P(i=1) = p j Pick the j that yields best merit function until no improvement in merit function

28 Experimental Results: s1423 Uniform biases Optimized biases

29 s1423 – early in simulation Uniform biases Optimized biases

30 Experimental Results: s5378 Second round of bias optimization starts Uniform biases Optimized biases

# latches# inputs # tags # iterations Memory (MB)

32 ConclusionsConclusions  On some examples, significant improvement in coverage with reasonable computational cost –No manual effort required –Longer uniform random simulations do not achieve same result  Coverage feedback is a powerful tool in input vector generation.  On other examples, coverage not improved.  Hundreds of simulations with different biases show no improvement  Circuit size is not t limiting factor: –Good results on some large circuits, bad results on some small ones –Close to complete coverage on large combinational benchmarks  For examples with bad coverage, most latches show no activity.  Conjecture: Ignoring input constraints and initialization sequences cause circuits not to be driven properly

33 ConclusionsConclusions  Biased random patterns do not provide enough control on the simulation run for some circuits.  Not a standalone technique. Must be used in conjunction with more powerful, deterministic methods. –ATPG, approximate reachability, …  Better biased random simulation complements other approaches

34 Future research directions  Current method limited to multi-valued variables with small ranges ØGeneralize method to larger datapaths. ØHandle datapath-control interaction  Experiment with higher level RTL descriptions. ØE.g., explore biased random simulation at instruction level.  Pure biased random simulation is too weak ØExplore choice of state-dependent biases ØAid bias selection in randomized test programs