1. Chris Wilson and David L. Dill Computer Systems Laboratory Stanford University June, 2000 Reliable Verification Using Symbolic Simulation with Scalar Values

2. Verification Bottleneck time Bug rate Many “easy” fewer “hard” Directed testing Random testing “purgatory” tapeout

3. Current Approach time Bug rate Directed testing random testing Model checking emulation semi-formal methods

4. Our Approach time Bug rate Key issue: Reliability! Symbolic simulation

5. Reliability Definition:  Always gives some coverage when resource limits encountered.  Gives coverage proportional to effort. Ease of use  predictable coverage  useful feedback  easy to debug

6. Efficiency Efficiency = Coverage/Unit Effort Coverage  specified functionality  “input space” Effort  manpower  computer resource usage  schedule

7. Reliability vs. Efficiency Reliability Efficiency Directed testing Random testing Emulation Model Checking % of bugs found

8. Goal Have the reliability, ease of use of directed testing. AND… efficiency equal or greater than random testing.

9. Reliability vs. Efficiency Reliability Efficiency Directed testing Random testing Emulation Model Checking Target area

10. Symbolic test = directed test with symbolic values 23 408 0 5 0 Symbolic Simulation datain address interrupt dataout DUT req valid request = counter 0101 “read” “write” =0 pass/ fail dly

11. Symbolic Simulation Efficiency  1 symbolic test many directed tests. Ease of use  short tests => easy to write, debug. Blow up?  BDDs too unpredictable. How to prevent blow up?

12. Quasi-symbolic simulation Symbolic simulation externally scalar values internally  simulation run requires constant memory. Key ideas  Don’t compute exact value unless necessary. many don’t cares in large designs.  Trade time for memory. Multiple runs to generate exact values.

13. Don’t care logic Basic Algorithm & & & & X a a X b b X c c Symbolic variable X -a X a a 0 Obeys law of excluded middle! X Conservative approximation X X X “traditional” X value 0 Don’t care variables

14. Decision Procedure X a a X b b X X X & O O

15. Davis-Putnam Algorithm Tree Search…  Davis, Logemann, Loveland [DPLL62]. X a=0 a=1 X b=0 b=1 0 0 X 0 evaluate case split unit propagate

16. Decision Procedure X ? a=0 a=1 Variable selection heuristic: pick relevant variable by propagating from inputs. & & O X a a X b b X X X 0 0 0 X b b 0 1 0 X b b 0 ? 0 Test is Unsatisfiable!

17. Reactivity Reactive Test  test behavior depends on circuit. Most tests require reactivity  since goal is to find all bugs…  must support reactivity efficiently.

18. Reactivity example Set ‘request’ = READ; Set ‘reqv’ = “1”; wait for ‘ack’; check that ‘data’ = expected_data; stop;

19. Reactivity example Set ‘request’ = READ; Set ‘reqv’ = “1”; wait for ‘ack’; check that ‘data’ = expected_data; stop; What if ‘ack’ = “X”?

20. Wait Statement Set ‘request’ = READ; Set ‘reqv’ = “1”; wait for ‘ack’; check that ‘data’ = expected_data; stop; wait for ‘ack’ == “1”; ‘ack’ == F = “X” ‘ack’ == T = “X” Virtual thread

21. wait for ‘ack’; Cycle 2 check that ‘data’ = expected_data; stop; ‘ack’ == T = “X” wait for ‘ack’; ‘ack’ == F = “X”

22. Stopping check that ‘data’ = expected_data; stop; Guard = “X” Stop? or not stop?

23. Modify Davis-Putnam... if guard condition = “X” when stopped…  prove that test can really stop in this cycle.  Case split on guard condition. case split on fail/pass condition only if stop = “1”. Stopping

24. Modify Davis-Putnam... if guard condition = “X” when stopped…  prove that test can really stop in this cycle.  Case split on guard condition. Do not allow unit propagation. case split on fail/pass condition only if stop = “1”.  Unit propagation is allowed. Disallowing unit propagation allows method to be complete.

25. Related Work BDD-based Symbolic Simulation  STE [BryantSeger95], Innologic. Sequential ATPG SAT/ATPG-based Model Checking  BMC [Biere99], [Boppana99] Other SAT-based Semi-Formal Methods  [Ganai99]

26. Experiments Show that quasi-symbolic simulation can find bugs.  Test case bugs do not cause bottlenecks. Demonstrate graceful degradation  get good coverage if simulation time limit hit.

27. Experiment 1 Write/debug testcase for “hard” bug.  140K gate industrial design.  Not found in simulation or bringup! Four possible results  SAT - test case error.  TIMEOUT - test case error (device timeout.)  UNSAT - no bug found.  BUG - bug found.

28. Experiment 1 SAT193.831.4 TIMEOUT221.649.0 UNSAT952.3445.9 BUG178863.0 casesevalstime(sec.)

29. Experiment 2 Time limit hit! Highest covered sub-node

30. Experiment 2 Number of dependent variables in the test Maximum tree size

31. Conclusions Want to find all bugs faster.  Reliability is key. Use quasi-symbolic simulation  has the efficiency of random testing.  And reliability of directed testing. Experiments show it can be used as primary verification method.