Improved Design Debugging using Maximum Satisfiability Sean Safarpour, Hratch Mangassarian, Andreas Veneris Mark H. Liffiton, Karem A. Sakallah University of Toronto University of Michigan

Outline Introduction & Motivation Debugging with max-sat Debugging with approximate max-sat Experiments Conclusion

Introduction & Motivation Verification is a major pain in VLSI design cycle check correctness against specs identify corner cases evaluate test coverage Automated solutions exist When design fails, find error source manual debugging process time consuming specification specification design specification verification specification pass? yes pass? no debug debug

Introduction & Motivation What is “debugging” identifying error source/locations in design i.e. lines of code in the RTL (Verilog/VHDL) Debugging is a verification bottleneck manual task consumes up to 50% of verification effort Automated debugging or error localization finds source of error in circuit automatically use information from verification (testbench, traces, etc.) current techniques not practical in industry

Introduction & Motivation Why is debugging hard to automate? Problem Complexity grows O(n# errors) many gate-level errors in high-level designs Algorithms work on gate/Boolean models circuits are big: > 100,000 gates Algorithms must model sequential behavior traces are long: > 1000s of clock cycles expensive using time frame expansion Debugging techniques over past decades Simulation, path tracing, BDDs SAT & QBF techniques: last 5 years promising results

Introduction & Motivation What are the inputs to automated debuggers? Erroneous design (one or multiple gate/high-level bugs) Simulation Trace: stimulus to observe erroneous behavior Correct/Expected Response: trace provided from specs Erroneous design Automated Debugger potential error locations Simulation & Expected traces

Introduction & Motivation Previous SAT & QBF techniques How can erroneous circuit be modified to satisfy the expected response under the given stimulus? Insert instruments in circuit/CNF such that satisifying assignment => error locations Find all error location tuples up to N Observation The union of the constraints is UNSAT removing clauses corresponding error locations makes the problem SAT Maximum Satisfiability problem

Debugging example a a c c 1 1 b b e e 1 d d (!a+c) (!b+c) (a+b+!c) Simulation trace a b c d e a b c d e Correct circuit Correct response 1 1 1 Erroneous circuit (!a+c) (!b+c) (a+b+!c) (c+!e) (d+!e) (!c+!d+e) (!a+c) (!b+c) (a+b+!c) (c+!e) (d+!e) (!c+!d+e) (!a) (b) (d) (!e) = SAT (!a) (b) (d) (!e) = UNSAT

Debugging with max-sat Removing clauses satisfies the formula can only “remove” clauses corresponding to circuit constraints identify bugs in gates (not stimulus/response) max-sat algorithms do exactly that Requirements find all gate-level errors up to cardinality N must map clauses to gates develop concept of error cardinality

Debugging with max-sat Maximal Satisfiable Subset (MSS) generalization of max-sat (maximum cardinality) adding any extra clause makes formula UNSAT Minimal Correction Set (MCS) complement of MSS Can find all MCS of up to size k Can group clauses to approximate MCSes groups y1, y2,..., yn every clause C becomes C’= (y1 -> C)

Debugging with max-sat How to debug gates instead of clauses? define a group for each gate in circuit can only reason about satisfiablity at gate granularity How to get all solutions? block previous solutions and solve again cannot remove blocking clauses a b c d e 1 1 (!a+c+!yA) (!b+c+!yA) (a+b+!c+!yA) (c+!e+!yB) (d+!e+!yB) (!c+!d+e+!yB) (!a) (b) (d) (!e)

Debugging with max-sat What about sequential designs and multiple vectors? Time-frame expansion or Iterative Logic Array unfold/replicate combinational circuit in time Multiple stimulus/response are used to increase debugging resolution All clauses derived from same combinational gate are in same group

Replication Example a a a a 1 c c c c b e b b b e e e d 1 1 1 d d 1 d 2 a 3 1 c 1 c 2 c 3 c b e 1 2 b b b 3 e 1 e 2 e 3 d 1 1 1 d 1 d 1 2 d 1 3 1 (a+!c+!yA) (b+!c+!yA) (!a+!b+c+!yA) (c+!e+!yB) (d+!e+!yB) (!c+!d+e+!yB) (a1+!c1+!yA1) (b1+!c1+!yA1) (!a1+!b1+c1+!yA1) (c1+!e1+!yB1) (d1+!e1+!yB1) (!c1+!d1+e1+!yB1) (a2+!c2+!yA2) (b2+!c2+!yA2) (!a2+!b2+c2+!yA2) (c2+!e2+!yB2) (d2+!e2+!yB2) (!c2+!d2+e2+!yB2) (a3+!c3+!yA3) (b3+!c3+!yA3) (!a3+!b3+c3+!yA3) (c3+!e3+!yB3) (d3+!e3+!yB3) (!c3+!d3+e3+!yB3)

Replication Example a a a a 1 c c c c b e b b b e e e d 1 1 1 d d 1 d 2 a 3 1 c 1 c 2 c 3 c b e 1 2 b b b 3 e 1 e 2 e 3 d 1 1 1 d 1 d 1 2 d 1 3 1 (a+!c+!yA) (b+!c+!yA) (!a+!b+c+!yA) (c+!e+!yB) (d+!e+!yB) (!c+!d+e+!yB) (a1+!c1+!yA) (b1+!c1+!yA) (!a1+!b1+c1+!yA) (c1+!e1+!yB) (d1+!e1+!yB) (!c1+!d1+e1+!yB) (a2+!c2+!yA) (b2+!c2+!yA) (!a2+!b2+c2+!yA) (c2+!e2+!yB) (d2+!e2+!yB) (!c2+!d2+e2+!yB) (a3+!c3+!yA) (b3+!c3+!yA) (!a3+!b3+c3+!yA) (c3+!e3+!yB) (d3+!e3+!yB) (!c3+!d3+e3+!yB)

Debugging with approximate max-sat Can group clauses from single gate together single group represents a gate-based error Can group “same” gates from different vectors and different time frames together single group represents a gate-based error anywhere What about other types of groupings? based on modules/functions (higher level) based on topology arbitrarily

Debugging with approximate max-sat Example based on topology group 2 group 1 group 3 group 4

Debugging with approximate max-sat Empirical results show: max-sat is very efficient at finding approximate solutions the larger the groups the faster resolution vs. performance trade-off SAT-based debugger is good for exact solutions, not for approximate faster than max-sat for exact solutions Exploit the strengths of both approaches use approximate max-sat as preprocess/filter use SAT-based debugging on filtered problem

Debugging with approximate max-sat Two step debugging framework with max-sat divide and conquer approach approximate max-sat finds coarse solutions exact debugger refines solutions erroneous design exact error locations over-approximation of suspects approximate max-sat debugger exact debugger vectors/traces grouping

Experiments Circuits from ISCAS’85/89, ITC’99, opencores.org error inserted: random function change AND, OR, NAND, NOR, XOR swamped simulation to find error captured stimulus and correct trace 1GB memory limit, 1 hour timeout Clauses group by gate replication circuit topology max-sat solver from U of Michigan SAT-based debugger from U of Toronto

Experiments How many clauses or gates to group together bigger/fewer groups -> faster the max-sat portion, more suspects smaller/more groups -> few suspects, faster exact debugger Empirical results show groups of 20 gates are effective

Experiments sample experiments max-sat very fast on grouped problem exact debugger very fast on problem with eliminated suspects result: orders of magnitude faster circuit # error location debug only (s) max-sat (s) suspects eliminated exact debug(s) speed-up mot-comb3 16 357.67 0.27 98.32 % 0.47 483.34 ac97 4 [TO] 2.22 99.63 % 3.67 > 980 divider 1160.39 14.58 99.01 % 1.32 72.98 b14 45 1377.86 11.17 88.99 % 50.75 22.25 rsdecoder 11 572.68 0.67 98.95 % 0.65 433.85 s38584 35 [MO] 636.45 98.26% 376.62 > 3.59

Experiments Number of solved instances and runtime proposed framework against SAT-based debug only

Experiments Runtime comparison: proposed framework against SAT-based debug only

Conclusion SAT-based debugging is a natural max-sat problem formulate max-sat problem for circuits gate grouping replication grouping find all error suspects proposed framework combining strengths of max-sat and SAT-based debugging order of magnitude improvements achieved