1. Technion Haifa Research Labs Israel Institute of Technology Underapproximation for Model-Checking Based on Random Cryptographic Constructions Arie Matsliah (presenting) and Ofer Strichman

2. IBM / Technion 2 Introduction Motivation:  Efficient “bug-hunters” for heavy verification instances Underapproximation:  M, M’ – Kripke structures  M’ underapproximates M if for every LTL formula φ: M φ → M’ φ  M’ has a subset of the behaviors of M Our goal:  Automatic and efficient underapproximation-based model checking

3. IBM / Technion 3 Model-checking with underapproximation  Potentially good for falsification, not verification. M’ Model- checker M’ φ ? fail φ M Refine: add behaviors

4. IBM / Technion 4  The time complexity of model checking depends exponentially on the number of inputs Natural approach for Underapproximation: reduce # of inputs. What makes Model Checking hard? M’ inputs outputs M inputs outputs

5. IBM / Technion 5 Reducing the number of inputs  An underlying assumption: “The values of some of the inputs are immaterial for exposing the bug”  A simple technique for underapproximation: fixing inputs.  Pick those inputs manually (using high-level information).  Fix their value.  A similar process which is automatic and complete is ineffective.  Our method: reduce # inputs without fixing any.

6. IBM / Technion 6 Our contribution Underapproximation which:  Reduces the number of inputs  Maintains a measurable and uniform degree of freedom to the original inputs  Based on adding circuitry to the model.  Can be applied to any form of verification M original inputs outputs M’ C new inputs inputs outputs

7. IBM / Technion 7 Main idea - Universality  A (combinatorial) circuit C is k-universal if  any valuation of at most k of its outputs... ...can be reached under some assignment to its inputs.  Example: 2-universal circuit inputs outputs 00 0 0 0 10 1 0 1 01 0 1 1 11 1 1 0  Why universality?  if #(important inputs) ≤ k, then k-universal circuit is enough inputs outputs C

8. IBM / Technion 8 Universality of some naïve methods Fixing some of the inputs to constants  0-universal M’ M inputs outputs 01100110 Merge groups of inputs together  1-universal M’ M inputs outputs C C

9. IBM / Technion 9 Inspiration - Pseudo Random Generators (PRGs) Generator random string pseudorandom string looks random for any poly-time algorithm f f f f f f f PRG construction [NW 94]: -the circuit has certain properties -f is “hard to invert” Our construction: -the circuit is random -f is a XOR function

10. IBM / Technion 10 Using universal circuits M original inputs outputs M’ C new inputs

11. IBM / Technion 11 Constructing universal circuits 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 outputs (inputs of M) inputs (inputs of M’) o1o1 o2o2 o3o3 o4o4 o5o5 o6o6 o7o7 i1i1 i2i2 i3i3 i4i4 i5i5 i6i6 i1i1 i2i2 i3i3 i4i4 i5i5 i6i6 o1o1 o2o2 o3o3 o4o4 o5o5 o6o6 o7o7 C A random matrix mod 2

12. IBM / Technion 12 How universal is C?  Lemma: if every k rows in A are linearly independent – C is k-universal  Proof (for k=3, n=7, m=6): 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 i1i1 i2i2 i3i3 i4i4 i5i5 i6i6 o1o1 o2o2 o3o3 o4o4 o5o5 o6o6 o7o7 A 1 1 1 1 1 1 1 11 i1i1 i2i2 i3i3 i4i4 i5i5 i6i6 o2o2 o4o4 o7o7 A’ A’ has full rank  all 2 3 values covered

13. IBM / Technion 13 How universal is C?  Lemma: for k=O(m/log n), with high probability, every k rows in A are linearly independent  Proof (for k=3, n=7, m=6): 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 i1i1 i2i2 i3i3 i4i4 i5i5 i6i6 o1o1 o2o2 o3o3 o4o4 o5o5 o6o6 o7o7 A 1 1 1 1 1 1 1 1 1 i1i1 i2i2 i3i3 i4i4 i5i5 i6i6 o1o1 o4o4 o6o6 A’ Pr[A 1 is in span(A 4,A 6 )] ≤ 2 2 /2 6 for general k,m,n: Pr[ … ] ≤ 2 -m+k-1  Apply Union Bound A1A1 A4A4 A6A6

14. IBM / Technion 14 How universal is C?  Lemma: for k=O(m/log n), with high probability, every k rows in A are linearly independent  Lemma: if every k rows in A are linearly independent – C is k-universal  Corollary: for k=O(m/log n), with high probability, C is k-universal Sample values:

15. IBM / Technion 15 Better bounds for k  What if we relax the requirement?  Lemma: for any ε > 0 and k ≤ m - log m – log (1/ε), each subset of k outputs is covered with probability 1-ε  for any k ≤ m - log m – 7, each subset of k outputs is covered with probability ~0.99 Sample values: k cannot be larger than m m 20 30 40 50 70 100 200 500 800 1000 k 7 18 28 37 57 86 185 484 783 983

16. IBM / Technion 16 What now?...  The main contribution of the work is theoretical:  Suggesting the relevance of PRG to model-checking  Proving universality properties of such circuits.  Expected difficulties in achieving results:  BDD-based model-checking is mostly ‘out’.  SAT does not distinguish between inputs and other variables  Hence, we can only guarantee an improvement in the worst-case.  Simulation: Typically start from a constrained environment in which only a small part of the inputs is allowed. Combining with PRG does not seem to be simple.  Nevertheless…  The main purpose of the experiments is to examine the effect of universality.

17. IBM / Technion 17 What now?...  The main contribution of the work is theoretical:  Showing relevance of universality to model-checking.  Proving universality properties of PRG-like circuits.  Experiments show that indeed universality matters.  The challenge: from theory to practice.

18. IBM / Technion 18 Experiments  Implemented in IBM RuleBase PE  17 BMC instances with known bugs  For each design with n inputs, we generated a new design with m inputs, for m = n/2, n/3, n/5, n/10  We compared the following methods:  Our: Our circuit with m inputs.  Orig: No underapproximation  Fix: Fixing n-m inputs to some constant.  Set: Partitioning the inputs to m sets. All inputs in the same set are mapped to a single input.

19. IBM / Technion 19 OrigOurFix Designinputs (n)n n/2 n/3 n/5 n/10n/2n/3 n/5 n/10 IBM#1459666636663246 - - - IBM#276173149767268 - - - - IBM#3761911277779 -373 - - - IBM#485211170121105140191317 - - IBM#568616520592 - - - - - IBM#668735914661 - - - - - IBM#7684823084652 - - - - - IBM#8681221521690 - - - - - IBM#964210119151966165412081693 - - - IBM#10801270139218301137 - - - - - IBM#11832640236422541845 - - - - - IBM#12682017191 - - - - - - - IBM#1360942453432351 -1206 - - - IBM#14218965735778510396 - - - - IBM#15521206 - - - - - - - - IBM#16157953 - - - - - - - - IBM#176821503 TO - - - - Run-times -13.6% -17.5% -22.7% -47.1% 4.7% 50.2%

20. IBM / Technion 20 Orig Our Set Designinputs (n)n n/2 n/3 n/5 n/10n/2n/3 n/5 n/10 IBM#1459666636663223229227231 IBM#276173149767268361446 - - IBM#3761911277779 -168317 - - IBM#485211170121105140306289405 - IBM#568616520592 -410 - - - IBM#668735914661 - - - - - IBM#7684823084652 -561491 - - IBM#8681221521690 -113 - - - IBM#964210119151966165412082150 - - - IBM#10801270139218301137 - - - - - IBM#11832640236422541845 - - - - - IBM#12682017191 - - - - - - - IBM#1360942453432351 -413407 - - IBM#142189657357785103969691102 - - IBM#15521206 - - - ----- IBM#16157953 - - - ----- IBM#176821503 TO --- Run-times -13.6% -17.5% -22.7% -47.1% 6.2% 7.2% 105.9% 140.6%

21. IBM / Technion 21 The effect of m and p  Tested 4 heaviest designs with various m and p’s  Depth in which bug was found, was increased in this many designs: 1/21/31/51/10 n/20000 n/30000 n/50000 n/100001 m p inputs probability of each input to be included in the fanin

22. IBM / Technion 22 Future work 1.Attach the circuit C to the unrolled model 2.Refinement strategies 3.Construct universal circuits without XORs 4.Construct universal circuits deterministically 5.Experiments with (unbounded) model-checking + simulation M0M0 C M1M1 M2M2 MkMk

23. IBM / Technion 23 Thank you!

24. IBM / Technion 24 Naïve Under-Approximation 1 Restrict some of the inputs to constants M’ M inputs outputs 01100110 ? fully automatic process + M’ is easier to check than M ? M’ captures the “interesting” behaviors of M “important inputs”

25. IBM / Technion 25 Naïve Under-Approximation 2 Merge groups of inputs together M’ M inputs outputs ? fully automatic process + M’ is easier to check than M ? M’ captures the “interesting” behaviors of M “important inputs”

26. IBM / Technion 26 Our goal Uniformly free inputs M’ M inputs outputs for every subset of k inputs, all 2 k assignments are achievable  if #(interesting inputs) ≤ k + fully automatic process + M’ captures the “interesting” behaviors of M ?

27. IBM / Technion 27 Pseudo Random Generators Generator random string pseudorandom string looks random for any poly-time algorithm input output random string BPP algorithm