1 Acceleration of Inductive Inference of Causal Diagrams Olexandr S. Balabanov Institute of Software Systems of NAS of Ukraine

2 Generic Task: Our Goal : to speed up a model induction We obey a constraint-based approach to model induction Statistical Data (Sample) Structure of Data Generation Process (Causal Model) No prior knowledge No Temporal Order of Variables

3 3 Phases of Causal Inference Calculating parameter values Structure of model Model skeleton identification via searching for separators Separators Edges Edge orientation Data

4 A constraint-based algorithm deletes an edge X — Y when finds a fact that variables X and Y are conditionally independent under some condition. The algorithm tries to find a separator for each pair of variables. The key idea of PC-algorithm is: to include in a tentative separator for pair (X, Y) only those variables which are supposedly adjacent to X or to Y.

5 But still a task of searching for separators remains computationally very expensive even for networks of moderate density. The worst situation: When there edge X — Y exists, the PC continues an attempts to find a separator for ( X,Y). The algorithm would examine all subsets of Adj(X) and all subsets of Adj(Y) as tentative separators.

6 It is especiall desirable to recognize the edge presence as early as possible. Also very useful is to come with tests of low rank whenever possible. This means to find minimal separators. Idea to achieve the goal Idea to achieve the goal – to exploit pairwise Markov properties of ADG-model, concept of locally-minimal separator and their logical consequences.

7 We have developed several rules of inductive inference acceleration. These rules perform: 1) recognition of edge presence; 2) recognition of edge absence; 3) deleting some variables from list of candidates in supposed separator; 4) recognition of some variables as obligate members of respective separator (if it exists at all).

8 Rule of ‘placing aside’ : If there Ds(Z;X;Y) &  Ds(Z;;Y) holds in model G, then vertex Z is not a member of any locally-minimal d-separator for pair (X,Y) in G. One of the most effective rules:

9 If we equip an algorithm (like PC) with just the two rules (placing aside rule and “lack of separator’s pivot” rule), then the algorithm would recover a forest (or poly-forest) by executing tests of zero- and first-rank. In particular, algorithm Razor-1.1 would identify a forest, presented below, by tests of 0-rank and 1-rank only. Basic PC algorithm for the same model would work out test up to 9-rank.

10 Algorithm Razor-1.1 (or even simpler one, but with the two rules) requires tests of first rank at max. PC algorithm requires tests of 8-rank at max

11 More complicated and realistic example. This structure consists of 15 vertices and 30 edges. Razor-1.1 requires tests of 4th rank at max. PC algorithm requires tests of 8th rank at max

12 Below – r esults of inference from data samples. ADG structures were generated randomly for 20 vertices (variables) and number of edges = 40 – 70. Variables – binary and ternary. Model’s parameters – also randomly generated. Sample size =

13 Experimental results: Performance PC Razor 20 vertices (variables), 50 edges.

14 Inference Errors PC Razor PC Razor 20 vertices (variables), 50 edges. Notice : These results present uncomfortable cases (with binary and ternary variables and random parameters).

15 As demonstrated, algorithm equipped with proposed rules performs learning Bayesian nets (of moderate density) multiple times faster then PC algorithm. At the same time, number of errors grows much more slowly. Thus inductive inference acceleration rules facilitate fast identification of skeleton of causal model. Most of the rules of inductive inference acceleration may be extended to the case of causal diagrams with latent variables (some corrections to the algorithm should be done). Algorithm needs to be upgraded for the case of causally- insufficient models. Extension Conclusion

16 Thanks for attention Balabanov A.S. Minimal separators in dependency structures: Properties and identification. Cybernetics and Systems Analysis. – Vol. 44, – No 6, 2008, – P.803–815. – Springer N.Y. Balabanov A. S. Construction of minimal d-separators in a dependency system. Cybernetics and Systems Analysis. – Vol. 45, – No 5, –2009. – P. 703–713. Balabanov O. S. Accelerating algorithms for Bayesian network recovery. Adaptation to structures without cycles (in Ukrainian). Problems in programming journal, – – No 1. – P.63–69. – Kiev, Ukraine, ISBN Balabanov O.S., O.S. Gapyeyev, A.M. Gupal, S.S. Rzhepetskyy. Fast algorithm for learning Bayesian networks from data. Journal of Automation and Information Sciences. – Vol. 43, – No 10, – 2011, to appear. Recent publications