一般化された確率伝搬法の数学的構造 東北大学大学院情報科学研究科 田中和之 kazu@smapip.is.tohoku.ac.jp http://www.smapip.is.tohoku.ac.jp/~kazu/ 16 December, 2008 DEX-SMI2008

Introduction Factor Graph and Loopy Belief Propagation F. R. Kschischang, B. J. Frey and H.-A. Loeliger: IEEE Transactions on information theory, vol. 47, no. 2, 2001. Generalized Belief Propagation J. S. Yedidia, W. T. Freeman and Y. Weiss: IEEE Transactions on Information Theory, vol.51, no.7, pp.2282-2312, 2005. Simplified Cluster Variation Method T. Morita: Physica A, vol.155, no.1, pp.73-83, 1989. Factor Graph による確率伝搬法の表現を一般化された確率伝搬法(Generalized Belief Propagation)とSimplified Cluster Variation Methodの定式化の立場から導出する． 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation Cluster: Set of nodes Every subcluster of the element of C does not belong to C. Example: System consisting of 4 nodes 1 2 3 4 1 2 3 4 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation KL Divergence Sum of all the Basic Clusters and their Sub-Clusters Example 1 2 3 4 1 2 3 4 Subclusters Basic Clusters 16 December, 2008 DEX-SMI2008

Selection of Basic Cluster in LBP and GBP (Bethe Approx.） 1 2 4 5 3 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 4 5 3 6 7 8 9 GBP (Square Approx. in CVM; Kikuchi Approx.) 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation V: すべての頂点の集合 : 3個の頂点からなるハイパエッジの集合 : C に属するハイパエッジの副クラスターの集合 B の要素はノードとエッジのみ : Bに属するのエッジの副クラスターの集合 A の要素はノードのみ 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation 5 3 4 1 2 5 1 2 3 4 5 3 4 1 2 1 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation 5 3 4 1 2 5 3 4 1 2 1 2 5 1 2 5 3 4 1 2 1 1 2 5 1 2 5 1 2 1 1 1 2 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation Marginal Probabilities are determined so as to minimize the Kikuchi Free Energy under some Reducibility Conditions. 5 1 2 3 4 + - 5 3 4 1 2 ~ = Kikuchi Free Energy 5 1 2 Reducibility Conditions 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation 5 1 2 3 4 5 1 2 4 3 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation 5 1 2 3 4 5 1 2 4 3 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation 5 1 2 4 3 5 1 2 4 3 5 1 2 4 3 5 1 2 4 3 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation 5 1 2 5 1 2 4 3 4 3 5 1 2 4 3 5 1 2 4 3 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation 5 1 2 5 1 2 4 3 4 3 5 1 2 5 1 2 4 3 4 3 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation 5 1 2 4 3 5 1 2 4 3 5 1 2 4 3 5 1 2 4 3 5 1 2 4 3 5 1 2 4 3 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation Marginal Probabilities are determined so as to minimize the Kikuchi Free Energy under some Reducibility Conditions. 5 1 2 3 4 + - 5 3 4 1 2 ~ = Kikuchi Free Energy 1 2 5 1 2 5 1 2 1 1 2 1 Reducibility Conditions 16 December, 2008 DEX-SMI2008

Simplified Cluster Variation Method Marginal Probabilities are determined so as to minimize the Kikuchi Free Energy under some Reducibility Conditions. 5 1 2 3 4 + - 5 3 4 1 2 ~ = 1 2 5 1 2 5 1 2 1 1 2 1 Reducibility Conditions 16 December, 2008 DEX-SMI2008

Simplified Cluster Variation Method 5 1 2 3 4 5 1 2 4 3 16 December, 2008 DEX-SMI2008

Simplified Cluster Variation Method 5 1 2 3 4 5 1 2 4 3 16 December, 2008 DEX-SMI2008

Simplified Cluster Variation Method 5 1 2 4 3 5 1 2 4 3 5 1 2 5 1 2 4 3 4 3 16 December, 2008 DEX-SMI2008

Simplified Cluster Variation Method 5 1 2 4 3 5 1 2 4 3 5 1 2 4 3 16 December, 2008 DEX-SMI2008

Simplified Cluster Variation Method and Factor Graph 5 3 4 1 2 5 3 4 1 2 5 1 2 3 4 5 3 4 1 2 1 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation 5 3 4 1 2 5 1 2 3 4 5 3 4 1 2 1 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation 5 3 4 1 2 5 1 2 3 4 5 3 4 1 2 1 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation Marginal Probabilities are determined so as to minimize the Kikuchi Free Energy under some Reducibility Conditions. 5 3 4 1 2 1 2 3 1 5 4 5 1 2 3 4 1 ~ + + + = - - - - 5 1 1 2 4 1 3 1 + 1 Kikuchi Free Energy 5 1 2 Reducibility Conditions 16 December, 2008 DEX-SMI2008

Generalized Belief Propagation Marginal probabilities are determined so as to minimize the Kikuchi Free Energy under some Reducibility Conditions. 5 3 4 1 2 1 2 3 1 5 4 5 1 2 3 4 1 ~ + + + = 1 2 3 4 5 Kikuchi Free Energy 5 1 2 5 1 2 Reducibility Conditions 16 December, 2008 DEX-SMI2008

Simplified Cluster Variation Method 5 2 5 1 2 2 1 5 4 1 1 4 3 1 3 4 3 16 December, 2008 DEX-SMI2008

Simplified Cluster Variation Method 5 1 2 4 3 5 1 2 4 3 5 1 2 4 3 16 December, 2008 DEX-SMI2008

Simplified Cluster Variation Method and Factor Graph 5 3 4 1 2 5 3 4 1 2 5 1 2 3 4 5 3 4 1 2 16 December, 2008 DEX-SMI2008

Selection of Basic Clusters and their Sub-Clusters in Generalized Belief Propagation on Square Grid The set of Basic Clusters The Set of Basic Clusters and Their Sub-Clusters LBP (Bethe Approx.) 16 December, 2008 DEX-SMI2008

(Square Approximation in CVM） Selection of Basic Clusters and their Sub-Clusters in Generalized Belief Propagation on Square Grid The Set of Basic Clusters and Their Sub-Clusters The set of Basic Clusters GBP (Square Approximation in CVM） 16 December, 2008 DEX-SMI2008

Critical Points for Ising Model (V,E)=Square Lattice (V,E)=Cube Lattice TC/J Exact (Onsager) 2.2692 GBP 2.4257 SCVM 2.6253 Square Cactus CVM 2.7708 Bethe Approx. 2.8854 Factor Graph 3.4659 Mean Field Approx. 4.0000 TC/J High Temperature Series 4.5103 GBP 4.6097 SCVM 4.7611 Cube Cactus CVM 4.8396 Bethe Approx. 4.9326 Mean Field Approx. 6.0 Factor Graph 8.6181 16 December, 2008 DEX-SMI2008

Summary We reviewed the generalized belief propagation based on cluster variation method. Some extensions of the generalized belief propagation have been given by using the simplified cluster variation method. The relationship between the factor graph and the present extensions by means of the simplified cluster variation method are clarified. 16 December, 2008 DEX-SMI2008