1. 一般化された確率伝搬法の数学的構造 東北大学大学院情報科学研究科 田中和之 kazu@smapip.is.tohoku.ac.jp
16 December, 2008
DEX-SMI2008

2. Introduction Factor Graph and Loopy Belief Propagation
F. R. Kschischang, B. J. Frey and H.-A. Loeliger: IEEE Transactions on information theory, vol. 47, no. 2, 2001.
Generalized Belief Propagation
J. S. Yedidia, W. T. Freeman and Y. Weiss: IEEE Transactions on Information Theory, vol.51, no.7, pp , 2005.
Simplified Cluster Variation Method
T. Morita: Physica A, vol.155, no.1, pp.73-83, 1989.
Factor Graph による確率伝搬法の表現を一般化された確率伝搬法(Generalized Belief Propagation)とSimplified Cluster Variation Methodの定式化の立場から導出する．
16 December, 2008
DEX-SMI2008

3. Generalized Belief Propagation
Cluster: Set of nodes
Every subcluster of the element of C does not belong to C.
Example: System consisting
of 4 nodes 1 2 3 4 1 2 3 4
16 December, 2008
DEX-SMI2008

4. Generalized Belief Propagation
KL Divergence
Sum of all the Basic Clusters and their Sub-Clusters
Example 1 2 3 4 1 2 3 4
Subclusters
Basic Clusters
16 December, 2008
DEX-SMI2008

5. Selection of Basic Cluster in LBP and GBP
(Bethe Approx.） 1 2 4 5 3 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 4 5 3 6 7 8 9
GBP
(Square Approx. in CVM;
Kikuchi Approx.)
16 December, 2008
DEX-SMI2008

6. Generalized Belief Propagation
V: すべての頂点の集合
: 3個の頂点からなるハイパエッジの集合
: C に属するハイパエッジの副クラスターの集合
B の要素はノードとエッジのみ
: Bに属するのエッジの副クラスターの集合
A の要素はノードのみ
16 December, 2008
DEX-SMI2008

7. Generalized Belief Propagation5 3 4 1 2 5 1 2 3 4 5 3 4 1 2 1
16 December, 2008
DEX-SMI2008

8. Generalized Belief Propagation5 3 4 1 2 5 3 4 1 2 1 2 5 1 2 5 3 4 1 2 1 1 2 5 1 2 5 1 2 1 1 1 2
16 December, 2008
DEX-SMI2008

9. Generalized Belief Propagation
Marginal Probabilities are determined so as to minimize
the Kikuchi Free Energy under some Reducibility Conditions. 5 1 2 3 4 + - 5 3 4 1 2 ~ =
Kikuchi Free Energy 5 1 2
Reducibility Conditions
16 December, 2008
DEX-SMI2008

10. Generalized Belief Propagation5 1 2 3 4 5 1 2 4 3
16 December, 2008
DEX-SMI2008

11. Generalized Belief Propagation5 1 2 3 4 5 1 2 4 3
16 December, 2008
DEX-SMI2008

12. Generalized Belief Propagation5 1 2 4 3 5 1 2 4 3 5 1 2 4 3 5 1 2 4 3
16 December, 2008
DEX-SMI2008

13. Generalized Belief Propagation5 1 2 5 1 2 4 3 4 3 5 1 2 4 3 5 1 2 4 3
16 December, 2008
DEX-SMI2008

14. Generalized Belief Propagation5 1 2 5 1 2 4 3 4 3 5 1 2 5 1 2 4 3 4 3
16 December, 2008
DEX-SMI2008

15. Generalized Belief Propagation5 1 2 4 3 5 1 2 4 3 5 1 2 4 3 5 1 2 4 3 5 1 2 4 3 5 1 2 4 3
16 December, 2008
DEX-SMI2008

16. Generalized Belief Propagation
Marginal Probabilities are determined so as to minimize
the Kikuchi Free Energy under some Reducibility Conditions. 5 1 2 3 4 + - 5 3 4 1 2 ~ =
Kikuchi Free Energy 1 2 5 1 2 5 1 2 1 1 2 1
Reducibility Conditions
16 December, 2008
DEX-SMI2008

17. Simplified Cluster Variation Method
Marginal Probabilities are determined so as to minimize
the Kikuchi Free Energy under some Reducibility Conditions. 5 1 2 3 4 + - 5 3 4 1 2 ~ = 1 2 5 1 2 5 1 2 1 1 2 1
Reducibility Conditions
16 December, 2008
DEX-SMI2008

18. Simplified Cluster Variation Method5 1 2 3 4 5 1 2 4 3
16 December, 2008
DEX-SMI2008

19. Simplified Cluster Variation Method5 1 2 3 4 5 1 2 4 3
16 December, 2008
DEX-SMI2008

20. Simplified Cluster Variation Method5 1 2 4 3 5 1 2 4 3 5 1 2 5 1 2 4 3 4 3
16 December, 2008
DEX-SMI2008

21. Simplified Cluster Variation Method5 1 2 4 3 5 1 2 4 3 5 1 2 4 3
16 December, 2008
DEX-SMI2008

22. Simplified Cluster Variation Method and Factor Graph5 3 4 1 2 5 3 4 1 2 5 1 2 3 4 5 3 4 1 2 1
16 December, 2008
DEX-SMI2008

23. Generalized Belief Propagation5 3 4 1 2 5 1 2 3 4 5 3 4 1 2 1
16 December, 2008
DEX-SMI2008

24. Generalized Belief Propagation5 3 4 1 2 5 1 2 3 4 5 3 4 1 2 1
16 December, 2008
DEX-SMI2008

25. Generalized Belief Propagation
Marginal Probabilities are determined so as to minimize
the Kikuchi Free Energy under some Reducibility Conditions. 5 3 4 1 2 1 2 3 1 5 4 5 1 2 3 4 1 ~ + + + = - - - - 5 1 1 2 4 1 3 1 + 1
Kikuchi Free Energy 5 1 2
Reducibility Conditions
16 December, 2008
DEX-SMI2008

26. Generalized Belief Propagation
Marginal probabilities are determined so as to minimize
the Kikuchi Free Energy under some Reducibility Conditions. 5 3 4 1 2 1 2 3 1 5 4 5 1 2 3 4 1 ~ + + + = 1 2 3 4 5
Kikuchi Free Energy 5 1 2 5 1 2
Reducibility Conditions
16 December, 2008
DEX-SMI2008

27. Simplified Cluster Variation Method5 2 5 1 2 2 1 5 4 1 1 4 3 1 3 4 3
16 December, 2008
DEX-SMI2008

28. Simplified Cluster Variation Method5 1 2 4 3 5 1 2 4 3 5 1 2 4 3
16 December, 2008
DEX-SMI2008

29. Simplified Cluster Variation Method and Factor Graph5 3 4 1 2 5 3 4 1 2 5 1 2 3 4 5 3 4 1 2
16 December, 2008
DEX-SMI2008

30. Selection of Basic Clusters and their Sub-Clusters in Generalized Belief Propagation on Square Grid
The set of Basic Clusters
The Set of Basic Clusters and Their Sub-Clusters
LBP
(Bethe Approx.)
16 December, 2008
DEX-SMI2008

31. (Square Approximation in CVM）
Selection of Basic Clusters and their Sub-Clusters in Generalized Belief Propagation on Square Grid
The Set of Basic Clusters and Their Sub-Clusters
The set of Basic Clusters
GBP
(Square Approximation in CVM）
16 December, 2008
DEX-SMI2008

32. Critical Points for Ising Model
(V,E)=Square Lattice
(V,E)=Cube Lattice
TC/J
Exact (Onsager)
2.2692
GBP
2.4257
SCVM
2.6253
Square Cactus CVM
2.7708
Bethe Approx.
2.8854
Factor Graph
3.4659
Mean Field Approx.
4.0000
TC/J
High Temperature Series
4.5103
GBP
4.6097
SCVM
4.7611
Cube Cactus CVM
4.8396
Bethe Approx.
4.9326
Mean Field Approx.
6.0
Factor Graph
8.6181
16 December, 2008
DEX-SMI2008

33. Summary
We reviewed the generalized belief propagation based on cluster variation method.
Some extensions of the generalized belief propagation have been given by using the simplified cluster variation method.
The relationship between the factor graph and the present extensions by means of the simplified cluster variation method are clarified.
16 December, 2008
DEX-SMI2008