1. ベーテ自由エネルギーに対するCCCPアルゴリズムの拡張
東京工業大学総合理工学研究科
知能システム科学専攻　渡辺研究室　
　　西山　悠，　渡辺澄夫

2. Background Probabilistic information processing
Inference for Bayesian networks
Error correcting codes
000101
noise
000111
decode
CDMA multi-user detection
Probabilistic image processing
degrade image
restored image
Task
High-dimensional distribution
Marginals
Intractable

3. Belief propagation (Sum product algorithm)
Task
Intractable
Belief propagation (Sum product algorithm)
Bethe free energy
Optimization techniques
BO (M. Welling, et. al., 2001)
CCCP (A. L. Yuille, 2002)
SEQ (Tonosaki, et. al., 2007)
(Approximate) marginals

4. Performance of CCCP CCCP
guarantees to monotonically decrease Bethe free energy unlike belief propagation.
requires huge computational cost compared to belief propagation.
does not always converge for synchronous update of inner loop.

5. Purpose
We extend CCCP algorithm for Bethe free energy and present a new CCCP (NCCCP) algorithm.
New CCCP (NCCCP) algorithm
includes conventional CCCP.
guarantees to monotonically decrease Bethe free energy.
is more stable even for synchronous inner loop.
can reduce huge computational cost underlying CCCP.

6. Concave and convex procedure(CCCP)
CCCP algorithm guarantees to monotonically decrease the function(al)
Convex
Concave
by the update rule
Extremum

7. Bethe free energy Bethe free energy Concave Convex Conventional CCCP
Approximate marginals

8. Main result (Key idea) NCCCP Bethe free energy an arbitrary convex
functional
Trivial pair creation

9. NCCCP
Particularly,
CCCP
Free parameters

10. NCCCP algorithm CCCP algorithm Outer loop Outer loop Inner loop
Approximate marginals
Approximate marginals

11. NCCCP algorithm for Bethe free energy
Theorem (Outer loop)
Outer loop of NCCCP algorithm for minimizing Bethe free energy is given by as follows:
where satisfies for all

12. NCCCP algorithm for Bethe free energy
Theorem (Inner loop)
Inner loop of NCCCP is given by as follows:

13. Update Manner of Inner Loop
Theorem (Inner loop)
Asynchronous update
Inner loop of NCCCP is given by as follows:
time
time
time
CCCP guarantees to converge.
Synchronous update
time
time
time
CCCP does not always converge.
NCCCP converges.

14. Role of free parameters
When are small　　　
When are large　　　
Outer loop
Outer loop
fast
slow
Outer loop or
fast
convergence
Outer loop
Outer loop
slow
fast
Outer loop or
Outer loop
convergence
fast
Outer loop
Approximate marginals
Approximate marginals
There exist optimal values

15. Numerical Results(1/2)
(i) Asynchronous inner loop
CCCP
CCCP

16. Numerical Results (2/2)
(ii) Synchronous inner loop
CCCP
CCCP

17. Conclusion
We presented a new CCCP (NCCCP) algorithm for Bethe free energy.
New CCCP (NCCCP) algorithm
includes conventional CCCP.
guarantees to monotonically decrease Bethe free energy.
is more stable even for synchronous inner loop.
can reduce huge computational cost underlying CCCP.

18. Future works
To design efficient NCCCP algorithm based on the optimality of free parameters.
To apply NCCCP algorithm to practical problems such as CDMA multi-user detection problems or decoding algorithm for LDPC codes.
1.外崎幸徳，樺島祥介, “CCCPに基づくCDMAマルチユーザ検出アルゴリズム,” 電子情報通信学会論文誌 D, vol. J89-D, no. 5, pp , 2006.
2.T. Shibuya, K. Harada, R. Tohyama, and K. Sakaniwa, “Iterative Decoding Based on the Concave-Convex Procedure,” IEICE Trans. Fundam. Electron. Commu. Comput. Sci., vol. E88-A, no. 5, pp , 2005.

19. Example: Gaussian Distributions
NCCCP algorithm= Outer loop + Inner loop
Theorem (Outer loop in Gaussian distributions)
Theorem (Inner loop in Gaussian distributions)