KITPC Osamu Watanabe Tokyo Inst. of Tech. Finding Most-Likely Solution of the Perturbed k -Linear-Equation Problem k -Linear-Equation = k LIN 渡辺 治 東京工業大学

Preliminaries: Problem & Algorithm Most Likely 3LIN Instance: system of 3-linear equations F generated following 3LIN(n,p,q) Ans: an assignment a = (a1,...,an) that is most likely for F ⇔ minimizing # of broken eqn.s MAX-3LIN MAX-3XSAT

Computational Learning Data Mining Inverse Problem Observed Data reasoning solution randomness e.g., noise s3s3 s1s1 s2s2 s4s4 Problem of Finding Most Likely Solution

Input: x Task: Find a most likely solution s for x Inverse Problem for Computational Learning ← observed data Most Likely Solution Problem max arg Pr [ g ( s, r ) = x ] r s Complexity and Algorithms 1. Optimization problem often NP-hard in the worst case 2.Interesting algorithms are known that works well on average ! x = g ( s, r ) with r : D good scenario

Target Problems System of Linear Equations (mod 2) Instance: system of linear equations F generated randomly under a certain scenario Ans: an assignment for var.s of F that is most likely for the scenario Ex. ・ linear code decoding ・ bisection ・ solving perturbed eqn.s ↑ many applications

random generation model: 3LIN(n,p,q) Generate a eqn. system F for n, p, and q 1. randomly generate a target solution a* = (a1,...,an) ; 2. for each 3-tuple of variables, select it with prob. p and make an eqn. consistent to a* ; 3. flip the righthand side value b of the eqn. with prob. q and then add it to F ; eqn. density noise prob.

3LIN(n,p,q) n : # of variables p : eqn. density param. q : noise prob. some remarks ・ fix our planted solution a* = (a1,...,an) to (+1,...,+1) ・ average # of eqn.s is sat. threshold θ( n ) equations ↑ p = c / n 2

Proposition For some constants c1 and c2, we have ⇒ the planted solution is the unique opt. solution for ML-3LIN instances w.h.p. w.h.p. = prob. 1 － o (1)

Algorithm: simple message passing algo. m = +1 f 1→1 m = 0 f 3→1

problem: Most Likely 3LIN F most likely → a = (a1,...,an) instance generation model: 3LIN( n p q ) noise prob. eqn. density planted solution a* = (+1,..., +1) Summary algHMP( I, MAXSTEP ) 1. Most Likely 2LIN ・ p = c / n for large c. ・ q = 1/2 － ε. ・ I = any one var. 2. Most Likely 3LIN ・ p = c / n log n for large c. ・ q = 1/2 － ε. ・ I = any log n var.s Experiments 平均式数 cn 変数毎登場回数 c 平均式数 cn / log n 変数毎登場回数 cn / log n 2

１． Most Likely 2LIN Problem Algorithm power method

p ・ (1 － q) p・qp・q A = 総和は平均で n ・ p ・ (1 － 2q) = p ・ 2 ε 平均で最大固有値は 2 ε np q = ½ － ε

最大固有ベクトル 1 未満の定数

Most Likely 2LIN Problem Algorithm

３． Most Likely ３ LIN Problem Algorithm

３． Most Likely ３ LIN Problem Algorithm log n correct values 平均式数 cn 変数毎登場回数 c 平均式数 cn / log n 変数毎登場回数 cn / log n 2

Theorem p > c / n log n and q < 1/2 － ε ⇒ the modified algorithm works w.h.p. p > c / n log n too big!? But ．．． O.W, M.O, & A.Coja-Oghlan

Simple Case Analysis 1. Consider the case q = 0; that is, no perturbation occurs. 2. Execution of the algorithm with MAXSTEP = n and I = all possible { xi, xj } and the correct initial assignment to { xi, xj } When the execution succeeds w.h.p. ? Success threshold for p ? nonzero message is sent to all var.s

Propagation Connectivity new hypergraph connectivity 3LIN instance F Hypergraph H = ( V, HE ) 3LIN( n, p, q ) ⇔ H( n, p ) marking process from { 1, 2 }

Propagation Connectivity - connectivity notion on hypergraphs 3-uniform hyp.graphs marking process from (1,2) Def. hyp.graph H is propagation connected ⇔ ∃ starting vertex pair s.t. the above process marks all vertices from the starting pair

H ( n, p ) : the standard distribution of 3-uniform random hyper graphs where n = # of vertices, and p = hyp.edge prob. whp = with. prob. 1 － o (1) c n ln n = Thm. Let H denote a random graph following the distribution H ( n, p ) for p. Lower Bound: c < 0.16 ⇒ H is not prop. connected whp Upper Bound: c > 0.25 ⇒ H is prop. connected whp. c n ln n =

Proof Outline Lower Bound: c < 0.16 Lemma. For any p = for any c < 0.16, consider the propagation process on n vertices from any fixed vertex pair ( v 1, v 2 ). Then for some d > 0, we have Pr[ ] = o ( n ) －2－2 T ( v 1, v 2 ) > d ln n c n ln n

Proof Outline Upper Bound: c > 0.25 Def. ( v 1, v 2 ) is good ⇔ < T ( v 1, v 2 ) ( ln n ) 3 Lemma. For any p = for any c > 0, consider the propagation process on n vertices from any fixed vertex pair ( v 1, v 2 ). Pr[ < T ( v 1, v 2 ) < n － 1 ] = n ( ln n ) 3 － Ω(1) c n ln n

Proof Outline Upper Bound: c > 0.25 N = # of good starting pairs of vertices Lemma B. If E[ N ] > n, then we have Pr[ N > 0 ] > 1 － o (1). δ V[ N ] is small because two processes from different starting pairs are not so much related during steps. ( ln n ) 3 Lemma A. For any p = for any c > 0.25 consider the propagation process on n vertices. Then for some δ > 0, we have E[ N ] > n. δ c n ln n

some remarks ・ fix our planted solution a* = (a1,...,an) to (+1,...,+1) ・ average # of eqn.s is 3LIN(n,p,q) n : # of variables p : eqn. density param. q : noise prob. Summary [results] Algorithm's success threshold seems

3LIN(n,p,q) n : # of variables p : eqn. density param. q : noise prob. some remarks ・ fix our planted solution a* = (a1,...,an) to (+1,...,+1) ・ average # of eqn.s is Discussions [results] Algorithm's success threshold seems random walk based algorithm, also [R.Montegegro] [Feige-Ofek '05] One step G.elimination + spectral algo. for 2LIN sat. threshold θ( n ) equations ↑ p = c / n 2

References A. Coja-Oghlan, M. Onsjo, and O. Watanabe, Propagation connectivity of random hypergraphs, in Proc. 13th APPROX 2010 and 14th RANDOM 2010, LNCS 6302, , U. Feige and E. Ofek, Spectral techniques applied to sparse random graphs, Random Structures and Algorithms 27, , M. Onsjo and O. Watanabe, A simple message passing algorithm for graph partition problem, in Proc. 17th ISAAC'06, LNCS 4288, , 2006.