Algorithms and Complexity Lecture 9 Randomized Algorithms (and more)

Randomized Algorithms Algorithm has access to a (hypothetical) subroutine outputting a random bit. Performance measures of algorithm (running time, output quality) now are random variables: Still interested in the worst-case instance! For this instance we want guarantees on distributions. Why?! Often simpler, faster, provably better algo’s. Error probability often negligible.

Example 1 (a) Zero-Polynomial: Given a polynomial 𝑝(𝑥) of degree 𝑑 in formula form, is it the zero polynomial? (𝑥 2 −𝑥)4+2( 𝑥 3 (3+𝑥+ 𝑥 2 )) is a no instance (𝑥 2 −𝑥)4+4(𝑥− 𝑥 2 ) is a yes instance How to solve Zero-Polynomial quickly? Expanding parentheses may require exponential time! Algo: 1) Pick 𝑥 ∈ 𝑅 {1,…,2𝑑} uniformly at random; 2) Return YES iff 𝑝 𝑥 =0. If yes instance, Pr 𝑌𝑒𝑠 =1; if not, Pr 𝑁𝑜 ≥1/2

Example 1 (b) Thm: Let 𝑝(𝑥) be a non-zero (univariate) polynomial of degree at most 𝑑 over any field 𝔽. If 𝑥 ∈ 𝑅 𝔽, then Pr 𝑝 𝑥 =0 ≤𝑑/|𝔽|. Alice has string a∈{0,1 } 𝑛 , Bob has b∈{0,1 } 𝑛 . Goal: verify a=𝑏, with little communication. In deterministic setting, 𝑛 bits needed! Algo: 1) A field 𝔽 of size ~2𝑛 is fixed (a priori) 2) Alice picks 𝑥 ∈ 𝑅 𝔽; sends 𝑥 and a 𝑥 ≔ 𝑖=1 𝑛 𝑎 𝑖 𝑥 𝑖 3) Bob claims equality if 𝑖=1 𝑛 𝑏 𝑖 𝑥 𝑖 −𝑎 𝑥 =0 Alice and Bob use 𝑂( log 𝑛 ) bits of communication If equal, always correct. If not, Pr 𝑐𝑜𝑟𝑟 ≥~1/2

Example 1 (c) Thm: Let 𝑝(𝑥) be a non-zero (univariate) polynomial of degree at most 𝑑 over any field 𝔽. If 𝑥 ∈ 𝑅 𝔽, then Pr 𝑝 𝑥 =0 ≤𝑑/|𝔽|. Perfect matching: Let 𝐺=(𝐴∪𝐵,𝐸) be a bipartite graph. Does it have a perfect matching? Let 𝐴={ 𝑎 1 ,…, 𝑎 𝑛 }, 𝐵={ 𝑏 1 ,…, 𝑏 𝑛 } and define 𝑀[𝑖,𝑗]= 0, if { 𝑎 𝑖 , 𝑏 𝑗 }∉𝐸 & 𝑥 𝑖,𝑗 , if { 𝑎 𝑖 , 𝑏 𝑗 }∈𝐸 Recall det 𝑀 = 𝜋∈ 𝑆 𝑛 −1 𝑠𝑔𝑛(𝜋) 𝑖=1 𝑛 𝑀[𝑖,𝜋(𝑖)] det⁡(𝑀) is a polynomial of degree ≤𝑛 in the 𝑥 𝑖,𝑗 ’s which is non-zero iff 𝐺 has perfect matching. Algo: 1) Fix a field 𝔽 of size ~2𝑛, and pick 𝑥 𝑖,𝑗 ∈ 𝑅 𝔽 2) Use a fast algorithm to compute determinant

Example 2 Given an 𝑛-vertex graph,find a max. size IS. A randomized algorithm finds an IS of expected size 𝑣∈V 1/( 𝑑 𝑣 +1) : Pick a random permutation of the vertices Iterate over permuted sequence Include vertex when no neighbor has been included yet

Example 3 Directed 𝑘-path problem: given directed graph 𝐺, is there a simple path on at least 𝑘 vertices. In polynomial time if 𝐺 is acyclic: Crux: cannot collide with previous part of the path 𝑣 1 𝑣 2 𝑣 3 𝑣 𝑖 𝑣 𝑛 1 2 1

Example 3 Idea: try to reduce the general case to acyclic case. If true is returned, 𝐺 ′ has a 𝑘-path so 𝐺 as well If 𝐺 has a 𝑘-path 𝑝 1 ,…, 𝑝 𝑘 , look at one iteration: Pr 𝑐 𝐺 ′ has k−path ≥ Pr 𝑐 ∀𝑖∈{1,…,𝑘}:𝑐 𝑝 𝑖 =𝑖 = 1 𝑘 𝑘 So prob. we won’t find a path in the whole loop is at most 1+𝑥≤ 𝑒 𝑥 for 𝑥=−1/ 𝑘 𝑘 1− 1 𝑘 𝑘 𝑘 𝑘 ≤1/𝑒, if 𝑘-path exists, return true with prob 1− 1 𝑒 𝑂 ∗ ( 𝑘 𝑘 ) time algorithm with constant one-sided err.

Example 4 Determine whether a given 𝑘-CNF formula on 𝑛 vars is satisfiable in 𝑂 ∗ 2𝑘 𝑘+1 𝑛 time, for constant 𝑘. for 𝑘=100, this is about 𝑂 ∗ ( 1.9802 𝑛 ) relies on local search subroutine Given 𝑥,𝑦∈{0,1 } 𝑛 , 𝐻(𝑥,𝑦) is Hamming distance (i.e. nr. Of coordinates where 𝑥 and 𝑦 differ)

Example 4 Run time: recursion tree of depth ≤𝑑, ℓ≤𝑘 fanout Polynomial time per recursive call so 𝑂 ∗ ( 𝑘 𝑑 ) time. Only considers assignments of HD ≤𝑑: decreases 𝑑 if it flips a coordinate if true is returned, found a solution of HD ≤𝑑 By induction on 𝑑: if sol 𝑦 of HD ≤𝑑, it returns true True for 𝑑=0, since condition on L2 fails For 𝑑>0, 𝑥,𝑦 must differ on variable occuring in 𝐶 𝑗 If variable occurs in 𝑖’th literal 𝐻 𝑧 𝑖 ,𝑦 ≤𝑑−1

Example 4 x x x x x

Example 4 If returns true it is correct since so is localSearch If sol y exists, prob true is returned in one iteration is , so in some iteration ≥1− 1 𝑒 And the running time boils down to:

More on randomized algo’s Randomized rounding Important approach to turn LP-solutions into ILP solutions for obtaining good apx algo’s Many more cool examples LNMB course `Randomized Algorithms’.

Algorithms & Complexity We saw: NP,co-NP,NP-completeness, undecidability Approximation algorithms Exponential time (and FPT) algorithms Treewidth Randomized algorithms What did we skip?

Algorithms Many `basic’ algorithms for classic problems on Sequences: Sorting, Pattern Matching,… Graphs: Shortest Path, Matching, Max-Flow, .. Algebra: Matrix multiplication, FFT, number-theoretic Computational Geometry: Convex hull, closest pair Online,quantum,parallel,game-theoretic algorithms Important one we skipped: Heuristic algorithms We’ve seen many of the important techniques Greedy,DP,Divide&Conquer,LP-rounding,randomization

Complexity Models of Computation Diagonalization Proof systems Turing machine, Circuits, branching programs… Diagonalization Polynomial hierarchy, NP-intermediate problems Proof systems PSPACE = IP PCP-theorem Derandomization L=NL, Adleman’s theorem Communication Complexity `Fine-grained complexity’ (S)ETH, W-hardness, conditional hardness results

Further reading

Multiplication (if time left)

Arithmetic on primary school 0001100 23958233 23958233 5830 + 5830 × ---------- -------------- 23964063 00000000 (=23,958,233× 0) 71874699 (=23,958,233× 30) 191665864 (=23,958,233× 800) 119791165 + (=23,958,233×5,000) ------------ 139676498390 (=139,676,498,390)

Arithmetic on primary school 0001100 23958233 23958233 5830 + 5830 × ---------- -------------- 23964063 00000000 71874699 191665864 119791165 + -------------- 139676498390

Arithmetic on primary school 0001100 23958233 23958233 5830 + 5830 × ---------- -------------- 23964063 00000000 71874699 191665864 119791165 + -------------- 139676498390

Arithmetic on primary school 0001100 23958233 23958233 5830 + 5830 × ---------- -------------- 23964063 00000000 71874699 191665864 119791165 + -------------- 139676498390

Arithmetic on primary school 0001100 23958233 23958233 5830 + 5830 × ---------- -------------- 23964063 00000000 71874699 191665864 119791165 + -------------- 139676498390

Arithmetic on primary school 0001100 23958233 23958233 5830 + 5830 × ---------- -------------- 23964063 00000000 71874699 191665864 119791165 + -------------- 139676498390

High school: complex numbers (a+bi)*(c+di) = (ac-bd)+(ad+bc)i Given a,b,c,d, compute (ac-bd) and (ad+bc). How many multiplications are needed? Only three! Use M1 = ac, M2=bd, M3=….. Then (ac-bd) = M1-M2 (ad+bc) = M3-ac-bd = M3-M1-M2

High school: complex numbers (a+bi)*(c+di) = (ac-bd)+(ad+bc)i Given a,b,c,d, compute (ac-bd) and (ad+bc). How many multiplications are needed? Only three! Use M1 = ac, M2=bd, M3=(a+b)(c+d). Then (ac-bd) = M1-M2 (ad+bc) = M3-ac-bd = M3-M1-M2

Matrix Multiplication b c d e f g h ae+bg af+bh ce+dg cf+dh = X Seems to require 8 multiplications. Multiplying nxn matrices seems to take n3 multiplications. Was thought by researchers for a while as well But then…. in 1969…… : n2.81 multiplications! Volker Strassen

Matrix Multiplication b c d e f g h ae+bg af+bh ce+dg cf+dh = X = M1: (a+d)(e+h) M2: (c+d)e M3: a(f-h) M4: d(g-e) M5: (a+b)*h M6: (c-a)*(e+f) M7: (b-d)*(g+h) M1+M4-M5+M7 M3+M5 M2+M4 M1-M2+M3+M6

Matrix Multiplication = X

Matrix Multiplication b11 b12 a21 a22 b21 b22 c11 c12 d11 d12 c21 c22 d21 d22 A B C D E F G H e11 e12 f11 f12 e21 e22 f21 f22 g11 g12 h11 h12 g21 g22 h21 h22 AE+FG AF+BH CE+DG CF+DH = X = M1: (A+D)(E+H) M2: (C+D)E M3: A(F-H) M4: D(G-E) M5: (A+B)*H M6: (C-A)*(E+F) M7: (B-D)*(G+H) M1+M4-M5+M7 M3+M5 M2+M4 M1-M2+M3+M6

Matrix Multiplication B C D a11 a12 b11 b12 a21 a22 b21 b22 c11 c12 d11 d12 c21 c22 d21 d22 E F G H e11 e12 f11 f12 e21 e22 f21 f22 g11 g12 h11 h12 g21 g22 h21 h22 AE+FG AF+BH CE+DG CF+DH = X 1 mult of 4x4 matrices -> 7 mult’s of 2x2 matrices -> 49 multiplications of numbers 1 mult of nxn matrices -> 7lg(n)=2lg(7)*lg(n)=nlg(7)=n2.81 multiplications of numbers. can also be used for matrix inversion, among others