1. Randomized Algorithms Kyomin Jung KAIST Applied Algorithm Lab Jan 12, WSAC 2010 1

2. Randomness in Algorithms A Probabilistic Turing Machine is a TM given with a (binary) random “coin”. In the computation process of the PTM, PTM can toss a coin to decide its decision. Ex) Computer games, random screen savor… Note: in computer programming, the random number generating function (ex, rand() in C) takes an initial random seed, and generates a “deterministic” sequence of numbers. Hence they are not “truly” random. 2

3. Why randomness can be helpful? A Simple example Suppose we want to check whether an integer set has an even number or not. Even when A has n/2 many even numbers, if we run a Deterministic Algorithm, it may check n/2+1 many elements in the worst case. A Randomized Algorithm: At each time, choose an elements (to check) at random. Smooths the “worst case input distribution” into “randomness of the algorithm” 3

4. Property of “independent” random trials Law of Large Number  If we repeat independent random samplings many times according to a fixed distribution D, the “the average becomes close to the expectation” (ex: dice rolling) Ex) Erdos-Renyi Random Graph G(n,p). Voting poll … 4

5. Suppose we have a coin with probability of heads is p. Let X be a random variable where X=1 if the coin flip gives heads and X=0 otherwise. E[X] = 1*p + 0*(1-p) = p After flipping a coin w times we can estimate the heads prob by average of x i. The Chernoff Bound tells us that this estimate converges exponentially fast to the true mean p. Chernoff Bound 5

6. Las Vegas vs Monte Carlo A Las Vegas algorithm is a randomized algorithm that always gives correct results Ex: randomized quick sort A Monte Carlo algorithm is one whose running time is deterministic, but whose output may be correct only with a certain probability. Class BPP (Bounded-error, Probabilistic, Polynomial time)  problem which has a Monte Carlo algorithm to solve it 6

7. Randomized complexity classes Model: probabilistic Turing Machine BPP  L  BPP if there is a p.p.t. TM M: x  L  Pr y [M(x,y) accepts] ≥ 2/3 x  L  Pr y [M(x,y) rejects] ≥ 2/3  “p.p.t” = probabilistic polynomial time  y : (a sequence of) random coin tosses 7

8. Randomized complexity classes RP (Random Polynomial-time)  L  RP if there is a p.p.t. TM M: x  L  Pr y [M(x,y) accepts] ≥ ½ x  L  Pr y [M(x,y) rejects] = 1 coRP (complement of Random Polynomial-time)  L  coRP if there is a p.p.t. TM M: x  L  Pr y [M(x,y) accepts] = 1 x  L  Pr y [M(x,y) rejects] ≥ ½ 8

9. Error reduction for BPP given L, and p.p.t. TM M: x  L  Pr y [M(x,y) accepts] ≥ ½ + ε x  L  Pr y [M(x,y) rejects] ≥ ½ + ε new p.p.t. TM M’:  simulate M k/ ε 2 times, each time with independent coin flips  accept if majority of simulations accept  otherwise reject 9

10. Error reduction for BPP  X i random variable indicating “correct” outcome in i-th simulation (out of m = k/ ε 2 ) Pr[X i = 1] ≥ ½ + ε Pr[X i = 0] ≤ ½ - ε  E[X i ] ≥ ½+ ε  X = Σ i X i  μ = E[X] ≥ (½ + ε )m  By Chernoff Bound: Pr[X ≤ m/2] ≤ 2 -  ( ε 2 m) 10

11. Derandomization Question: is BPP=P? Goal: try to simulate BPP in polynomial time use Pseudo-Random Generator (PRG): Good if t=O(log m) Blum-Micali-Yao PRG seed length t = m δ seedoutput string G t bitsm bits 11

12. Example of randomized algorithm : Minimum Cut Problem A B C D Note: Size of the min cut must is no larger than the smallest node degree in graph 12

13. Application: Internet Minimum Cut 13

14. Randomized Algorithm (by David Karger) While |V| > 2:  Pick a random edge (x, y) from E  Contract the edge: Keep multi-edges, remove self-loops Combine nodes The two remaining nodes represent reasonable choices for the minimum cut sets 14

15. Analysis Suppose C is a minimum cut (set of edges that disconnects G) When we contract edge e:  Unlikely that e  C  So, C is likely to be preserved What is the probability a randomly chosen edge is in C? 15

16. Random Edge in C? |C| must be  degree of every node in G How many edges in G: |E| = sum of all node degrees / 2  n |C| / 2 Probability a random edge is in C  2/n 16

17. Iteration How many iterations? Probability for first iteration: Prob(e 1  C)  1 – 2/n Probability for second iteration: Prob(e 2  C | e 1  C)  1 – 2/(n-1)... Probability for last iteration: Prob(e n-2  C |…)  1 – 2/(n-(n-2-1))  1 – 2/3 n - 2 17

18. Probability of finding C? Probability of not finding C = 1 – 2/(n(n-1)) 18

19. Probability of finding C? Probability of not finding C on one trial:  1 – 2/(n(n-1))  1 – 2/n 2 Probability of not finding C on k trials:  [1 – 2/n 2 ] k If k = cn 2, Prob failure  (1/e) c Recall: lim (1 – 1/x) x = 1/e x   19

20. Example of randomized algorithm : Random Sampling What is a random sampling?  Given a probability distribution, pick a point according to.  e.g. Monte Carlo method for integration Choose numbers uniformly at random from the integration domain, and sum up the value of f at those points 20

21. How to use Random Sampling? Volume computation in Euclidean space. 21

22. Hit and Run Hit and Run algorithm is used to sample from a convex set in an n-dimensional Eucliden space. Assume that we can evaluate its ratios at given points It converges in time. 22

23. Example of randomized algorithm : Markov Chain Monte Carlo Let be a probability density function on S={1,2,..n}. f( ‧ ) is any function on S and we want to estimate Construct P ={P ij }, the transition matrix of an irreducible Markov chain with states S, where and π is its unique stationary distribution. 23

24. Markov Chain Monte Carlo Run this Markov chain for times t=1,…,N and calculate the Monte Carlo sum then 24

25. Applied Algorithm Lab ( http://aa.kaist.ac.kr/ ) Research Projects Markov Random Field and Image segmentation Network consensus algorithm design Community detection in complex networks Routing/scheduling algorithm analysis in wirel ess networks Motion surveillance in video images Character recognition in images SAT solver design based on formula structures Decentralized control algorithms for multi-agent rob ot system Financial data mining Yongsub LimBoyoung Kim ProfessorGraduate Students Kyomin JungNam-ju Kwak Members