1. Randomized Algorithms Morteza ZadiMoghaddam Amin Sayedi

2. Types of Randomized algorithms Las Vegas Las Vegas Monte Carlo Monte Carlo

3. Las Vegas Always gives the true answer. Always gives the true answer. Running time is random. Running time is random. Running time is bounded. Running time is bounded. Quick sort is a Las Vegas algorithm. Quick sort is a Las Vegas algorithm.

4. Monte Carlo It may produce incorrect answer! It may produce incorrect answer! We are able to bound its probability. We are able to bound its probability. By running it many times on independent random variables, we can make the failure probability arbitrarily small at the expense of running time. By running it many times on independent random variables, we can make the failure probability arbitrarily small at the expense of running time.

5. Monte Carlo Example Suppose we want to find a number among n given numbers which is larger than or equal to the median.

6. Suppose A 1 < … < A n. We want A i, such that i ≥ n/2. It’s obvious that the best deterministic algorithm needs O(n) time to produce the answer. n may be very large! Suppose n is 100,000,000,000 !

7. Monte Carlo Example Choose 100 of the numbers with equal probability. Choose 100 of the numbers with equal probability. find the maximum among these numbers. find the maximum among these numbers. Return the maximum. Return the maximum.

8. Monte Carlo Example The running time of the given algorithm is O(1). The running time of the given algorithm is O(1). The probability of Failure is 1/(2 100 ). The probability of Failure is 1/(2 100 ). Consider that the algorithm may return a wrong answer but the probability is very smaller than the hardware failure or even an earthquake! Consider that the algorithm may return a wrong answer but the probability is very smaller than the hardware failure or even an earthquake!

9. Monte Carlo Suppose the output is Yes or No. Suppose the output is Yes or No. One sided error. One sided error. Two sided error. Two sided error.

10. RP Class ( randomized polynomial ) Bounded polynomial time in the worst case. Bounded polynomial time in the worst case. If the answer is Yes; Pr[ return Yes] > ½. If the answer is Yes; Pr[ return Yes] > ½. If the answer is No; Pr[ return Yes] = 0. If the answer is No; Pr[ return Yes] = 0. ½ is not actually important. ½ is not actually important.

11. PP Class ( probabilistic polynomial ) Bounded polynomial time in worst case. Bounded polynomial time in worst case. If the answer is Yes; Pr[ return Yes] > ½. If the answer is Yes; Pr[ return Yes] > ½. If the answer is No; Pr[ return Yes] < ½. If the answer is No; Pr[ return Yes] < ½. Unfortunately the definition is weak because the distance to ½ is important but is not considered. Unfortunately the definition is weak because the distance to ½ is important but is not considered.

12. Routing Problem There are n computers. There are n computers. Each computer has a packet. Each computer has a packet. Each packet has a destination D(i). Each packet has a destination D(i). Packets can not follow the same edge simultaneously. Packets can not follow the same edge simultaneously. An oblivious algorithm is required. An oblivious algorithm is required.

13. Routing Problem For any deterministic oblivious algorithm on a network of N nodes each of outdegree d, there is an instance of permutation routing requiring (N/d) ½. For any deterministic oblivious algorithm on a network of N nodes each of outdegree d, there is an instance of permutation routing requiring (N/d) ½.

14. Routing Problem Pick random intermediate destination. Pick random intermediate destination. Packet i first travels to the intermediate destination and then to the final destination. Packet i first travels to the intermediate destination and then to the final destination. With probability at least 1-(1/N), every packet reaches its destination in 14n of fewer steps in Q n. With probability at least 1-(1/N), every packet reaches its destination in 14n of fewer steps in Q n. The expected number of steps is 15n. The expected number of steps is 15n.

15. Maximum Satisfiability You have m clauses and n boolean variables. You have m clauses and n boolean variables. Each clause contains some of variables or some of complements. Each clause contains some of variables or some of complements. A clause is satisfied if at least one of it’s variables are satisfied. A clause is satisfied if at least one of it’s variables are satisfied. We want to set the variables such that the number of satisfied clauses is maximized. We want to set the variables such that the number of satisfied clauses is maximized.

16. Example for Maximum Sat There are 3 variables A, B and C. There are 3 variables A, B and C. M 1 = (A) or (B) M 1 = (A) or (B) M 2 = (A) or (not B) or (not C) M 2 = (A) or (not B) or (not C) M 3 = (C) M 3 = (C) M 4 = (B) or (not C) M 4 = (B) or (not C) M 5 = (not C) M 5 = (not C)

17. Example of Maximum Sat Set A = True Set A = True Set B = True Set B = True Set C = False Set C = False Four of the clauses are satisfied. Four of the clauses are satisfied.

18. Maximum Sat This problem is a famous problem which has no polynomial time algorithm yet. It’s NP-hard. This problem is a famous problem which has no polynomial time algorithm yet. It’s NP-hard.

19. Maximum Sat For any set of m clauses, there is truth assignment for the variables that satisfies at least m/2 clauses. For any set of m clauses, there is truth assignment for the variables that satisfies at least m/2 clauses.

20. Maximum Sat Let Z i =1 if the i-th clause is satisfied and 0 otherwise. Let Z i =1 if the i-th clause is satisfied and 0 otherwise. Set the variables in a random way. Set the variables in a random way. The probability of a clause with k variables to be true is 1- (1/(2 k )) >= ½. The probability of a clause with k variables to be true is 1- (1/(2 k )) >= ½. So E[Z 1 ]+…+E[Z m ] >= ½. So E[Z 1 ]+…+E[Z m ] >= ½. Thus there exist at least one assignment such that Z 1 +…+Z m >= ½. Thus there exist at least one assignment such that Z 1 +…+Z m >= ½.

21. Maximum Sat algorithm This problem is NP-hard so we seek for approximation algorithms. This problem is NP-hard so we seek for approximation algorithms. We have an algorithm that produces an answer which is at least ½ of the best answer. We have an algorithm that produces an answer which is at least ½ of the best answer. If all clauses have at least 2 literals then we have an algorithm that produces an answer which is at least ¾ of the best one. If all clauses have at least 2 literals then we have an algorithm that produces an answer which is at least ¾ of the best one.

22. Maximum Sat algorithm We want to maximize Z 1 +…+Z m. We want to maximize Z 1 +…+Z m. We have some inequalities: We have some inequalities: ∑ y i (if X i is in Z j and is uncomplemented) ∑ y i (if X i is in Z j and is uncomplemented) ∑ (1- y i ) (if X i is in Z j and is complemented). ∑ (1- y i ) (if X i is in Z j and is complemented). This inequality must be hold: This inequality must be hold: ∑ y i + ∑ (1- y i ) >= Z j ∑ y i + ∑ (1- y i ) >= Z j This problem could be solved using integer linear programming. This problem could be solved using integer linear programming. We have to use linear programming. We have to use linear programming.

23. Maximum Sat Solve the problem using linear programming. Solve the problem using linear programming. You get a real number for each yi or z i. You get a real number for each yi or z i. Assign X i true with the probability y i. Assign X i true with the probability y i. The expected number of clauses that are satisfied is (1- 1/e) of the best answer. The expected number of clauses that are satisfied is (1- 1/e) of the best answer.

24. Maximum Sat algorithm Using both algorithms and choosing the better answer gives us an answer which is at least ¾ of the best answer!!! Which is better than ½ and 1- 1/e. Using both algorithms and choosing the better answer gives us an answer which is at least ¾ of the best answer!!! Which is better than ½ and 1- 1/e.

25. 2-Sat Every clause has at most 2 literals. Every clause has at most 2 literals. We want to check if all clauses can be satisfied. We want to check if all clauses can be satisfied. It has polynomial algorithm. It has polynomial algorithm. Assign random values to the variables. Assign random values to the variables. If all clauses are satisfied we are finished. If all clauses are satisfied we are finished. If there is an unsatisfied clause, the value of one of it’s literals is different from the best answer. If there is an unsatisfied clause, the value of one of it’s literals is different from the best answer. Change the value of one of the variables in this clause. You may make a good change or bad one. Change the value of one of the variables in this clause. You may make a good change or bad one.

26. 2-Sat You are walking on a path. You are walking on a path. If you are on 0 you go to 1. If you are on 0 you go to 1. If you are on i you go to i+1 or i-1 with equal probability. If you are on i you go to i+1 or i-1 with equal probability. The expected number of steps to reach the end of the path is O(n 2 ). The expected number of steps to reach the end of the path is O(n 2 ). So the given algorithm is O(n 3 ). So the given algorithm is O(n 3 ).

27. Graph Connectivity You want to check if two vertices u and v are in the same connected component. You want to check if two vertices u and v are in the same connected component. Start a random walk from v. Start a random walk from v. Have a random walk of length 2n 3. Have a random walk of length 2n 3. If you haven’t visited u, the probability of u to be in this component is less than ½. If you haven’t visited u, the probability of u to be in this component is less than ½. By repeating this algorithm, you can make the probability of failure arbitrarily small. By repeating this algorithm, you can make the probability of failure arbitrarily small.

28. Graph Connectivity Running time of algorithm is O(n3). Required space is O(logn).

29. Diameter of a Point Set You want to find the diameter of set of n points in the space. You want to find the diameter of set of n points in the space. Suppose I(x) is the convex body formed by the intersection of n sphere centered at n points with radius x. Suppose I(x) is the convex body formed by the intersection of n sphere centered at n points with radius x. F(p) is distance between p and the point in the set that is farthest from p. F(p) is distance between p and the point in the set that is farthest from p.

30. Diameter of a Point Set Consider I(x) when x=F(p). Consider I(x) when x=F(p). For any q in S, if q is in I(x) then F(q)<F(p). For any q in S, if q is in I(x) then F(q)<F(p). And if q is not in I(x) then F(p)<F(q). And if q is not in I(x) then F(p)<F(q).

31. Diameter of a Point Set Pick a point p in s at random. Computer F(p). [O(n)] Pick a point p in s at random. Computer F(p). [O(n)] Set x=F(p). Compute I(x). [O(n logn)] Set x=F(p). Compute I(x). [O(n logn)] Find the points outside I(x). Call this set T. [O(n logn)] Find the points outside I(x). Call this set T. [O(n logn)] If T is empty return x as the answer, else continue on T. If T is empty return x as the answer, else continue on T.

32. Diameter of a Point Set The running time of algorithm above is O(n log n). The running time of algorithm above is O(n log n). In each step, all points that have smaller F(x) than the chosen point are removed. In each step, all points that have smaller F(x) than the chosen point are removed.

33. All-Pairs Shortest Paths Let G(V,E) be an undirected, connected graph with V={1,…,n} and |E|=m. Let G(V,E) be an undirected, connected graph with V={1,…,n} and |E|=m. The adjacency matrix A is an n  n 0-1 matrix with A ij =A ji =1 if and only if the edge (i,j) is present in E. The adjacency matrix A is an n  n 0-1 matrix with A ij =A ji =1 if and only if the edge (i,j) is present in E. We are going to compute matrix D which D ij equals the length of a shortest path from vertex i to vertex j. We are going to compute matrix D which D ij equals the length of a shortest path from vertex i to vertex j.

34. All-Pairs Distances Z A 2 Z A 2 Compute matrix A’ such that A’ ij =1 if and only if i ≠ j and (A ij =1 or Z ij >0) Compute matrix A’ such that A’ ij =1 if and only if i ≠ j and (A ij =1 or Z ij >0) If A’ ij =1 for all i ≠ j then return D = 2A’-A. If A’ ij =1 for all i ≠ j then return D = 2A’-A. Recursively compute the APD matrix D’ for the graph G’ with adjacency matrix A’. Recursively compute the APD matrix D’ for the graph G’ with adjacency matrix A’. S AD’ S AD’ Return matrix D with D ij =2D’ ij if S ij ≥D’ ij Z ii, otherwise D ij =2D’ ij -1. Return matrix D with D ij =2D’ ij if S ij ≥D’ ij Z ii, otherwise D ij =2D’ ij -1.

35. APSP The APD algorithm computes the distance matrix for an n-vertex graph in time O(MM(n)log(n)) using integer matrix multiplication algorithm. The APD algorithm computes the distance matrix for an n-vertex graph in time O(MM(n)log(n)) using integer matrix multiplication algorithm. Matrix multiplication algorithm running in time O(n 2.376 ). Matrix multiplication algorithm running in time O(n 2.376 ).

36. Boolean Product Witness Matrix Suppose A and B are n  n boolean matrices and P=AB is their product under Boolean matrix multiplication. Suppose A and B are n  n boolean matrices and P=AB is their product under Boolean matrix multiplication. A witness for P ij is an index k  {1,…,n} such that A ik =A kj =1. Observe that P ij =1 if and only if it has some witness k. A witness for P ij is an index k  {1,…,n} such that A ik =A kj =1. Observe that P ij =1 if and only if it has some witness k.

37. BPWM W -AB W -AB 2. for t=0,…,  log(n)  do 2. for t=0,…,  log(n)  do 2.1. r 2 t 2.1. r 2 t 2.2. Repeat  3.77log(n)  times 2.2. Repeat  3.77log(n)  times 2.2.1. Choose random R  {1,…,n} with |R|=r. 2.2.1. Choose random R  {1,…,n} with |R|=r. 2.2.2. Compute A R and B R. 2.2.2. Compute A R and B R. 2.2.3. Z A R B R. 2.2.3. Z A R B R. 2.2.4. for all (i,j) do 2.2.4. for all (i,j) do if W ij < 0 and Z ij is witness then W ij Z ij if W ij < 0 and Z ij is witness then W ij Z ij 3. for all (i,j) do 3. for all (i,j) do if W ij < 0 then find witness W ij by brute force. if W ij < 0 then find witness W ij by brute force.

38. BPWM The BPWM algorithm is a Las Vegas algorithm for the BPWM problem with expected running time O(MM(n)log 2 (n)). The BPWM algorithm is a Las Vegas algorithm for the BPWM problem with expected running time O(MM(n)log 2 (n)). The probability that no witness is found for P ij before the end of Step 2 is at most The probability that no witness is found for P ij before the end of Step 2 is at most (1-1/2e) 3.77log(n)  1/n. (1-1/2e) 3.77log(n)  1/n.

39. Determining Shortest Path A successor matrix S for an n-vertex graph G is an n  n matrix such that S ij is the index of a neighbor of vertex i that lies on a shortest path from i to j. A successor matrix S for an n-vertex graph G is an n  n matrix such that S ij is the index of a neighbor of vertex i that lies on a shortest path from i to j.

40. APSP Compute the distance matrix D=APD(A). Compute the distance matrix D=APD(A). for s={0,1,2} do for s={0,1,2} do Compute 0-1 matrix D kj (s) =1 if and only if D kj +1 = s (mod 3) Compute 0-1 matrix D kj (s) =1 if and only if D kj +1 = s (mod 3) Compute the witness matrix W (s) =BPWM(A,D (s) ). Compute the witness matrix W (s) =BPWM(A,D (s) ). Compute successor matrix S for G. Compute successor matrix S for G.

41. APSP Algorithm APSP computes the successor matrix for an n-vertex graph G in expected time O(MM(n)log 2 (n)). Algorithm APSP computes the successor matrix for an n-vertex graph G in expected time O(MM(n)log 2 (n)).

42. Algorithm contract H G H G While H has more than 2 vertices do While H has more than 2 vertices do Choose an edge (x,y) uniformly at random from the edges in H. Choose an edge (x,y) uniformly at random from the edges in H. F F  {(x,y)}. F F  {(x,y)}. H H / (x,y). H H / (x,y). (C,V/C) the sets of vertices corresponding to the two meta-vertices in H=G/F. (C,V/C) the sets of vertices corresponding to the two meta-vertices in H=G/F.

43. FastCut n |v| n |v| if n  6 then compute min-cut of G by brute- force enumeration else if n  6 then compute min-cut of G by brute- force enumeration else t  1+n/  2  t  1+n/  2  Using Algorithm Contract, perform two independent contraction sequences to obtain graphs H 1 and H 2 each with t vertices. Using Algorithm Contract, perform two independent contraction sequences to obtain graphs H 1 and H 2 each with t vertices. Recursively compute min-cuts in each of H 1 and H 2. Recursively compute min-cuts in each of H 1 and H 2. Return the smaller of the two min-cuts. Return the smaller of the two min-cuts.

44. Fastcut Algorithm Fastcut succeeds in finding a min-cut with probability  (1/log(n)). Algorithm Fastcut succeeds in finding a min-cut with probability  (1/log(n)). Algorithm Fastcut runs in O(n 2 log(n)) time and uses O(n 2 ) space. Algorithm Fastcut runs in O(n 2 log(n)) time and uses O(n 2 ) space.

45. MST Finding MST in a graph with n vertices and m edges has a Las Vegas algorithm which has the expected running time O(n+m). Finding MST in a graph with n vertices and m edges has a Las Vegas algorithm which has the expected running time O(n+m). But We don’t have enough time to Explain it !!! But We don’t have enough time to Explain it !!!

46. Research problems Devise an algorithm for the all-pairs shortest paths problem that does not use matrix multiplication and runs in time O(n 3-  ) for a positive constant . Devise an algorithm for the all-pairs shortest paths problem that does not use matrix multiplication and runs in time O(n 3-  ) for a positive constant . Devise an algorithm for computing the diameter of an unweighted graph that does not use matrix multiplication and runs in time O(n 3-  ) for a positive constant . Devise an algorithm for computing the diameter of an unweighted graph that does not use matrix multiplication and runs in time O(n 3-  ) for a positive constant .

47. Research problems Devise a Las Vegas or a deterministic algorithm for min-cuts with running time close to O(n 2 ). Devise a Las Vegas or a deterministic algorithm for min-cuts with running time close to O(n 2 ). Is there a randomized algorithm for min- cuts with expected running time close to O(m)? Is there a randomized algorithm for min- cuts with expected running time close to O(m)?