Lecture 20: April 12 Introduction to Randomized Algorithms and the Probabilistic Method

Making Decision Flip a coin.

Making Decision Flip a coin! An algorithm which flip coins is called a randomized algorithm.

Why Randomness? A randomized algorithm is simpler. Making decisions could be complicated. Consider the minimum cut problem Can be solved by max flow. Randomized algorithm? Pick a random edge and contract. And repeat until two vertices left.

Why Randomness? A randomized algorithm is faster. Making good decisions could be expensive. Consider a sorting procedure Picking an element in the middle makes the procedure very efficient, but it is expensive (i.e. linear time) to find such an element. Picking a random element will do.

Why Randomness? A randomized algorithm is faster. Making good decisions could be expensive.  Minimum spanning trees A linear time randomized algorithm, but no known linear time deterministic algorithm.  Primality testing A randomized polynomial time algorithm, but it takes thirty years to find a deterministic one.  Volume estimation of a convex body A randomized polynomial time approximation algorithm, but no known deterministic polynomial time approximation algorithm.

Why Randomness? Probabilistic method: Proving the existence of an object satisfying certain properties without actually constructing it. Ramsey graph Find a two colouring of a complete graph of size n so that there is no monochromatic complete subgraph of size 2log(n) Show that a random object satisfying those properties with positive probability! A random 2 colouring will do.

Why Randomness? Probabilistic method: Proving the existence of an object satisfying certain properties without actually constructing it. High girth high chromatic number graphs Find a graph with high girth (i.e. without short cycles) which requires many colours to colour it properly Show that a random object satisfying those properties with positive probability! A (cleverly constructed) random graph will do.

Why Randomness? In many practical problems, we need to deal with HUGE input, and don’t even have time to read it once. But can we still do something useful? Sublinear algorithm: randomness is essential.  Fingerprinting: verifying equality of strings, pattern matching.  The power of two choices: load balancing, hashing.  Random walk: check connectivity in log-space.

Why Randomness? To fool an adversary, or not to be fooled by an adversary. Graph isomorphism: to test if two graphs are the same. Suppose Bob wants to convince you that two graphs are the same. You want to verify Bob’s claim, and you can ask Bob questions. What do you ask? This problem is hard and no one knows how to do it in polynomial time. Simply ask Bob to give you an ordering of the graphs to prove his claim.

Graph non-isomorphism Suppose Bob wants to convince you that two graphs are not the same. You want to verify Bob’s claim, and you can ask him questions. What do you ask? Goal: If the graphs are not the same, then you believe Bob. If the graphs are the same, then you will catch Bob lying. The idea is very simple. Use randomness.

An Analogy Suppose Bob wants to convince you that two coins are not the same. (Bob claims that one is real and one is counterfeit, but you don’t know how to check.) You want to verify Bob’s claim, and you can ask him questions. What do you ask? Goal: If the coins are not the same, then you believe Bob. If the coins are the same, then you will catch Bob lying. Idea: put the coins in your back, pick one randomly (but you know which one you pick), and ask Bob whether the coin picked is real or counterfeit. Why does it work?

Graph non-isomorphism You can do the following: - Label the graphs as G1 and G2. - Randomly pick one graph and randomly permutes the ordering of its vertices. - Ask Bob if the resulting graph is G1 or G2. Believe Bob if his answer is correct; otherwise not. Claim: if G1 and G2 are the same graphs, then you’ll catch Bob with probability at least ½. Proof: If G1 and G2 are the same graphs, then Bob doesn’t know which one you originally picked, and so can just guess, and hence with probability ½ he is incorrect.

Interactive Proof System This kind of protocol is called an interactive proof system, it has important applications in cryptography and complexity theory. This also shows that asking questions is really important in learning, e.g. can ask questions to your teacher to verify his claim! Theorem: Any PSPACE problem has an interactive proof system.

Interactive Proof System Counting 3-SAT: Given a boolean formula, check if the number of satisfying assignments is equal to k. 3-SAT: Given a boolean formula, check if there is a satisfying assignment. Reversi: Given a broad configuration, check if the black player has a winning strategy. All have an interactive proof system. Implication: even being foolish doesn’t mean someone can fool you easily!

Probabilistic Checkable Proofs 3-SAT: Given a boolean formula, check if this formula is satisfiable. If this is satisfiable, you can convince me by showing a truth assignment, then I can check in linear time whether it indeed satisfies the formula. Can we do it in sublinear time? PCP theorem: There is a proof protocol with the following property: I only need to read 3 bits of the proof and if you are honest, then I always trust you; if you are lying, then I can catch you lying with probability at least ½!! Has deep implication in hardness of approximation!

Plan  Randomized algorithms  Randomized rounding  Probabilistic method  Sublinear algorithm Tool: basic probability random variables, expected value, linearity of expectations, some basic inequalities, etc.