Randomized Algorithms CS648 Lecture 17 Miscellaneous applications of Backward analysis

Minimum spanning tree

Minimum spanning tree b a c d h x y u v 18 7 1 19 22 10 3 12 15 11 5 16 17 13 4 2 9 6

Less known but it is the first algorithm for MST Minimum spanning tree 16 d 17 9 Algorithms: Prim’s algorithm Kruskal’s algorithm Boruvka’s algorithm v x h 4 2 5 19 6 11 y 3 3 22 c 12 a 18 1 15 10 u b 7 13 Less known but it is the first algorithm for MST

Minimum spanning tree 𝑮=(𝑽,𝑬) : undirected graph with weights on edges 𝒏=|𝑽|, 𝒎=|𝑬|. Deterministic algorithms: Prim’s algorithm O((𝒎 + 𝒏) log 𝒏) using Binary heap O(𝒎 + 𝒏 log 𝒏) using Fibonacci heap Best deterministic algorithm: O(𝒎 + 𝒏 𝐥𝐨𝐠 ∗ 𝒏) bound Too complicated to design and analyze Fails to beat Prim’s algorithm using Binary heap

Minimum spanning tree When finding an efficient solution of a problem appears hard, one should strive to design an efficient verification algorithm. MST verification algorithm: [King, 1990] Given a graph 𝑮=(𝑽,𝑬) and a spanning tree 𝑻, it takes O(𝒎 + 𝒏) time to determine if 𝑻 is MST of 𝑮. Interestingly, no deterministic algorithm for MST could use this algorithm to achieve O(𝒎 + 𝒏) time.

Minimum spanning tree 𝑮=(𝑽,𝑬) : undirected graph with weights on edges 𝒏=|𝑽|, 𝒎=|𝑬|. Randomized algorithm: Karger-Klein-Tarjan’s algorithm [1995] Las Vegas algorithm O(𝒎 + 𝒏) expected time This algorithm uses Random sampling MST verification algorithm Boruvka’s algorithm Elementary data structure

The notion of closeness is formalized in the following slide. Minimum spanning tree 𝑮=(𝑽,𝑬) : undirected graph with weights on edges 𝒏=|𝑽|, 𝒎=|𝑬|. Randomized algorithm: Karger-Klein-Tarjan’s algorithm [1994] Las Vegas algorithm O(𝒎 + 𝒏) expected time Random sampling : How close is MST of a random sample of edges to MST of original graph ? The notion of closeness is formalized in the following slide.

Light Edge b a c d h x y u v 18 7 1 19 22 10 3 12 15 11 5 16 17 13 4 2 9 6 Definition: Let 𝒂⊂𝑬. An edge 𝒆∈𝑬\𝒂 is said to be light with respect to 𝒂 if Question: If 𝒂 ⊂ 𝑟 𝑬 and |𝒂|= 𝒌, how many edges from 𝑬\𝒂 are light with respect to 𝒂 on expectation ? Answer: ?? 31 MST({𝒆}∪𝒂) ≠ MST(𝒂) < 𝒏 𝒌 (𝒎−𝒌)

USING Backward analysis for Miscellaneous Applications

problem 1 SMALLEST Enclosing circle

Smallest Enclosing Circle

Smallest Enclosing Circle Question: Suppose we sample 𝒌 points randomly uniformly from a set of 𝒏 points, what is the expected number of points that remain outside the smallest circle enclosing the sample? For 𝒌= 𝒏 𝟐 , the answer is <𝟏𝟎

problem 2 smallest length interval

Sampling points from a unit interval Question: Suppose we select 𝒌 points from interval [0,1], what is expected length of the smallest sub-interval ? for 𝒌=1, it is ?? for 𝒌=2, it is ?? General solution : ?? This bound can be derived using two methods. One method is based on establishing a relationship between uniform distribution and exponential distribution. Second method (for nearly same asymptotic bound) using Backward analysis. 𝟏 𝟒 𝟏 𝟗 𝟏 𝒌+𝟏 𝟐 1

problem 3 Minimum spanning tree

Light Edge b a c d h x y u v 18 7 1 19 22 10 3 12 15 11 5 16 17 13 4 2 9 6 Definition: Let 𝒂⊂𝑬. An edge 𝒆∈𝑬\𝒂 is said to be light with respect to 𝒂 if Question: If 𝒂 ⊂ 𝑟 𝑬 and |𝒂|= 𝒌, how many edges from 𝑬\𝒂 are light with respect to 𝒂 on expectation ? 31 MST({𝒆}∪𝒂) ≠ MST(𝒂)

using Backward analysis for The 3 problems : A General framework

A General framework Let 𝒁 be the desired random variable in any of these problems/random experiment. Step 1: Define an event 𝜺 related to the random variable 𝒁. Step 2: Calculate probability of event 𝜺 using standard method based on definition. (This establishes a relationship between ) Step 3: Express the underlying random experiment as a Randomized incremental construction and calculate the probability of the event 𝜺 using Backward analysis. Step 4: Equate the expressions from Steps 1 and 2 to calculate E[𝒁].

problem 3 Minimum spanning tree

A Better understanding of light edges

Minimum spanning tree d (𝑽,𝑬) v x h y c a u b Random sampling x b a c 16 d (𝑽,𝑬) 17 19 v x h 4 2 5 19 6 31 11 y 3 3 22 c 42 a 18 1 15 10 u b 7 13 Random sampling x b a c d h y u v 18 1 19 22 10 15 11 5 16 4 31 (𝑽,𝑹)

Minimum spanning tree d (𝑽,𝑬) v x h y c a u b d MST(𝑹) (𝑽,𝑹) v x h y c 16 d (𝑽,𝑬) 17 19 v x h 4 2 5 19 6 31 11 y 3 3 22 c 42 a 18 1 15 10 u b 7 13 16 d MST(𝑹) (𝑽,𝑹) v x h 4 5 18 22 31 19 11 y c a 1 15 10 u b

Minimum spanning tree d (𝑽,𝑬) v x h y c a u b d MST(𝑹) (𝑽,𝑹) v x h 𝑬\𝑹 16 d (𝑽,𝑬) 17 19 v x h 4 2 5 19 6 31 11 y 3 3 22 c 42 a 18 1 15 10 u b 7 23 16 d MST(𝑹) 7 3 42 17 23 2 19 6 (𝑽,𝑹) v x h 4 5 𝑬\𝑹 19 11 y c a 1 15 10 u b

Minimum spanning tree d (𝑽,𝑬) v x h y c a u b d MST(𝑹) (𝑽,𝑹) v x h 𝑬\𝑹 16 d (𝑽,𝑬) 17 19 v x h 4 2 5 19 6 31 11 y 3 3 22 c 42 a 18 1 15 10 u b 7 23 16 d MST(𝑹) 17 19 (𝑽,𝑹) v x h 4 2 5 𝑬\𝑹 19 6 11 y 3 3 Light c 42 a 1 15 10 u b 7 23

First useful insight Lemma1: An edge 𝒆 is light with respect to 𝒂⊂𝑬 if and only if 𝒆 belongs to MST( 𝒆 ∪𝒂).

Minimum spanning tree d (𝑽,𝑬) v x h y c a u b d MST(𝑹) (𝑽,𝑹) v x h 𝑬\𝑹 16 d (𝑽,𝑬) 17 19 v x h 4 2 5 19 6 31 11 y 3 3 22 c 42 a 18 1 15 10 u b 7 23 16 d MST(𝑹) 17 19 (𝑽,𝑹) v x h 4 2 5 𝑬\𝑹 19 6 11 y 3 3 Light heavy c 42 a 1 15 10 u b 7 23

Minimum spanning tree MST(𝑬) d (𝑽,𝑬) v x h y c a u b 4 2 5 6 y 3 3 c a 1 u b 7 Is there any relationship among MST(𝑬), MST(𝑹) and Light edges from 𝑬\𝑹 ? 16 d MST(𝑹) 17 42 23 19 (𝑽,𝑹) v x h 4 2 5 𝑬\𝑹 19 6 11 y 3 3 Light heavy c a 1 15 10 u b 7

MST(𝑬) = MST(𝑳∪𝐌𝐒𝐓(𝒂)) Second useful insight Lemma2: Let 𝒂⊂𝑬 and 𝑳 be the set of all edges from 𝑬\𝒂 that are light with respect to 𝒂. Then MST(𝑬) = MST(𝑳∪𝐌𝐒𝐓(𝒂)) This lemma is used in the design of randomized algorithm for MST as follows (just a sketch): Compute MST of a sample of 𝒎/𝟐 edges (recursively). Let it be T’. There will be expected 𝒏 edges light edges among all unsampled edges. Recursively compute MST of 𝑳∪ T’ edges which are less than 2𝒏 on expectation.

Light Edge b a c d h x y u v 18 7 1 19 22 10 3 12 15 11 5 16 17 13 4 2 9 6 Definition: Let 𝒂⊂𝑬. An edge 𝒆∈𝑬\𝒂 is said to be light with respect to 𝒂 if Question: If 𝒂 ⊂ 𝑟 𝑬 and |𝒂|= 𝒌, how many edges from 𝑬\𝒂 are light with respect to 𝒂 on expectation ? 31 MST({𝒆}∪𝒂) ≠ MST(𝒂) We shall answer the above question using the Generic framework. But before that, we need to get a better understanding of the corresponding random variable.

𝒌 𝑹 MST(𝑹) 𝑬\𝑹

𝒌 𝑹 MST(𝑹) 𝑬\𝑹 Light

𝒌 𝑹 MST(𝑹) 𝑬\𝑹 Light heavy

Can you express 𝐄[𝒁] in terms of 𝒇 𝒂 and 𝑺 only ? 𝒁: random variable for the number of light edges in 𝑬\𝑹 when 𝑹 is a random sample of 𝒌 edges. 𝑺: set of all subsets of 𝑬 of size 𝒌. 𝒇 𝒂 : number of light edges in 𝑬\𝑹 when 𝑹=𝒂. 𝐄[𝒁] = ?? Can you express 𝐄[𝒁] in terms of 𝒇 𝒂 and 𝑺 only ? 𝟏 |𝑺| 𝒂∈𝑺 𝒇 𝒂

Step 1 Question: Let 𝑹 be a uniformly random sample of 𝒌 edges from 𝑬. What is the prob. 𝒑 that an edge selected randomly from 𝑬\𝑹 is a light edge ? Two methods to find 𝒑

Calculating 𝒑 using definition Step 2 Calculating 𝒑 using definition

Calculating 𝒑 using definition Step 2 Calculating 𝒑 using definition 𝒌 Light edges =𝒇 𝒂 𝒂 MST(𝒂) 𝑬\𝒂 Light heavy

Calculating 𝒑 using definition Step 2 Calculating 𝒑 using definition 𝑺: set of all subsets of 𝑬 of size 𝒌. The probability 𝒑 is equal to = ? 𝒂∈𝑺 ? = 𝟏 𝒎−𝒌 𝟏 |𝑺| 𝒂∈𝑺 𝒇 𝒂 = 𝟏 𝒎−𝒌 𝐄[𝒁] 𝟏 |𝑺| 𝒇 𝒂 𝒎−𝒌

Spend some time on this question before proceeding further. Step 3 Expressing the entire experiment as Randomized Incremental Construction A slight difficulty in this process is the following: The underlying experiment talks about random sample from a set. But RIC involves analyzing a random permutation of a set of elements.  Question: What is relation between random sample from a set and a random permutation of the set ? Spend some time on this question before proceeding further.

random sample and random permutation Random permutation of 𝑬 Observation: 𝑨 is indeed a uniformly random sample of 𝑬 𝑨 𝑬\𝑨 𝒓 𝒎−𝒓

Spend some time on this question before proceeding further. Step 3 The underlying random experiment as Randomized Incremental Construction: Permute the edges randomly uniformly. Find the probability that 𝒊th edge is light relative to the first 𝒊−𝟏 edges. Question: Can you now calculate probability 𝒑 ? Spend some time on this question before proceeding further.

Random permutation of 𝑬 Step 3 Random permutation of 𝑬 𝒆 𝒊 𝒆 𝟏 𝒆 𝟐 … 𝑬 𝒊−𝟏

Random permutation of 𝑬 Step 3 Random permutation of 𝑬 𝑿 𝒊 : a random variable taking value 1 if 𝒆 𝒊 is a light edge with respect to MST( 𝑬 𝒊−𝟏 ). … 𝒆 𝒊 𝒆 𝟏 𝒆 𝟐 𝑬 𝒊−𝟏 𝑬\ 𝑬 𝒊−𝟏

Random permutation of 𝑬 Step 3 Random permutation of 𝑬 𝑿 𝒊 : a random variable taking value 1 if 𝒆 𝒊 is a light edge with respect to MST( 𝑬 𝒊−𝟏 ). Question: What is relation between 𝒑 and 𝑿 𝒊 ’s? Answer: ?? … 𝒆 𝒊 𝒆 𝟏 𝒆 𝟐 𝑬 𝒊−𝟏 𝑬\ 𝑬 𝒊−𝟏 𝒑= 𝐏(𝑿 𝒌+𝟏 =𝟏).

Random permutation of 𝑬 Calculating 𝐏(𝑿 𝒊 =𝟏). Random permutation of 𝑬 𝑺 𝒊−𝟏 : set of all subsets of 𝑬 of size 𝒊−𝟏. 𝐏(𝑿 𝒊 =𝟏) = 𝒂∈ 𝑺 𝒊−𝟏 𝐏(𝑿 𝒊 =𝟏 𝑬 𝒊−𝟏 =𝒂 ∙𝐏( 𝑬 𝒊−𝟏 =𝒂)  depends upon 𝒂 … 𝒆 𝒊 𝒆 𝟏 𝒆 𝟐 𝑬 𝒊−𝟏 Forward analysis MST(𝒂)

Random permutation of 𝑬 Calculating 𝐏(𝑿 𝒊 =𝟏). Random permutation of 𝑬 𝑺 𝒊 : set of all subsets of 𝑬 of size 𝒊. 𝐏(𝑿 𝒊 =𝟏)= 𝒂∈ 𝑺 𝒊−𝟏 𝐏(𝑿 𝒊 =𝟏 𝑬 𝒊 =𝒂 ∙𝐏( 𝑬 𝒊 =𝒂) … 𝒆 𝒊 𝒆 𝟏 𝒆 𝟐 𝑬 𝒊 Backward analysis

Random permutation of 𝑬 𝐏(𝑿 𝒊 =𝟏 𝑬 𝒊 =𝒂 Random permutation of 𝑬 𝐏(𝑿 𝒊 =𝟏 𝑬 𝒊 =𝒂 = ?? = ?? ≤ 𝒏−1 𝒊 … 𝒆 𝒊 𝒆 𝟏 𝒆 𝟐 Use Lemma 2. 𝑬 𝒊 Backward analysis MST(𝒂) 𝐏 𝒊th edge 𝐛𝐞𝐥𝐨𝐧𝐠𝐬 to MST(𝒂) |MST(𝒂)| 𝒊

Random permutation of 𝑬 Calculating 𝐏(𝑿 𝒊 =𝟏) Random permutation of 𝑬 𝑺 𝒊 : set of all subsets of 𝑬 of size 𝒊. 𝐏(𝑿 𝒊 =𝟏)= 𝒂∈ 𝑺 𝒊−𝟏 𝐏(𝑿 𝒊 =𝟏 𝑬 𝒊 =𝒂 ∙𝐏( 𝑬 𝒊 =𝒂) ≤ 𝒂∈ 𝑺 𝒊−𝟏 𝒏−1 𝒊 𝐏( 𝑬 𝒊 =𝒂) = 𝒏−1 𝒊 𝒂∈ 𝑺 𝒊−𝟏 𝐏( 𝑬 𝒊 =𝒂) = 𝒏−1 𝒊 … 𝒆 𝒊 𝒆 𝟏 𝒆 𝟐 𝑬 𝒊 Backward analysis

Combining the two methods for calculating 𝒑 Using method 1: 𝒑 = 𝟏 𝒎−𝒌 𝐄[𝒁] Using method 2: 𝒑 = 𝐏(𝑿 𝒌+𝟏 =𝟏) < 𝒏−1 𝒌+𝟏 Hence: 𝐄[𝒁] < 𝒏−1 𝒌+𝟏 (𝒎−𝒌) ≤ 𝒏 𝒌 (𝒎−𝒌)

Theorem: If we sample 𝒌 edges uniformly randomly from an undirected graph on 𝒏 vertices and 𝒎 edges, the number of light edges among the unsampled edges will be less than 𝒏 𝒌 (𝒎−𝒌) on expectation.