1. Randomized Algorithms CS648
Lecture 17
Miscellaneous applications of Backward analysis

2. Minimum spanning tree

3. Minimum spanning tree b a c d h x y u v 18 7 1 19 22 10 3 12 15 11 516 17 13 4 2 9 6

4. Less known but it is the first algorithm for MST
Minimum spanning tree 16 d 17 9
Algorithms:
Prim’s algorithm
Kruskal’s algorithm
Boruvka’s algorithm v x h 4 2 5 19 6 11 y 3 3 22 c 12 a 18 1 15 10 u b 7 13
Less known but it is the first algorithm for MST

5. Minimum spanning tree 𝑮=(𝑽,𝑬) : undirected graph with weights on edges
𝒏=|𝑽|,
𝒎=|𝑬|.
Deterministic algorithms:
Prim’s algorithm
O((𝒎 + 𝒏) log 𝒏) using Binary heap
O(𝒎 + 𝒏 log 𝒏) using Fibonacci heap
Best deterministic algorithm:
O(𝒎 + 𝒏 𝐥𝐨𝐠 ∗ 𝒏) bound
Too complicated to design and analyze
Fails to beat Prim’s algorithm using Binary heap

6. Minimum spanning tree
When finding an efficient solution of a problem appears hard, one should strive to design an efficient verification algorithm.
MST verification algorithm: [King, 1990]
Given a graph 𝑮=(𝑽,𝑬) and a spanning tree 𝑻, it takes O(𝒎 + 𝒏) time to
determine if 𝑻 is MST of 𝑮.
Interestingly, no deterministic algorithm for MST could use this algorithm to achieve O(𝒎 + 𝒏) time.

7. Minimum spanning tree 𝑮=(𝑽,𝑬) : undirected graph with weights on edges
𝒏=|𝑽|,
𝒎=|𝑬|.
Randomized algorithm:
Karger-Klein-Tarjan’s algorithm [1995]
Las Vegas algorithm
O(𝒎 + 𝒏) expected time
This algorithm uses
Random sampling
MST verification algorithm
Boruvka’s algorithm
Elementary data structure

8. The notion of closeness is formalized in the following slide.
Minimum spanning tree
𝑮=(𝑽,𝑬) : undirected graph with weights on edges
𝒏=|𝑽|,
𝒎=|𝑬|.
Randomized algorithm:
Karger-Klein-Tarjan’s algorithm [1994]
Las Vegas algorithm
O(𝒎 + 𝒏) expected time
Random sampling :
How close is MST of a random sample of edges to MST of original graph ?
The notion of closeness is formalized in the following slide.

9. Light Edge b a c d h x y u v 18 7 1 19 22 10 3 12 15 11 5 16 17 13 4 2 9 6
Definition: Let 𝒂⊂𝑬. An edge 𝒆∈𝑬\𝒂 is said to be light with respect to 𝒂 if Question: If 𝒂 ⊂ 𝑟 𝑬 and |𝒂|= 𝒌, how many edges from 𝑬\𝒂 are light with respect to 𝒂 on expectation ? Answer: ?? 31
MST({𝒆}∪𝒂) ≠ MST(𝒂)
< 𝒏 𝒌 (𝒎−𝒌)

10. USING Backward analysis for Miscellaneous Applications

11. problem 1 SMALLEST Enclosing circle

12. Smallest Enclosing Circle

13. Smallest Enclosing Circle
Question: Suppose we sample 𝒌 points randomly uniformly from a set of 𝒏 points, what is the expected number of points that remain outside the smallest circle enclosing the sample?
For 𝒌= 𝒏 𝟐 , the answer is <𝟏𝟎

14. problem 2 smallest length interval

15. Sampling points from a unit interval
Question: Suppose we select 𝒌 points from interval [0,1], what is expected length of the smallest sub-interval ?
for 𝒌=1, it is ??
for 𝒌=2, it is ??
General solution : ??
This bound can be derived using two methods.
One method is based on establishing a relationship between uniform distribution and exponential distribution.
Second method (for nearly same asymptotic bound) using Backward analysis.
𝟏 𝟒
𝟏 𝟗
𝟏 𝒌+𝟏 𝟐 1

16. problem 3 Minimum spanning tree

17. Light Edge b a c d h x y u v 18 7 1 19 22 10 3 12 15 11 5 16 17 13 4 2 9 6
Definition: Let 𝒂⊂𝑬. An edge 𝒆∈𝑬\𝒂 is said to be light with respect to 𝒂 if Question: If 𝒂 ⊂ 𝑟 𝑬 and |𝒂|= 𝒌, how many edges from 𝑬\𝒂 are light with respect to 𝒂 on expectation ? 31
MST({𝒆}∪𝒂) ≠ MST(𝒂)

18. using Backward analysis for The 3 problems : A General framework

19. A General framework
Let 𝒁 be the desired random variable in any of these problems/random experiment.
Step 1: Define an event 𝜺 related to the random variable 𝒁.
Step 2: Calculate probability of event 𝜺 using standard method based on definition. (This establishes a relationship between )
Step 3: Express the underlying random experiment as a Randomized incremental construction and calculate the probability of the event 𝜺 using Backward analysis.
Step 4: Equate the expressions from Steps 1 and 2 to calculate E[𝒁].

20. problem 3 Minimum spanning tree

21. A Better understanding of light edges

22. Minimum spanning tree d (𝑽,𝑬) v x h y c a u b Random sampling x b a c16 d
(𝑽,𝑬) 17 19 v x h 4 2 5 19 6 31 11 y 3 3 22 c 42 a 18 1 15 10 u b 7 13
Random sampling x b a c d h y u v 18 1 19 22 10 15 11 5 16 4 31
(𝑽,𝑹)

23. Minimum spanning tree d (𝑽,𝑬) v x h y c a u b d MST(𝑹) (𝑽,𝑹) v x h y c16 d
(𝑽,𝑬) 17 19 v x h 4 2 5 19 6 31 11 y 3 3 22 c 42 a 18 1 15 10 u b 7 13 16 d
MST(𝑹)
(𝑽,𝑹) v x h 4 5 18 22 31 19 11 y c a 1 15 10 u b

24. Minimum spanning tree d (𝑽,𝑬) v x h y c a u b d MST(𝑹) (𝑽,𝑹) v x h 𝑬\𝑹16 d
(𝑽,𝑬) 17 19 v x h 4 2 5 19 6 31 11 y 3 3 22 c 42 a 18 1 15 10 u b 7 23 16 d
MST(𝑹) 7 3 42 17 23 2 19 6
(𝑽,𝑹) v x h 4 5
𝑬\𝑹 19 11 y c a 1 15 10 u b

25. Minimum spanning tree d (𝑽,𝑬) v x h y c a u b d MST(𝑹) (𝑽,𝑹) v x h 𝑬\𝑹16 d
(𝑽,𝑬) 17 19 v x h 4 2 5 19 6 31 11 y 3 3 22 c 42 a 18 1 15 10 u b 7 23 16 d
MST(𝑹) 17 19
(𝑽,𝑹) v x h 4 2 5
𝑬\𝑹 19 6 11 y 3 3
Light c 42 a 1 15 10 u b 7 23

26. First useful insight
Lemma1: An edge 𝒆 is light with respect to 𝒂⊂𝑬 if and only if 𝒆 belongs to MST( 𝒆 ∪𝒂).

27. Minimum spanning tree d (𝑽,𝑬) v x h y c a u b d MST(𝑹) (𝑽,𝑹) v x h 𝑬\𝑹16 d
(𝑽,𝑬) 17 19 v x h 4 2 5 19 6 31 11 y 3 3 22 c 42 a 18 1 15 10 u b 7 23 16 d
MST(𝑹) 17 19
(𝑽,𝑹) v x h 4 2 5
𝑬\𝑹 19 6 11 y 3 3
Light
heavy c 42 a 1 15 10 u b 7 23

28. Minimum spanning tree MST(𝑬) d (𝑽,𝑬) v x h y c a u b4 2 5 6 y 3 3 c a 1 u b 7
Is there any relationship among MST(𝑬), MST(𝑹)
and Light edges from 𝑬\𝑹 ? 16 d
MST(𝑹) 17 42 23 19
(𝑽,𝑹) v x h 4 2 5
𝑬\𝑹 19 6 11 y 3 3
Light
heavy c a 1 15 10 u b 7

29. MST(𝑬) = MST(𝑳∪𝐌𝐒𝐓(𝒂))
Second useful insight
Lemma2: Let
𝒂⊂𝑬 and
𝑳 be the set of all edges from 𝑬\𝒂 that are light with respect to 𝒂.
Then
MST(𝑬) = MST(𝑳∪𝐌𝐒𝐓(𝒂))
This lemma is used in the design of randomized algorithm for MST as follows (just a sketch):
Compute MST of a sample of 𝒎/𝟐 edges (recursively). Let it be T’.
There will be expected 𝒏 edges light edges among all unsampled edges.
Recursively compute MST of 𝑳∪ T’ edges which are less than 2𝒏 on expectation.

30. Light Edge b a c d h x y u v 18 7 1 19 22 10 3 12 15 11 5 16 17 13 4 2 9 6
Definition: Let 𝒂⊂𝑬. An edge 𝒆∈𝑬\𝒂 is said to be light with respect to 𝒂 if Question: If 𝒂 ⊂ 𝑟 𝑬 and |𝒂|= 𝒌, how many edges from 𝑬\𝒂 are light with respect to 𝒂 on expectation ? 31
MST({𝒆}∪𝒂) ≠ MST(𝒂)
We shall answer the above question using the Generic framework. But before that, we need to get a better understanding of the corresponding random variable.

31. 𝒌 𝑹
MST(𝑹)
𝑬\𝑹

32. 𝒌 𝑹
MST(𝑹)
𝑬\𝑹
Light

33. 𝒌 𝑹
MST(𝑹)
𝑬\𝑹
Light
heavy

34. Can you express 𝐄[𝒁] in terms of 𝒇 𝒂 and 𝑺 only ?
𝒁: random variable for the number of light edges in 𝑬\𝑹 when 𝑹 is a random sample of 𝒌 edges. 𝑺: set of all subsets of 𝑬 of size 𝒌. 𝒇 𝒂 : number of light edges in 𝑬\𝑹 when 𝑹=𝒂. 𝐄[𝒁] = ??
Can you express 𝐄[𝒁] in terms of 𝒇 𝒂 and 𝑺 only ?
𝟏 |𝑺| 𝒂∈𝑺 𝒇 𝒂

35. Step 1
Question: Let 𝑹 be a uniformly random sample of 𝒌 edges from 𝑬. What is the prob. 𝒑 that an edge selected randomly from 𝑬\𝑹 is a light edge ?
Two methods to find 𝒑

36. Calculating 𝒑 using definition
Step 2
Calculating 𝒑 using definition

37. Calculating 𝒑 using definition
Step 2
Calculating 𝒑 using definition 𝒌
Light edges
=𝒇 𝒂 𝒂
MST(𝒂)
𝑬\𝒂
Light
heavy

38. Calculating 𝒑 using definition
Step 2
Calculating 𝒑 using definition
𝑺: set of all subsets of 𝑬 of size 𝒌.
The probability 𝒑 is equal to
= ? 𝒂∈𝑺 ?
= 𝟏 𝒎−𝒌 𝟏 |𝑺| 𝒂∈𝑺 𝒇 𝒂
= 𝟏 𝒎−𝒌 𝐄[𝒁]
𝟏 |𝑺|
𝒇 𝒂 𝒎−𝒌

39. Spend some time on this question before proceeding further.
Step 3
Expressing the entire experiment as Randomized Incremental Construction
A slight difficulty in this process is the following:
The underlying experiment talks about random sample from a set.
But RIC involves analyzing a random permutation of a set of elements. 
Question: What is relation between random sample from a set and a random permutation of the set ?
Spend some time on this question before proceeding further.

40. random sample and random permutation
Random permutation of 𝑬
Observation:
𝑨 is indeed a uniformly random sample of 𝑬 𝑨
𝑬\𝑨 𝒓
𝒎−𝒓

41. Spend some time on this question before proceeding further.
Step 3
The underlying random experiment as Randomized Incremental Construction:
Permute the edges randomly uniformly.
Find the probability that 𝒊th edge is light relative to the first 𝒊−𝟏 edges.
Question: Can you now calculate probability 𝒑 ?
Spend some time on this question before proceeding further.

42. Random permutation of 𝑬
Step 3
Random permutation of 𝑬
𝒆 𝒊
𝒆 𝟏
𝒆 𝟐 …
𝑬 𝒊−𝟏

43. Random permutation of 𝑬
Step 3
Random permutation of 𝑬
𝑿 𝒊 : a random variable taking value 1 if 𝒆 𝒊 is a light edge with respect to MST( 𝑬 𝒊−𝟏 ). …
𝒆 𝒊
𝒆 𝟏
𝒆 𝟐
𝑬 𝒊−𝟏
𝑬\ 𝑬 𝒊−𝟏

44. Random permutation of 𝑬
Step 3
Random permutation of 𝑬
𝑿 𝒊 : a random variable taking value 1 if 𝒆 𝒊 is a light edge with respect to MST( 𝑬 𝒊−𝟏 ). Question: What is relation between 𝒑 and 𝑿 𝒊 ’s? Answer: ?? …
𝒆 𝒊
𝒆 𝟏
𝒆 𝟐
𝑬 𝒊−𝟏
𝑬\ 𝑬 𝒊−𝟏
𝒑= 𝐏(𝑿 𝒌+𝟏 =𝟏).

45. Random permutation of 𝑬
Calculating 𝐏(𝑿 𝒊 =𝟏).
Random permutation of 𝑬
𝑺 𝒊−𝟏 : set of all subsets of 𝑬 of size 𝒊−𝟏. 𝐏(𝑿 𝒊 =𝟏) = 𝒂∈ 𝑺 𝒊−𝟏 𝐏(𝑿 𝒊 =𝟏 𝑬 𝒊−𝟏 =𝒂 ∙𝐏( 𝑬 𝒊−𝟏 =𝒂)  depends upon 𝒂 …
𝒆 𝒊
𝒆 𝟏
𝒆 𝟐
𝑬 𝒊−𝟏
Forward analysis
MST(𝒂)

46. Random permutation of 𝑬
Calculating 𝐏(𝑿 𝒊 =𝟏).
Random permutation of 𝑬
𝑺 𝒊 : set of all subsets of 𝑬 of size 𝒊. 𝐏(𝑿 𝒊 =𝟏)= 𝒂∈ 𝑺 𝒊−𝟏 𝐏(𝑿 𝒊 =𝟏 𝑬 𝒊 =𝒂 ∙𝐏( 𝑬 𝒊 =𝒂) …
𝒆 𝒊
𝒆 𝟏
𝒆 𝟐
𝑬 𝒊
Backward analysis

47. Random permutation of 𝑬
𝐏(𝑿 𝒊 =𝟏 𝑬 𝒊 =𝒂
Random permutation of 𝑬
𝐏(𝑿 𝒊 =𝟏 𝑬 𝒊 =𝒂
= ??
= ??
≤ 𝒏−1 𝒊 …
𝒆 𝒊
𝒆 𝟏
𝒆 𝟐
Use Lemma 2.
𝑬 𝒊
Backward analysis
MST(𝒂)
𝐏 𝒊th edge 𝐛𝐞𝐥𝐨𝐧𝐠𝐬 to MST(𝒂)
|MST(𝒂)| 𝒊

48. Random permutation of 𝑬
Calculating 𝐏(𝑿 𝒊 =𝟏)
Random permutation of 𝑬
𝑺 𝒊 : set of all subsets of 𝑬 of size 𝒊.
𝐏(𝑿 𝒊 =𝟏)=
𝒂∈ 𝑺 𝒊−𝟏 𝐏(𝑿 𝒊 =𝟏 𝑬 𝒊 =𝒂 ∙𝐏( 𝑬 𝒊 =𝒂)
≤ 𝒂∈ 𝑺 𝒊−𝟏 𝒏−1 𝒊 𝐏( 𝑬 𝒊 =𝒂)
= 𝒏−1 𝒊 𝒂∈ 𝑺 𝒊−𝟏 𝐏( 𝑬 𝒊 =𝒂)
= 𝒏−1 𝒊 …
𝒆 𝒊
𝒆 𝟏
𝒆 𝟐
𝑬 𝒊
Backward analysis

49. Combining the two methods for calculating 𝒑
Using method 1: 𝒑 = 𝟏 𝒎−𝒌 𝐄[𝒁] Using method 2: 𝒑 = 𝐏(𝑿 𝒌+𝟏 =𝟏) < 𝒏−1 𝒌+𝟏 Hence: 𝐄[𝒁] < 𝒏−1 𝒌+𝟏 (𝒎−𝒌) ≤ 𝒏 𝒌 (𝒎−𝒌)

50. Theorem: If we sample 𝒌 edges uniformly randomly from an undirected graph on 𝒏 vertices and 𝒎 edges, the number of light edges among the unsampled edges will be less than 𝒏 𝒌 (𝒎−𝒌) on expectation.