Master Theorem Yin Tat Lee CSE 421 Master Theorem Yin Tat Lee

Almost 1D Problem Partition each side of 𝐿 into 𝛿 2 × 𝛿 2 squares Claim: No two points lie in the same 𝛿 2 × 𝛿 2 box. Say we want to check if there is a point < 𝛿 close to point 31. Only need to check points with indices roughly 31. (coz we sorted). How many? Not too many coz every box only have 1 point.  29 30 31 28 26 25 49 27 >𝛿

Closest Pair (2 dimension) if(𝒏≤𝟐) return | 𝒑 𝟏 − 𝒑 𝟐 | Compute separation line 𝑳 such that half the points are on one side and half on the other side. 𝜹 𝟏 = Closest-Pair(left half) 𝜹 𝟐 = Closest-Pair(right half) 𝜹 = min( 𝜹 𝟏 , 𝜹 𝟐 ) Delete all points further than  from separation line L Sort remaining points p[1]…p[m] by y-coordinate. for 𝒊=𝟏,𝟐,⋯,𝒎 for k = 𝟏,𝟐,⋯,𝟏𝟏 if 𝒊+𝒌≤𝒎  = min(, distance(p[i], p[i+k])); return . }

Closest Pair Analysis Let 𝐷(𝑛) be the number of pairwise distance calculations in the Closest-Pair Algorithm 𝐷 𝑛 ≤ 1 if 𝑛=1 2𝐷 𝑛 2 +11 𝑛 o.w. ⇒𝐷 𝑛 =O(𝑛log 𝑛) BUT, that’s only the number of distance calculations What if we counted running time? 𝑇 𝑛 ≤ 1 if 𝑛=1 2𝑇 𝑛 2 +𝑂(𝑛 log 𝑛) o.w. ⇒𝐷 𝑛 =O(𝑛 log 2 𝑛)

Closest Pair (2 dimension) Improved if(𝒏≤𝟐) return | 𝒑 𝟏 − 𝒑 𝟐 | Compute separation line 𝑳 such that half the points are on one side and half on the other side. ( 𝜹 𝟏 , 𝒑 𝟏 ) = Closest-Pair(left half) ( 𝜹 𝟐 , 𝒑 𝟐 ) = Closest-Pair(right half) 𝜹 = min( 𝜹 𝟏 , 𝜹 𝟐 ) 𝒑 𝒔𝒐𝒓𝒕𝒆𝒅 = merge( 𝒑 𝟏 , 𝒑 𝟐 ) (merge sort it by y-coordinate) Let 𝒒 be points (ordered as 𝒑 𝒔𝒐𝒓𝒕𝒆𝒅 ) that is 𝜹 from line L. for 𝒊=𝟏,𝟐,⋯,𝒎 for k = 𝟏,𝟐,⋯,𝟏𝟏 if 𝒊+𝒌≤𝒎  = min(, distance(q[i], q[i+k])); return  and 𝒑 𝒔𝒐𝒓𝒕𝒆𝒅 . } 𝑇 𝑛 ≤ 1 if 𝑛=1 2𝑇 𝑛 2 +𝑂 𝑛 o.w. ⇒𝐷 𝑛 =𝑂(𝑛 log 𝑛)

Master Theorem Suppose 𝑇 𝑛 =𝑎 𝑇 𝑛 𝑏 +𝑐 𝑛 𝑘 for all 𝑛>𝑏. Then, If 𝑎< 𝑏 𝑘 then 𝑇 𝑛 =Θ 𝑛 𝑘 If 𝑎= 𝑏 𝑘 then 𝑇 𝑛 =Θ 𝑛 𝑘 log 𝑛 If 𝑎> 𝑏 𝑘 then 𝑇 𝑛 =Θ 𝑛 log 𝑏 𝑎 Works even if it is 𝑛 𝑏 instead of 𝑛 𝑏 . We also need 𝑎≥1,𝑏>1 , 𝑘≥0 and 𝑇 𝑛 =𝑂(1) for 𝑛≤𝑏.

Master Theorem Suppose 𝑇 𝑛 =𝑎 𝑇 𝑛 𝑏 +𝑐 𝑛 𝑘 for all 𝑛>𝑏. Then, If 𝑎< 𝑏 𝑘 then 𝑇 𝑛 =Θ 𝑛 𝑘 If 𝑎= 𝑏 𝑘 then 𝑇 𝑛 =Θ 𝑛 𝑘 log 𝑛 If 𝑎> 𝑏 𝑘 then 𝑇 𝑛 =Θ 𝑛 log 𝑏 𝑎 Example: For mergesort algorithm we have 𝑇 𝑛 =2𝑇 𝑛 2 +𝑂 𝑛 . So, 𝑘=1,𝑎= 𝑏 𝑘 and 𝑇 𝑛 =Θ(𝑛 log 𝑛)

Proving Master Theorem 𝑛 Problem size 𝑛/𝑏 𝑛/ 𝑏 2 𝑏 1 𝑇 𝑛 =𝑎𝑇(𝑛/𝑎)+𝑐 𝑛 𝑘 # probs 𝑎2 𝑎 1 𝑎𝑑 cost 𝑐 𝑛 𝑘 𝑎⋅𝑐 𝑛/𝑏 𝑘 𝑎 2 ⋅𝑐 𝑛/ 𝑏 2 𝑘 𝑎 𝑘 ⋅𝑐 𝑛/ 𝑏 𝑘 𝑘 𝑑= log 𝑏 𝑛 𝑎 𝑇 𝑛 = 𝑖=0 𝑑= log 𝑏 𝑛 𝑎 𝑖 𝑐 𝑛 𝑏 𝑖 𝑘

Master Theorem Suppose 𝑇 𝑛 =𝑎 𝑇 𝑛 𝑏 +𝑐 𝑛 𝑘 for all 𝑛>𝑏. Then, If 𝑎< 𝑏 𝑘 then 𝑇 𝑛 =Θ 𝑛 𝑘 If 𝑎= 𝑏 𝑘 then 𝑇 𝑛 =Θ 𝑛 𝑘 log 𝑛 If 𝑎> 𝑏 𝑘 then 𝑇 𝑛 =Θ 𝑛 log 𝑏 𝑎 # of problems increases slower than the decreases of cost. First term dominates. # of problems increases faster than the decreases of cost Last term dominates.

A Useful Identity Theorem: 1+𝑥+ 𝑥 2 +…+ 𝑥 𝑑 = 𝑥 𝑑+1 −1 𝑥−1 Proof: Let 𝑆=1+𝑥+ 𝑥 2 +…+ 𝑥 𝑑 Then, 𝑥𝑆=𝑥+ 𝑥 2 +…+ 𝑥 𝑑+1 So, 𝑥𝑆−𝑆= 𝑥 𝑑+1 −1 i.e., 𝑆 𝑥−1 = 𝑥 𝑑+1 −1 Therefore, 𝑆= 𝑥 𝑑+1 −1 𝑥−1 Corollary: 1+𝑥+ 𝑥 2 +…+ 𝑥 𝑑 = 𝑂 𝑥 1 if 𝑥<1 𝑑+1 if 𝑥=1 𝑂 𝑥 𝑥 𝑑+1 if 𝑥>1 𝑂 𝑥 means the hidden constant depends on 𝑥

Solve: 𝑇 𝑛 =𝑎𝑇 𝑛 𝑏 +𝑐 𝑛 𝑘 Corollary: 1+𝑥+ 𝑥 2 +…+ 𝑥 𝑑 = Θ 𝑥 1 if 𝑥<1 Θ 𝑑 if 𝑥=1 Θ 𝑥 𝑥 𝑑+1 if 𝑥>1 Going back, we have 𝑇 𝑛 = 𝑖=0 𝑑= log 𝑏 𝑛 𝑎 𝑖 𝑐 𝑛 𝑏 𝑖 𝑘 =𝑐 𝑛 𝑘 𝑖=0 𝑑= log 𝑏 𝑛 𝑎 𝑏 𝑘 𝑖 Hence, we have 𝑇 𝑛 =Θ 𝑛 𝑘 1 if 𝑎< 𝑏 𝑘 log 𝑏 𝑛 if 𝑎= 𝑏 𝑘 𝑎 𝑏 𝑘 log 𝑏 𝑛 if 𝑎> 𝑏 𝑘 constant depends on 𝑎,𝑏,𝑐 When we can hide constant?

Solve: 𝑇 𝑛 =𝑎𝑇 𝑛 𝑏 +𝑐 𝑛 𝑘 𝑇 𝑛 =Θ 𝑛 𝑘 1 if 𝑎< 𝑏 𝑘 log 𝑏 𝑛 if 𝑎= 𝑏 𝑘 𝑎 𝑏 𝑘 log 𝑏 𝑛 if 𝑎> 𝑏 𝑘 For 𝑎< 𝑏 𝑘 , we simply have 𝑇 𝑛 =Θ 𝑛 𝑘 . For 𝑎= 𝑏 𝑘 , we have 𝑇 𝑛 =Θ 𝑛 𝑘 log 𝑏 𝑛 =Θ( 𝑛 𝑘 log 𝑛 ). For 𝑎> 𝑏 𝑘 , we have 𝑇 𝑛 =Θ 𝑛 𝑘 𝑎 𝑏 𝑘 log 𝑏 𝑛 =Θ( 𝑛 log 𝑏 𝑎 ). 𝑎 log 𝑏 𝑛 = (𝑏 log 𝑏 𝑎 ) log 𝑏 𝑛 = (𝑏 log 𝑏 𝑛 ) log 𝑏 𝑎 = 𝑛 log 𝑏 𝑎 𝑏 𝑘 log 𝑏 𝑛 = 𝑏 log 𝑏 𝑛 𝑘 = 𝑛 𝑘

Example