Mathematical Analysis of Algorithms 刘豪 瞿凡轶 徐逸飞 2019/5/14

Table of content Introduction to Donald Ervin Knuth The background In situ permutation 2019/5/14

Donald Ervin Knuth 高德纳（姚储枫教授,姚期智） January 10, 1938,农历牛年，摩羯座 American computer scientist, mathematician, and professor emeritus at Stanford University The Art of Computer Programming the rigorous analysis of the computational complexity of algorithms and systematized formal mathematical techniques for it Tex and METAFONT The term "analysis of algorithms" 5/14/2019

Donald Ervin Knuth First ACM Grace Murray Hopper Award, 1971 Turing Award, 1974 Lester R. Ford Award, 1975 and 1993 National Medal of Science, 1979 Franklin Medal, 1988 John von Neumann Medal, 1995 Kyoto Prize, 1996 … 5/14/2019

2019/5/14

2019/5/14

2019/5/14

2019/5/14

Introduction 2019/5/14

Two general kinds of problems Type A. Analysis of a particular algorithm Important characteristics of a certain algorithm Frequency analysis Storage analysis Predict the execution time of various algorithms for sorting numbers Used since the early days of computer programming Type B. Analysis of a class of algorithms Entire family of algorithms solving one particular problem Identify a “best possible” one Find the bounds of computational complexity Estimate the minimum number of operations later 2019/5/14

Is type B far superior to type A? 2019/5/14 It seems so Handle infinitely many algorithms at once We would prefer to prove once and find out a “best possible” one But True only to a limited extent Extremely technology-dependent: even slight changes may make a big difference 𝑥 31 needs at least 9 multiplications .But once division allowed ,6 ops Bubble sorting is optimal for a certain class of Demuth’s automata Nonsense Is type B far superior to type A?

Two main reasons why B do not supersede A 2019/5/14 For type B ,a rather simple model of the complexity is required Even with simple models , still too difficult Comparatively few problems have been solved Computing 𝑥 𝑛 is far from being thoroughly understood Comparison counting is not a good way to rate a sorting algorithm Two main reasons why B do not supersede A

Thus Computational complexity in CS number theory in Math Type B is not superior to A. Type A may be more important in practice All the relevant factors can be measured Not so sensitive to changes in technology Two algorithms will be analyzed in detail In Situ Permutation Selecting the 𝑡th Largest 2019/5/14

In situ permutation 2019/5/14

Description Goal Input Replacing a one-dimensional array 𝑥 1 , 𝑥 2 ,…, 𝑥 𝑛 by 𝑥 𝑝 1 , 𝑥 𝑝 2 ,…, 𝑥 𝑝 𝑛 Space complexity O(1) Input A one-dimensional array 𝑥 1 , 𝑥 2 ,…, 𝑥 𝑛 A function 𝑝. For any 𝑘,we cannot assign a new value to 𝑝 𝑘 𝑝 might be the function corresponding to the transposition of a matrix 2019/5/14

Definition A set 𝐴 is called a cycle if and only if ∀𝑎,𝑏∈𝐴,𝑝 𝑎 ,𝑝 𝑏 ∈𝐴∧∃𝑘∈ 𝑁 𝑝 𝑘 𝑎 =𝑏 .𝐴𝑛𝑑 min⁡(𝐴)is called the cycle leader of 𝐴 To get all the elements in a cycle in place,only elements of this cycle will be influenced To get the whole array replaced means make all the cycles replaced For example,(𝑝(1),𝑝(2),…,𝑝(9))=(8,2,7,1,6,9,3,4,5) {1,8,4},{2},{5,6,9},{7,3} 2019/5/14

Don’t be so strict If 𝑝 1 ,𝑝 2 ,…,𝑝 𝑛 is in a read/write memory First ,traverse 𝑝. Let 𝑝 𝑖 =𝑥[𝑝(𝑖)] Then ,traverse 𝑥.Let 𝑥 𝑖 =𝑝(𝑖) If 𝑛 extra tag bits were allowed All bits set to 0 at the very beginning, each time we assign a new value for 𝑥[𝑖],set 𝑏𝑖𝑡(𝑖) to 1 Find out the first 𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛 𝑗 whose indicating bit equals 0 .let 𝑖=𝑗,𝑡𝑚𝑝= 𝑥[𝑗] let 𝑥[𝑖]=𝑥[𝑝(𝑖)] 𝑖=𝑝(𝑖), repeat the last step until 𝑝(𝑖)=𝑗 Then let 𝑥[𝑖]=𝑡𝑚𝑝. 2019/5/14

J.C.Gower’s procedure 5/14/2019

Frequency analysis 9 statements and 3 conditions 12 separate problems? Kirchhoff’s law to the rescue The number of times we get to any given place in the program is the number of times we leave it No goto statements Frequency analysis 5/14/2019

Interpret the remaining unknowns 𝑛 equals the length of array 𝑥 𝑏 is the number of cycles in 𝑝 𝑐+𝑏=𝑛.Each element of 𝑥 is assigned a new value once and only once. 𝑎, the sum of “distances” from 𝑗 to the first element of (𝑝 𝑗 ,𝑝 𝑝 𝑗 …)that is <𝑗 Interpret the remaining unknowns 5/14/2019

Worst case (𝑝(1),𝑝(2),…,𝑝(𝑛))=(2,…,𝑛,1) 𝑎= 𝑛−1 + 𝑛−2 +…+0= ( 𝑛 2 −𝑛)/2 𝑏=1 Worst for a but best for b (𝑝(1),𝑝(2),…,𝑝(𝑛))=(1,2,…,𝑛) 𝑎=0 𝑏=𝑛 Worst for b but best for a 2019/5/14

Average case What does average case mean? Typical input distributions are not always easy to specify For this one, we assume each of the 𝑛! permutations of 𝑝 is equally likely 2019/5/14

A useful transformation of permutations The rules The leader comes first in each cycle The leaders of different cycles sre in decreasing order from left to right For example (p(1),p(2),…,p(9))=(8,2,7,1,6,9,3,4,5) {1,8,4},{2},{5,6,9},{7,3} (596)(37)(2)(184) (5,9,6,3,7,2,1,8,4) Properties Parentheses are not necessary A one-to-one map 2019/5/14

Interpret 𝑏 again 𝑏 is the number of cycles in 𝑝 𝑏 is the number of the indices 𝑗 such that 𝑞(𝑗)=min⁡{𝑞(𝑖)|1≤𝑖≤𝑗} The number of permutations with 𝑘 left-to- right minimas is [ 𝑘 𝑛 ] Average value= 𝐻 𝑛 Variance is 𝐻 𝑛 − 𝐻 𝑛 (2) 𝐻 𝑛 =1+ 1 2 +…+ 1 𝑛 𝐻 𝑛 2 =1+ 1 4 +…+ 1 𝑛 2 2019/5/14

Interpret 𝑎 When 𝑗 takes on 𝑞(𝑖),it will take on the successive values 𝑞 𝑖+1 ,𝑞 𝑖+2 ,… until reaching a value 𝑞(𝑖+𝑟)<𝑞(𝑟) For fixed 𝑖, i<𝑗≤𝑛 𝑦 𝑖𝑗 is number of times line 6 of the program is performed 5/14/2019

For example (p(1),p(2),…,p(9))=(8,2,7,1,6,9,3,4,5) {1,8,4},{2},{5,6,9},{7,3} (596)(37)(2)(184) (5,9,6,3,7,2,1,8,4) 𝑦 12 = 𝑦 13 = 𝑦 23 = 𝑦 45 = 𝑦 78 = 𝑦 79 =1 All other y’s are 0 (2,1,0,1,0,0,2,0,0) 5/14/2019

Hence Let 𝑦 𝑖𝑗 be the average value of 𝑦 𝑖𝑗 ,as 𝑞 1 ,𝑞 2 ,…,𝑞 𝑛 ranges over all permutations This is the number of permutations with 𝑦 𝑖𝑗 =1,divided by 𝑛! So it is the probability that 𝑞(𝑖)= min⁡{𝑞(𝑘)|𝑖≤ 𝑘≤ 𝑗},namely 1/(𝑗−𝑖+1) 2019/5/14

Thank you! 5/14/2019