1. Mathematical Analysis of Algorithms
刘豪 瞿凡轶 徐逸飞
2019/5/14

2. Table of content Introduction to Donald Ervin Knuth The background
In situ permutation
2019/5/14

3. Donald Ervin Knuth 高德纳（姚储枫教授,姚期智） January 10, 1938,农历牛年，摩羯座
American computer scientist, mathematician, and professor emeritus at Stanford University
The Art of Computer Programming
the rigorous analysis of the computational complexity of algorithms and systematized formal mathematical techniques for it
Tex and METAFONT
The term "analysis of algorithms"
5/14/2019

4. Donald Ervin Knuth First ACM Grace Murray Hopper Award, 1971
Turing Award, 1974
Lester R. Ford Award, 1975 and 1993
National Medal of Science, 1979
Franklin Medal, 1988
John von Neumann Medal, 1995
Kyoto Prize, 1996 …
5/14/2019

5. 2019/5/14

6. 2019/5/14

7. 2019/5/14

8. 2019/5/14

9. Introduction
2019/5/14

10. Two general kinds of problems
Type A. Analysis of a particular algorithm
Important characteristics of a certain algorithm
Frequency analysis
Storage analysis
Predict the execution time of various algorithms for sorting numbers
Used since the early days of computer programming
Type B. Analysis of a class of algorithms
Entire family of algorithms solving one particular problem
Identify a “best possible” one
Find the bounds of computational complexity
Estimate the minimum number of operations
later
2019/5/14

11. Is type B far superior to type A?
2019/5/14
It seems so
Handle infinitely many algorithms at once
We would prefer to prove once and find out a “best possible” one
But
True only to a limited extent
Extremely technology-dependent: even slight changes may make a big difference
𝑥 31 needs at least 9 multiplications .But once division allowed ,6 ops
Bubble sorting is optimal for a certain class of Demuth’s automata
Nonsense
Is type B far superior to type A?

12. Two main reasons why B do not supersede A
2019/5/14
For type B ,a rather simple model of the complexity is required
Even with simple models , still too difficult
Comparatively few problems have been solved
Computing 𝑥 𝑛 is far from being thoroughly understood
Comparison counting is not a good way to rate a sorting algorithm
Two main reasons why B do not supersede A

13. Thus Computational complexity in CS number theory in Math
Type B is not superior to A. Type A may be more important in practice
All the relevant factors can be measured
Not so sensitive to changes in technology
Two algorithms will be analyzed in detail
In Situ Permutation
Selecting the 𝑡th Largest
2019/5/14

14. In situ permutation
2019/5/14

15. Description Goal Input
Replacing a one-dimensional array 𝑥 1 , 𝑥 2 ,…, 𝑥 𝑛 by 𝑥 𝑝 1 , 𝑥 𝑝 2 ,…, 𝑥 𝑝 𝑛
Space complexity O(1)
Input
A one-dimensional array 𝑥 1 , 𝑥 2 ,…, 𝑥 𝑛
A function 𝑝.
For any 𝑘,we cannot assign a new value to 𝑝 𝑘
𝑝 might be the function corresponding to the transposition of a matrix
2019/5/14

16. Definition
A set 𝐴 is called a cycle if and only if ∀𝑎,𝑏∈𝐴,𝑝 𝑎 ,𝑝 𝑏 ∈𝐴∧∃𝑘∈ 𝑁 𝑝 𝑘 𝑎 =𝑏 .𝐴𝑛𝑑 min⁡(𝐴)is called the cycle leader of 𝐴
To get all the elements in a cycle in place,only elements of this cycle will be influenced
To get the whole array replaced means make all the cycles replaced
For example,(𝑝(1),𝑝(2),…,𝑝(9))=(8,2,7,1,6,9,3,4,5)
{1,8,4},{2},{5,6,9},{7,3}
2019/5/14

17. Don’t be so strict If 𝑝 1 ,𝑝 2 ,…,𝑝 𝑛 is in a read/write memory
First ,traverse 𝑝. Let 𝑝 𝑖 =𝑥[𝑝(𝑖)]
Then ,traverse 𝑥.Let 𝑥 𝑖 =𝑝(𝑖)
If 𝑛 extra tag bits were allowed
All bits set to 0 at the very beginning, each time we assign a new value for 𝑥[𝑖],set 𝑏𝑖𝑡(𝑖) to 1
Find out the first 𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛 𝑗 whose indicating bit equals 0 .let 𝑖=𝑗,𝑡𝑚𝑝= 𝑥[𝑗]
let 𝑥[𝑖]=𝑥[𝑝(𝑖)] 𝑖=𝑝(𝑖),
repeat the last step until 𝑝(𝑖)=𝑗
Then let 𝑥[𝑖]=𝑡𝑚𝑝.
2019/5/14

18. J.C.Gower’s procedure
5/14/2019

19. Frequency analysis 9 statements and 3 conditions 12 separate problems?
Kirchhoff’s law to the rescue
The number of times we get to any given place in the program is the number of times we leave it
No goto statements
Frequency analysis
5/14/2019

20. Interpret the remaining unknowns
𝑛 equals the length of array 𝑥
𝑏 is the number of cycles in 𝑝
𝑐+𝑏=𝑛.Each element of 𝑥 is assigned a new value once and only once.
𝑎, the sum of “distances” from 𝑗 to the first element of (𝑝 𝑗 ,𝑝 𝑝 𝑗 …)that is <𝑗
Interpret the remaining unknowns
5/14/2019

21. Worst case (𝑝(1),𝑝(2),…,𝑝(𝑛))=(2,…,𝑛,1) 𝑎= 𝑛−1 + 𝑛−2 +…+0= ( 𝑛 2 −𝑛)/2
𝑏=1
Worst for a but best for b
(𝑝(1),𝑝(2),…,𝑝(𝑛))=(1,2,…,𝑛)
𝑎=0
𝑏=𝑛
Worst for b but best for a
2019/5/14

22. Average case What does average case mean?
Typical input distributions are not always easy to specify
For this one, we assume each of the 𝑛! permutations of 𝑝 is equally likely
2019/5/14

23. A useful transformation of permutations
The rules
The leader comes first in each cycle
The leaders of different cycles sre in decreasing order from left to right
For example
(p(1),p(2),…,p(9))=(8,2,7,1,6,9,3,4,5)
{1,8,4},{2},{5,6,9},{7,3}
(596)(37)(2)(184)
(5,9,6,3,7,2,1,8,4)
Properties
Parentheses are not necessary
A one-to-one map
2019/5/14

24. Interpret 𝑏 again 𝑏 is the number of cycles in 𝑝
𝑏 is the number of the indices 𝑗 such that 𝑞(𝑗)=min⁡{𝑞(𝑖)|1≤𝑖≤𝑗}
The number of permutations with 𝑘 left-to- right minimas is [ 𝑘 𝑛 ]
Average value= 𝐻 𝑛
Variance is 𝐻 𝑛 − 𝐻 𝑛 (2)
𝐻 𝑛 = …+ 1 𝑛
𝐻 𝑛 2 = …+ 1 𝑛 2
2019/5/14

25. Interpret 𝑎
When 𝑗 takes on 𝑞(𝑖),it will take on the successive values 𝑞 𝑖+1 ,𝑞 𝑖+2 ,… until reaching a value 𝑞(𝑖+𝑟)<𝑞(𝑟)
For fixed 𝑖, i<𝑗≤𝑛 𝑦 𝑖𝑗 is number of times line 6 of the program is performed
5/14/2019

26. For example (p(1),p(2),…,p(9))=(8,2,7,1,6,9,3,4,5)
{1,8,4},{2},{5,6,9},{7,3}
(596)(37)(2)(184)
(5,9,6,3,7,2,1,8,4)
𝑦 12 = 𝑦 13 = 𝑦 23 = 𝑦 45 = 𝑦 78 = 𝑦 79 =1
All other y’s are 0
(2,1,0,1,0,0,2,0,0)
5/14/2019

27. Hence
Let 𝑦 𝑖𝑗 be the average value of 𝑦 𝑖𝑗 ,as 𝑞 1 ,𝑞 2 ,…,𝑞 𝑛 ranges over all permutations
This is the number of permutations with 𝑦 𝑖𝑗 =1,divided by 𝑛!
So it is the probability that 𝑞(𝑖)= min⁡{𝑞(𝑘)|𝑖≤ 𝑘≤ 𝑗},namely 1/(𝑗−𝑖+1)
2019/5/14

28. Thank you!
5/14/2019