Foundations of Algorithm 유관우 알고리즘 설계 및 분석 Foundations of Algorithm 유관우

Chap1. 알고리즘 설계 및 분석 (Design & Analysis of Computer Algorithms) Motivations “The most important course for CS students.” Goal - “생각” Problem solving techniques Scientific approach Observation, Analysis, Organization → Conclusion Correct thinking habits Logical writing (“The Golden Rule”) Big know-How’s (NP-Completeness …) Presentation techniques “그러므로 무엇이든지 남에게 대접을 받고자 하는 대로 너희도 남들 대접하라. 이것이 율법이요 선지자니라.” – 마태7:12 Digital Media Lab.

Problem : A question to which we seek an answer. Answer Algorithm_Lecture Problem : A question to which we seek an answer. Answer Solution for CS problems : A step-by-step procedure → Algorithm (Eg) problem : Determine whether instance : answer → yes Algorithm1 : Sequential Search Procedure seqsearch ( n : integer; S : array[1..n] of keytype x : keytype; var location : index); begin location := 1; while location ≤ n and S[location] ≠ x do location := location + 1; if location > n then location := 0; end; Digital Media Lab. Digital Media Lab.

Algorithm2 : Binary Search Procedure binsearch ( n : integer; S : array[1..n] of keytype; x : keytype; var location : index); var low, high, mid : index; begin low = 1; high = n; location = 0; while low ≤ high and location = 0 do mid = ( low + high ) / 2; if x = S[mid] then location = mid; else if x < S[mid] then high = mid – 1; else low = mid + 1; end; 2 different techniques for same problem → Efficiency? time : space : Digital Media Lab.

Fibonacci Sequence Algorithm 1 (Recursion) function fib (n : integer) : integer; begin if n ≤ 1 then fib=n else fib = fib(n-1) + fib(n-2); end; n #comp(Seq. Srch) #comp(Bin. Srch) 128 8 1,024 11 1,048,576 21 4,294,967,296 33 Digital Media Lab.

Divide-and-conquer approach Binary Search : no overlap Fibonacci sequence : full of overlapping ƒ(5) ƒ(3) ƒ(4) ƒ(1) ƒ(2) ƒ(2) ƒ(3) ƒ(0) ƒ(1) ƒ(0) ƒ(1) ƒ(1) ƒ(2) ƒ(0) ƒ(1) Digital Media Lab.

proof : Induction on n Theorem <Induction Basis> n = 2 & n = 3 <Induction Hypothesis> Suppose for all m such that 2 ≤ m < n T(m) > 2m/2 <Induction Step> T(n) = T(n-1) + T(n-2) + 1 > 2(n-1)/2 + 2(n-2)/2 + 1 > 2(n-2)/2 + 2(n-2)/2 = 2·2(n-2)/2 = 2n/2 Digital Media Lab.

Algorithm 2 (Iteration) function fib2 (n : integer) : integer; var f : array[0..n] of integer; i : index; begin (n+1) terms for fib2(n) f[0] = 0; if n > 0 then f[1] = 1; for i = 2 to n do f[i] = f[I – 1] + f[I – 2] fib2 = f[n] end; n (n+1) Iterative Alg Recursive Alg 40 41 ns 1048 micro s. 60 61 ns 1 s 80 81 ns 18 min 100 101 ns 13 days 120 121 ns 36 years 160 161 ns 3.8 ×107 years 200 201 ns 4 ×1013 years Digital Media Lab.

Algorithm Design : techniques (D-&-C, Dynamic Programming, Greedy, Backtracking, Branch-&-Bound, … ) Several algorithms for the same problem. “Which is the most Efficient algorithm?” => Algorithm Analysis ♧ Note : problem analysis (solvable, unsolvable, NP-Complete, … ) Digital Media Lab.

Time complexity Analysis Actual CPU time × # instructions × (programmers, languages, …) → some independent measure ! # op’s : a function of input size n n+1, 2n/2, log n … Input size? (Eg) Graph 문제들: O(n+e), 소수결정문제: O(log n), 행렬곱: O(n2)… Time complexity analysis Determine input size Choose the basic operation Determine how many time the B.O. is done for each value of the input size Digital Media Lab.

Example Add array Members T(n) = n (n : array size) Matrix multiplication T(n) = n3 (n : # rows) “every-case time complexity” Sequential Search Worst-case t.c. T(n) = n = W(n) Best-case t.c B(n) = 1 × Average-case t.c. A(n) Digital Media Lab.

(Eg) Sorting n elements Insertion Sort every-case complexity × best-case : worst-case : average-case : Quick Sort best-case, avg-case : “practically best but not appropriate for real time applications.” Merge Sort, Heap Sort every-case t.c. : Digital Media Lab.

Space complexity (memory) Complexity function Time complexity Space complexity (memory) Complexity function Note : 2 algorithms for the same prob. 1000·n , n2 Which one is faster? (or better? ) n2 > 1000·n if n > 1000 Threshold : 1000 But, theoretically 1000·n is better Correctness analysis, Correctness proof Digital Media Lab.

Order 1000·n is better than 0.01·n2 if n > 100,000 Linear-time algorithm quadratic-time algorithm Quadratic term eventually dominates “Throw away” low-order terms “ algorithm” or “quadratic algorithm.” Fig 1.3 & Table 1.4 p.28 & p.29 n 0.1n2 0.1n2+n+100 10 120 20 40 160 50 250 400 100 1000 1200 100000 101100 Digital Media Lab.

Digital Media Lab.

Digital Media Lab.

Digital Media Lab.

Def) (“Big-Oh of ƒ of n”) but “asymptotic upper bound” : big-Oh O(n2) “at least as good as” 3lgn+8, 4n2 5n+7, 6n2+9 2nlgn, 5n2+2 Digital Media Lab.

Def) (“Omega of ƒ(n)”) but Ω(n2) “at least as bad as” 4n2 4n3+3n2 5n2+2n 2n+4n Digital Media Lab.

Def) (“Theta of ƒ(n)”, “order of ƒ(n)”) (Eg) Prove that (proof by contradiction) Assume that Then This implies Since is a constant, this can never be True. → Contradiction 3lgn+8 4n2 5n+7 6n2+9 2nlgn 5n2+2n 4n3+3n2 6n6+n4 2n+4n Digital Media Lab.

Def) (Small Oh of ƒ(n)) (Eg) Let be given Choose Then (By contradiction) Assume Let Then must exist Digital Media Lab.

Theorem : if , then (proof) Because Next, we prove by contradiction. Digital Media Lab.

Properties of Order : (Eg) 1. 2. 3. 상수 4. 5. 6. 3. 상수 4. 5. 6. strictly growing functions 7. (Eg) Digital Media Lab.

In practice, exact time comp. → not easy or : enough Note : instead of Theorem (Eg) , because Digital Media Lab.

(Eg) , because (Eg) (i) , Trivial (ii) , Let be big enough Digital Media Lab.

Theorem (L’Hôpital Rule) (Eg) Digital Media Lab.