1. Basic Concepts 2014, Fall Pusan National University Ki-Joune Li

2. STEMPNU Refrigerator Problem  If we have only one item in my refrigerator, no problem.  If we have several items in a refrigerator, it does matter.  Some Organization or Structures Data Structures  How to place data in memory 2 Data Structures ?

3. STEMPNU 3 What is good data structure? What is good placement in refrigerator ?  Easy to cook (and place) What is good data structure ?  Easy (and efficient) to use  It depends on what you want to do with the data structures  Two different aspects Functions and Internal Implementations

4. STEMPNU Example: Suppose we have 1,000,000 integer values, then what the difference between  Array and  Stack ? ADT 4 Two viewpoints and Abstract Data Types Implementation  Data Structure + Algorithm Implementation  Data Structure + Algorithm getElement(i) putElement(i,v) number() Interface for each function We need a clear separation

5. STEMPNU 5 Abstract Data Type Object-Oriented Programming  Object ? Abstraction (or Encapsulation)  Hiding the internal details Implementation Internal mechanism and process  Only provide Interfaces Abstract Data Type  Hiding the internal structures once it has been implemented  Provide only the interface to the users

6. STEMPNU 6 Algorithms Algorithm  A sequence of instructions with specifications of Input Output : at least one output Definiteness : Clear instructions Finiteness Effectiveness  Abstract description of a program Can be easily converted to a program

7. STEMPNU 7 Performance Analysis What is a good algorithm ?  Correctness  Good documentation and readable code  Proper structure  Effective How to measure the effectiveness of an algorithm ?  Space complexity Amount of memory it needs to run  Time complexity Amount of time (mostly CPU time) it needs to run

8. STEMPNU 8 Space Complexity Notation  Space complexity, f (n) : function of input size n  How should the constants be determined ?  Is it meaningful to count the number of bytes ? int sumAB(int a, int b) { int sum; sum=a+b; return sum; } int sumArray(int n, int *a) { int sum=0; for(int i=0;i<n;i++) sum += a[i]; return sum; } int sumArray(int n, int *a) { if(n<=0) return 0; return a[n-1]+sumArrary(n-1,a); } f (n) = c 1 c 1 : constant f (n) = c 2 a[n] ? f (n) = c 3 n why ?

9. STEMPNU 9 Time Complexity Notation  Time complexity, f (n) : function of input size n int sumAB(int a, int b) { int sum; sum=a+b; return sum; } int sumArray(int n, int *a) { int sum=0; for(int i=0;i<n;i++) sum += a[i]; return sum; } int sumArray(int n, int *a) { if(n<=0) return 0; return a[n-1]+sumArrary(n-1,a); } f (n) = 2 +  f (n) = 3 n +1 +  f (n) = (2+  ) n Is it really meaningful to determine these constants ?

10. STEMPNU 10 Asymptotic Notation Exact time (or step count) to run  Depends on machines and implementation (program)  NOT good measure More general (but inexact) notation : Asymptotic Notation  Big-O O(n) : Upper Bound  Omega-O  (n) : Lower Bound  Theta-O  (n) : More precise than Big-O and Omega-O  Big-O notation is the most popular one.

11. STEMPNU 11 Big-O notation Definition ( f of n is big-O of g of n)  f (n)  O(g (n))  there exist c and n 0 (c, n 0 >0) such that f (n)  c·g(n), for all n (  n 0 ) Example  3n + 2 = O(n), 3n + 2 = O(n 2 )  Time complexity of the following algorithm f (n)= O(n) int sumArray(int n, int *a) { if(n<=0) return 0; return a[n-1]+sumArrary(n-1,a); }

12. STEMPNU 12 Big-O notation : Some Properties Classification  O(1): constant, O(n): Linear, O(n 2 ): Quadratic, O(2 n ): exponential Polynomial function  If f (n) = a m n m + a m-1 n m-1 + … + a 1 n + a 0, then f (n)  O(n m ), where a i > 0 Big-O is determined by the highest order Only the term of the highest order is of our concern Big-O : useful for determining the upper bound of time complexity  When only an upper bound is known, Big-O notation is useful  In most cases, not easy to find the exact f (n)  Big-O notation is the most used.

13. STEMPNU 13 Omega-O notation Definition ( f of n is Omega-O of g of n)  f (n)   (g (n))  there exist c and n 0 (c, n 0 >0) such that f (n)  c·g(n), for all n (  n 0 ) Example  3n + 2 =  (n), 3n + 2 =  (1), 3n 2 + 2 =  (n),  Time complexity of the following algorithm f (n)=  (n) If f (n) = a m n m + a m-1 n m-1 +…+ a 1 n+a 0, then f (n)   (n m )  Omega-O is determined by the highest order Omega-O notation : useful to describe the lower bound int sumArray(int n, int *a) { if(n<=0) return 0; return a[n-1]+sumArrary(n-1,a); }

14. STEMPNU 14 Theta-O notation Definition ( f of n is theta-O of g of n)  f (n)   (g (n))  there exist c 1, c 2 and n 0 (c 1, c 2, and n 0 >0) such that c 1 ·g(n)  f (n)  c 2 ·g(n), for all n (  n 0 ) Example  3n + 2 =  (n), 3n + 2   (1), 3n 2 + 2   (n),  Time complexity of the following algorithm f (n)=  (n) If f (n) = a m n m + a m-1 n m-1 +…+ a 1 n+a 0, then f (n)   (n m )  Theta-O is determined by the highest order Theta-O  Possible Only if f (n)=  (g(n)), and f(n)=  (g(n))  Lower bound and Upper bound is the same  very exact but not easy to find such a g(n) int sumArray(int n, int *a) { if(n<=0) return 0; return a[n-1]+sumArrary(n-1,a); }

15. STEMPNU 15 Complexity Analysis Worst-Case Analysis  Time complexity for the worst case  Example Linear Search : f (n) = n Average Analysis  Average time complexity  f (n) = p 1 f 1 (n) + p 2 f 2 (n) + … p k f k (n), where p i is the probability for the i -th case.  Not easy to find p i In most cases, only worst-case analysis Why not Best-Case Analysis ?

16. STEMPNU 16 Example : Worst-Case Time complexity of Binary Search Binary Search  Big-O : O(n 2 ), O(log n)  Omega O :  (1),  (log n)  Theta O :  (log n) int BinarySearch(int v, int *a, int lower, int upper) // search m among a sorted array a[1ower], a[lower+1], … a[upper] { if(l<=u) { int m=(lower+upper)/2; if(v>a[m]), return BinarySearch(v,a,m+1,upper); else if(v<a[m]) return BinarySearch(v,a,lower,m-1); else /* v==a[m] */ return m; } return -1; // not found }

17. STEMPNU 17 Comparison : O(1), O(log n), O(n), O(n 2 ), O(2 n ) Graph In general  Algorithm of O(1) is almost impossible  Algorithms of O(log n) is excellent  Algorithms of O(n) is very good  Algorithms of O(n 2 ) is not bad  Algorithms of O(n 3 ) is acceptable  Algorithms of O(2 n ) is not useful