Tutorial 7 Sorting & Complexity Analysis (Almost) always asked in exam! e.g. a. Derive algorithm for this problem! b. What is the time complexity of your algorithm?

Asymptotic Algorithm Analysis What to analyze: –Given input of size n, what is the time (or space) required by the algorithm? f(n) = O(g(n)) // Big O –n is the input size! –f(n) is the actual algorithm complexity, can be “hard to compute”… –g(n) is a simple function that upper bounds f(n)! Drop lower order terms and constants in g(n) (hence the name: asymptotic) –Big O is usually for worst case analysis –Formal definition: 0 ≤ f(n) ≤ c*g(n) for all n ≥ n 0 This is like saying: “My algorithm’s growth rate cannot be WORSE than g(n)!” Purpose of Big O analysis: –Simply to see whether the required time (and/or) space of our current algorithm suits the given budget or not. –If no, a better algorithm is needed! We will practice some algorithm analysis skills today!

How long to process n items? Assume that 1 item can be processed in 1 ms Order of GrowthnTime (ms)Comment O(1)10001Excellent, But almost impossible for most cases O(log n) Very good, Example: Binary Search O(n)1000 Normal, Linear time algorithm, 1000 operations == 1000 ms O(n log n) Average, This is usually found in sorting algorithm such as Quick Sort O(n 2 ) Slow, Typical in programs that have two nested loops. O(n 3 ) Slow, btw, All Pairs Shortest Paths algorithm: Floyd Warshall, is O(n 3 ) O(2 n ) Poor, exponential growth... avoid this… O(n!)1000Uncountable Typical NP-Complete problems... Unless the problem size is very small, there is no hope...

What can you do in 1 m? Assume that 1 item can be processed in 1 ms Order of Growth Matters… –O(1) < O(log n) < O(n) < O(n log n) < O(n 2 ) < O(n 3 ) < O(2 n ) < O(n!) We will use this O notation from now towards the end of semester! Order of GrowthTime (ms)Max Possible nComment O(1) ms…Can be virtually infinite… O(log n) ms A (very) very large number O(n) ms60000Baseline, still a very big number… O(n log n) ms~ 5000Moderate, average size for many real life problems O(n 2 ) ms244Small O(n 3 ) ms39Very small O(2 n ) ms16Try to avoid this if you can… O(n!) ms8Extremely small...

Sorting: Motivation When a list is sorted, many operations become easier! Imagine if your dictionaries, phone books, etc are NOT sorted…

Comparison-Based Sorting From simple (slow) to complex (faster) –Bubble SortO(n 2 ) –Insertion SortO(n 2 ) –Selection SortO(n 2 ) –Merge SortO(n log n)  Discussed next week –Quick SortO(n 2 ), but on average O(n log n)  Discussed next 2 week Lower bound for comparison-based sorting algorithms –  (n log n)  Proven, we cannot do better than this!

Special Purpose Sorting Radix SortO(d(n+k)) Counting SortO(n+k) Bucket Sort O(n) Hey, you said that sorting is  (n log n) ? –These sorting algorithms are for special purposes only. –They do not do “comparison”… There are more sorting algorithms in this world… – – But in real life, –We just use generic built-in algorithms like Java “Collections.Sort”. –We are going to learn this in question 2 today.

Sort: Fundamental CS Problem “Give rise” or “used to explain” algorithmic concepts: –Divide and Conquer –Recursion –That efficiency matters Sorting is mostly used as preliminary step for many other algorithms. –When in doubt, sort!... Try this in CS1102 exams! Sorting problem is “well solved”. –The best algorithm to-date is  (n log n) –It matches the lower bound of sorting  (n log n) Also see: –

Student Presentation Gr3 1.Du Xin and (David Seo or Tanvir Islam) 2.Leow Wei Jie or Hema Kumar or Jacob Pang 3.Lim Wei Hong and (Nicholas Koh or Chia Jie Shen) Gr4 1.Wong Suet Teng, Melissa and Liew Hui Sun 2.Wang Kang or Ng Xue Lin Sherilyn or Tan Yan Hao (Gr5) 3.Goh Khoon Hiang and Chua Kien Chuan Overview of the questions: 1.Scenarios and Simple Problems (2 students, a-b and then c-d) 2.F1, Comparator, Collections.sort (1 student, a-c) 3.Complexity Analysis (2 students, a-b-c, and then d) Gr5 1.Joyeeta Biswas (all q1) 2.Ong Kian An 3.Wu Shujun and Zheng Yang Gr6 1.Koh Jye Yiing and (Zhang Chao or Siddhartha) 2.Nguyen Duy Hoang 3.Kwong Gary and Wang Shuling 9

Q1: Scenarios & Simple Probs 10 Sorting Scenarios: 1.Finding word in dictionary 2.Finding duplicates in data 3.Finding median in data 4.Finding pairs A and B, where A+B=V 5.etc… Finding Duplicates –Sort the numbers: O(n log n) –Scan the numbers if two adjacent numbers are the same: O(n) Finding Median –Sort the numbers: O(n log n) –Return number[size/2]; O(1) –Fastest method: O(n), find k-index (see note 5, slide 34 or this link)this link Finding Pair A and B, A+B=V –Sort the numbers: O(n log n) –Scan the numbers: O(n) * O(log n) If the first number is A, then the other number must be B=V-A! Use binary search to find B: O(log n) –If no pair found, report “no pair”.

Q2: F1 Again, but Sorted This is what you need to remember beyond this CS1102. –Something to tell Java which object should be in front of which object! Java Interface: Comparator –Or alternatively, Interface: Comparable Implements compare(o1, o2) –Or alternatively, implements: compareTo(o) By default, generic data type (integer, float, double, string, etc) are comparable! –Java generic sorting algorithm Collections.sort(array)  works for generic data type Collections.sort(array, comparator)  specify comparator for ADTs O(n log n) – efficient implementation of comparison based sorting algorithm Bug free Easy to use –See source codes for details!

Q3: Complexity Analysis (1) Case 1 j=1; while (j<=n) { for (i=n*n; i>=1; i=i-3) x=x+1; j=j*2; } O(n 2 log n) –The innermost loop is n 2 –The while loop is log n j=1 j=j*2 j will reach n in log 2 n steps!! Case 2 for (i=1; i<=100; i++) for (j=1; j<i*n; j++) x=x+1; O(n) –100*(100*n) ~ 10000*n is still O(n) Case 3 for (i=1; i<=100; i++) for (k=1; k<=i; k++) for (j=1; k<=k; j++) x=x+1; O(∞) –The innermost loop will never terminate… Please look at my website for more practice related to algorithm analysis!

Q3: Complexity Analysis (2) Suppose we have: int f(int n) { if (n > 1) { g(n); return f(n/2) + f(n/2); } } What is the time complexity of f(n), if g(n) is: To answer this, we must draw the recursive execution tree… a)g(n) = O(1) O(n), a sum of geometric series of …+2 log2 n = …+n = c*n b)g(n) = O(n) O(n log n), a sum of (n+n+n+…+n) log 2 n times, so, n log n c)g(n) = O(n 2 ) O(n 2 ), a sum of geometric series of n 2 +n 2 /2+n 2 /4+…+n = c*n 2 = n 2 Please look at my website for more practice related to algorithm analysis!

Food For Thought What is the time complexity of recursive program that we built last week? void CS1102_Life(int week_ID) { do_tutorial_and_lab(week_ID); if (week_ID != end of semester) CS1102_Life(week_ID+1); } O(13 teaching weeks) ~ O(1) :p One semester is just “constant time” The “torture” is not that long …