Introduction to Complexity Analysis

Computer Science, Silpakorn University 2 Complexity of Algorithm algorithm คือ ขั้นตอนการคำนวณ ที่ถูกนิยามไว้อย่างชัดเจนโดยจะ ทำงานกับ input หรือ ข้อมูลเข้า แล้วให้ผลลัพธ์หรือ output ออกมา การให้นิยามอัลกอริทึม เราอาจจะ ใช้ pseudo code หรือรหัสเทียม โดยทั่วไปเราจะต้องสร้างอัลกอริทึม เพื่อใช้ในการแก้ปัญหา

Computer Science, Silpakorn University 3 Example : Problem Sorting problem Input: a sequence of numbers Output: a permutation (reordering) of the input sequence such that

Computer Science, Silpakorn University 4 Example : Problem Algorithm input output {31,41,59,26,41,58} {26,31,41,41,59,58} Problem instance

Computer Science, Silpakorn University 5 Review : Insertion sort

Computer Science, Silpakorn University 6 Review : Insertion sort A = { } j=2 j=3 j=4 j=5 j=6 do ne

Computer Science, Silpakorn University 7 Analysis of Algorithm Assumption 1 Processor (RAM model) Sequential execution only Time to execute depends on input size Eg. insertion sort depends on # data to be sorted Performance of algorithm is measured by running time Which is # of steps to runs all operations

Computer Science, Silpakorn University 8 Analysis of Algorithm for (i=0; i < 3; i++) a[i] = b[i]+c[i]; For loop performs 3 time (i=0,1,2) Adding b[i]+c[i] is done 3 times Incrementing i++ is performed 3 times

Computer Science, Silpakorn University 9 Analysis of Algorithm for (j=0;j< n; j++) for (i=0; i < n; i++) a[i,j] = b[i,j]+c[i,j]; For is done n 2 time Adding b[i]+c[i] is done n 2 times For each j increase i totally n times Outter loop performs until j < n

Computer Science, Silpakorn University 10 Analysis of Algorithm :insertion sort

Computer Science, Silpakorn University 11 Analysis of Algorithm: insertion sort Let t j be# times loop while executed at jth step Each instruction in each line takes c i n since each instruction takes c i and is done n times

Computer Science, Silpakorn University 12 Analysis of Algorithm: insertion sort Total time for insertion sort T(n)

Computer Science, Silpakorn University 13 Analysis of Algorithm: insertion sort Best case running time occurs when the input list of data is already sorted. Why?

Computer Science, Silpakorn University 14 Analysis of Algorithm: insertion sort:best case For each j=2,3,4 … n, we found that Before enterring the loop i=j-1

Computer Science, Silpakorn University 15 Analysis of Algorithm: insertion sort:best case So, best case running time of insertion sort forms a linear function of n (where n is the # data in the list) n-1 00

Computer Science, Silpakorn University 16 Analysis of Algorithm: insertion sort :Worst case worst case running time occurs when list of data in มูล A is already sorted in reversed order Why?

Computer Science, Silpakorn University 17 Analysis of Algorithm: insertion sort : worst case For each j=2,3,4 … n we found We have to compare A[1..j-1] for every j

Computer Science, Silpakorn University 18 Analysis of Algorithm: Insertion sort:worst case A = { } j=2 j=3 j=4 do ne compare j-1 = 1 times compare j-1 = 2 times compare j-1 = 3 times

Computer Science, Silpakorn University 19 Analysis of Algorithm: insertion sort: worst case Worst case running time for insertion sort forms quadratic functions of n (where n is # data in the list)

Computer Science, Silpakorn University 20 Analysis of Algorithm Usually we consider worst case Worst case implies the upper bound for the time to execute the algorithm. Average case usually equals to worst case but average case implies average running time. Eg. insertion sort on average takes ….

Computer Science, Silpakorn University 21 Order of growth We measure running time of algorithm or complexity of algorithm. We consider running time in a function of T(n) to see the rate of growth. If rate of growth is high, the algorithm is not good. The algorithm will be slow when n is large.

Computer Science, Silpakorn University 22 Order of growth y=f(x) = ax+b x y y=g(x) = ax 2 +bx+c

Designing algorithms

Computer Science, Silpakorn University 24 Divide and conquer appoarch Divide problem into subproblems and solve by subproblems (DIVIDE) Solutions to the subproblems imply the solution for the whole problem. Take all solutions for the subproblems, combine them to be the whole solution. (CONQUER) Usually in recursive forms

Computer Science, Silpakorn University 25 Divide and conquer Step Divide: Divide into subproblems Conquer: Solve subproblems Combine: combine results of all subproblems into the whole solution.

Computer Science, Silpakorn University 26 Divide and conquer Merge sort Example Divide: divide list of data into 2 equal part (size n/2) Conquer: sort each part recursively Combine: merge sorted parts to be the whole sorted list.

Computer Science, Silpakorn University 27 Divide and conquer

Computer Science, Silpakorn University 28 divide and conquer A={5,2,4,6,1,3,2,6} merge

Computer Science, Silpakorn University 29 Running time forMergeSort in the form Divide: calculate center point of array A take constant time: D(n)=c 2 Conquer: Sorting each part of size n/2 takes aT(n/b) = 2T(n/2) Combine : merge takes linear functions : C(n) = c 3 n+c 4 divide and conquer

Computer Science, Silpakorn University 30 Running time for MergeSort divide and conquer Solve recurrence equation !!

Computer Science, Silpakorn University 31 Complexity of Binary Search recursive version of binary search

Computer Science, Silpakorn University 32 Complexity of Binary Search recursive version of binary search

Computer Science, Silpakorn University 33 Complexity of Binary Search

Computer Science, Silpakorn University 34 Complexity of Merge Sort i=log 2 n