1. Asymptotic Notation (O, Ω, )
Describes the behavior of the time or space complexity for large instance characteristics
Common asymptotic functions
1 (constant), log n, n (linear)
n log n, n2 , n3
2n ( exponential), n!
where n usually refer to the number of instance of data (data的個數)

2. Common asymptotic functions

3. Common asymptotic functions
Other types of complexity: programming complexity, debugging complexity, complexity of comprehension.

4. Common asymptotic functions

5. Common asymptotic functions

6. Big-O notation
The big O notation provides an upper bound for the function f
Definition:
f(n) = O(g(n)) iff positive constants c and n0 exist such that f(n)  c g(n) for all n, nn0
e.g. f(n) = 10n2 + 4n + 2
then, f(n) = O(n2), or O(n3), O(n4), …

7. Ω(Omega) notation
The Ω notation provides a lower bound for the function f
Definition:
f(n) = Ω(g(n)) iff positive constants c and n0 exist such that f(n)  c g(n) for all n, nn0
e.g. f(n) = 10n2 + 4n + 2
then, f(n) = Ω(n2), or Ω(n), Ω(1)

8.  notation
The  notation is used when the function f can be bounded both from above and below by the same function g
Definition:
f(n) =  (g(n)) iff positive constants c1and c2, and an n0 exist such that c1g(n)  f(n)  c2g(n) for all n, nn0
e.g. f(n) = 10n2 + 4n + 2
then, f(n) = Ω(n2), and f(n)=O(n2), therefore, f(n)=  (n2)

9. Sorting
Rearrange a[0], a[1], …, a[n-1] into ascending order. When done, a[0] <= a[1] <= … <= a[n-1]
8, 6, 9, 4, 3 => 3, 4, 6, 8, 9
Strictly speaking, nondecreasing order: 2, 2, 3, 6, 8, 8, 10.

10. Sort Methods Insertion Sort Bubble Sort Selection Sort Rank Sort
Shell Sort
Heap Sort
Merge Sort
Quick Sort
First 5 are simple (conceptually and to program). First 4 done in chapters 2 and 3. Shaker sort and shell sort are in the enrichment section of the text’s Web site. The remaining three are done in the algorithm design chapters of the text.

11. Insert An Element Given a sorted list/sequence, insert a new element
Result 3, 5, 6, 9, 14

12. Insert an Element 3, 6, 9, 14 insert 5
Compare new element (5) and last one (14)
Shift 14 right to get 3, 6, 9, , 14
Shift 9 right to get 3, 6, , 9, 14
Shift 6 right to get 3, , 6, 9, 14
Insert 5 to get 3, 5, 6, 9, 14

13. Insert An Element // insert t into a[0:i-1] int j;
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
a[j + 1] = t;

14. Insertion Sort Start with a sequence of size 1
Repeatedly insert remaining elements

15. Insertion Sort Sort 7, 3, 5, 6, 1 Start with 7 and insert 3 => 3, 7

16. Insertion Sort for (int i = 1; i < a.length; i++)
{// insert a[i] into a[0:i-1]
// code to insert comes here }

17. Insertion Sort for (int i = 1; i < a.length; i++)
{// insert a[i] into a[0:i-1]
int t = a[i];
int j;
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
a[j + 1] = t; }

18. Insertion Sort (C++)

19. Complexity Space/Memory Time Count a particular operation
Count number of steps
Asymptotic complexity
Other types of complexity: programming complexity, debugging complexity, complexity of comprehension.

20. Comparison Count
for (int i = 1; i < a.length; i++) {// insert a[i] into a[0:i-1] int t = a[i]; int j; for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a[j + 1] = t; }

21. Comparison Count
Pick an instance characteristic … n, n = a.length for insertion sort
Determine count as a function of this instance characteristic.

22. Comparison Count
for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; How many comparisons are made?

23. Comparison Count
for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; number of compares depends on a[]s and t as well as on n

24. Comparison Count Worst case count = maximum count
Best case count = minimum count
Average count

25. Worst-Case Comparison Count
for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j];
a = [1, 2, 3, 4] and t = 0 => 4 compares
a = [1,2,3,…,i] and t = 0 => i compares

26. Worst-Case Comparison Count
for (int i = 1; i < n; i++) for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j];
total compares = … + (n-1)
= (n-1)n/2

27. Step Count
A step is an amount of computing that does not depend on the instance characteristic n 10 adds, 100 subtracts, 1000 multiplies can all be counted as a single step n adds cannot be counted as 1 step

28. Step Count s/e for (int i = 1; i < a.length; i++)
{// insert a[i] into a[0:i-1]
int t = a[i];
int j;
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
a[j + 1] = t; }
for (int i = 1; i < a.length; i++)
{// insert a[i] into a[0:i-1]
int t = a[i];
int j;
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
a[j + 1] = t; }

29. Step Count s/e isn’t always 0 or 1 x = MyMath.sum(a, n);
where n is the instance characteristic
has a s/e count of n

30. Step Count i+ 1 i s/e steps for (int i = 1; i < a.length; i++)
{// insert a[i] into a[0:i-1]
int t = a[i];
int j;
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
a[j + 1] = t; }
for (int i = 1; i < a.length; i++)
{// insert a[i] into a[0:i-1]
int t = a[i];
int j;
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
a[j + 1] = t; }
i+ 1 i

31. Step Count
for (int i = 1; i < n ; i++) { 2i + 3} step count for is n step count for body of for loop is 2(1+2+3+…+n-1) + 3(n-1) = (n-1)n + 3(n-1) = (n-1)(n+3)

32. Asymptotic Complexity of Insertion Sort
What does this mean?

33. Complexity of Insertion Sort
Time or number of operations does not exceed c·n2 on any input of size n (n suitably large).
Actually, the worst-case time is Θ(n2) and the best- case is Θ(n)
So, the worst-case time is expected to quadruple each time n is doubled

34. Complexity of Insertion Sort
Is O(n2) too much time?
Is the algorithm practical?

35. Practical Complexities
109 instructions/second
Teraflop computer the 32yr time becomes approx 10 days.

36. Impractical Complexities
109 instructions/second

37. Faster Computer Vs Better Algorithm
Algorithmic improvement more useful than hardware improvement. E.g. 2n to n3