1. Design and Analysis of Algorithms
Dr. Muhammad Safysn
Spring 2019

2. Introduction
The word Algorithm comes from the name of the Muslim author Abu Ja’far Mohammad ibn Musa al-Khowarizmi.
He was born in the eighth century at Khwarizm (Kheva), a town south of river Oxus in present Uzbekistan.
Al-Khwarizmi parents migrated to a place south of Baghdad when he was a child.
It has been established from his contributions that he flourished under Khalifah Al-Mamun at Baghdad during 813 to 833 C.E. Al-Khwarizmi died around 840 C.E.

3. Definition
An algorithm is a mathematical entity, which is independent of a specific programming language, machine, or compiler.
Design is all about the mathematical theory behind the design of good programs.
Multi-facets to good program design
Aware of programming and machine issues as well
Essential for a good understanding

4. Analyzing Algorithms What is criteria for Good Algorithm?
What is criteria of measuring Algorithm?
Measure algorithms in terms of the amount of computational resources that the algorithm requires.
it is also known as complexity analysis
Primary Resouces are (i) Runing Time and (ii) Memory
Other Includes: number disk accesses in a database program , communication bandwidth in a networking application.
Also, one
of the luxuries we will have in this course is to be able to assume that we are given a clean, fully-specified
mathematical description of the computational problem.

5. Computing resources:
e Memory space: space required to store the data processed by algorithm
e CPU time: also known as running time. It is time needed to execute the operations of the algorithm.
An efficient algorithm uses a minimum amount of computing resources.

6. Efficiency Analysis e There are two types of efficiency analysis
) Space analysis: how much space an algorithm required to store the data in memory?
) Running time analysis: it refers to how fast an algorithm runs
e The key strategy in efficiency analysis is the amount of computing resources depends on the input size (problem size)
) Input size: the number of elements belonging to the input data.
e Hence the main question to be answered by efficiency analysis is how depends the time and/or space needed by algorithm on the input size?

7. Space and time trade off
e Often we have to make a compromise between space efficiency and time efficiency
e Example: adjacency matrix vs adjacency list for graph representation.
) A sparse graph can be represented by an adjacency matrix which would be time efficient for traversing an edge but it will be at the cost of space
) A sparse graph can be represented by an adjacency list which would be space efficient and but it would take longer time for traversing the edges.

8. e Programming language
How can be time efficiency measured? Our efficiency measure for running time must be independent of
e Programmer
e Programming language
e Machine

9. RAM: Computational Model
However to estimate running time we must use some computational model.
Computational model is an abstract machine having some properties.
For this purpose we will use RAM computational model(Random Access Machine) having the following properties
All processing steps are sequentially executed (there is no parallelism in the execution of the algorithm)

10. RAM: Computational Model
All processing steps are sequentially executed (there is no parallelism in the execution of the algorithm)
The time of executing the basic operations does not depend on the values of the operands (there is no time difference between computing 1+2 and computing )
The time to access data does not depend on their address (there are no differences between processing the first element of an array and processing the last element) All basic operations, assignment,
arithmetic, logical, relational take unit time.

11. Loop-holes two numbers may be of any length. Serlization . Also, one
of the luxuries we will have in this course is to be able to assume that we are given a clean, fully-specified
mathematical description of the computational problem.

12. Running Time Analysis Concern about measuring the execution time.
Concerned about the space (memory) required by the algorithm.
Ignore during Running Time
Speed of computer
Programming Language
optimization by the compiler
.Count
Different inputs of the same size may result in different running time
Also, one
of the luxuries we will have in this course is to be able to assume that we are given a clean, fully-specified
mathematical description of the computational problem.

13. Running Time Analysis
Two criteria for measuring running time are worst-case time and average-case time
Worst-case time
maximum running time over all (legal) inputs of size n. Let I denote an input instance, let |I| denote its length, and let T(I) denote the running time of the algorithm on input I. Then
Average-case time is the average running time over all inputs of size n. Let p(I) denote the probability of seeing this input. The average-case time is the weighted sum of running times with weights being the probabilities:
We will almost always work with worst-case time
Average-case time is more difficult to compute;
Also, one
of the luxuries we will have in this course is to be able to assume that we are given a clean, fully-specified
mathematical description of the computational problem.

14. Running Time Analysis Also, one
of the luxuries we will have in this course is to be able to assume that we are given a clean, fully-specified
mathematical description of the computational problem.

15. Running Time Analysis Also, one
of the luxuries we will have in this course is to be able to assume that we are given a clean, fully-specified
mathematical description of the computational problem.

16. Example1: running time
Now we see how running time expresses the dependence of the number of executed operations on the input size
Example: swapping iterations cost
aux ← x x ← y 1 c1
y ← x
T (n) = 3 c1
where c1 is some constant and 3c1 is also some constant. We conclude that running time of this algorithm is some constant. Hence we can understand that the running time of above algorithm is independent of the input size.

17. Example2: running time line no. cost iterations 1 c1 2 3 c2 n + 1 4 c3
Example 2: Compute sum of the series s = n precondition: n ≥ 1 postcondition: s = n
input: n
1: s ← 0
2: i ← 1
3: while i ≤ n do
4: s ← s + i
5: i ← i + 1
6: end while
line no.
cost
iterations 1 c1 2 3 c2
n + 1 4 c3
n.1 5

18. Example2: running time line no. cost iterations 1 c1 2 3 c2 n + 1 4 c3
Example 2: Compute sum of the series s = n precondition: n ≥ 1 postcondition: s = n
input: n
1: s ← 0
2: i ← 1
3: while i ≤ n do
4: s ← s + i
5: i ← i + 1
6: end while
line no.
cost
iterations 1 c1 2 3 c2
n + 1 4 c3
n.1 5
T (n) =
2c1 + c2(n + 1) + 2c3n

19. Example2: running time line no. cost iterations 1 c1 2 3 c2 n + 1 4 c3
Example 2: Compute sum of the series s = n precondition: n ≥ 1 postcondition: s = n
input: n
1: s ← 0
2: i ← 1
3: while i ≤ n do
4: s ← s + i
5: i ← i + 1
6: end while
line no.
cost
iterations 1 c1 2 3 c2
n + 1 4 c3
n.1 5
T (n) =
2c1 + c2(n + 1) + 2c3n
2c1 + c2.n + c2 + 2c3.n

20. Example2: running time line no. cost iterations 1 c1 2 3 c2 n + 1 4 c3
Example 2: Compute sum of the series s = n precondition: n ≥ 1 postcondition: s = n
input: n
1: s ← 0
2: i ← 1
3: while i ≤ n do
4: s ← s + i
5: i ← i + 1
6: end while
line no.
cost
iterations 1 c1 2 3 c2
n + 1 4 c3
n.1 5
T (n) =
2c1 + c2(n + 1) + 2c3n
2c1 + c2.n + c2 + 2c3.n
a.n + b

21. Example2: running time line no. cost iterations 1 c1 2 3 c2 n + 1 4 c3
Example 2: Compute sum of the series s = n precondition: n ≥ 1 postcondition: s = n
input: n
1: s ← 0
2: i ← 1
3: while i ≤ n do
4: s ← s + i
5: i ← i + 1
6: end while
Running time is linear function of input size
line no.
cost
iterations 1 c1 2 3 c2
n + 1 4 c3
n.1 5
T (n) =
2c1 + c2(n + 1) + 2c3n
2c1 + c2.n + c2 + 2c3.n
a.n + b

22. Example3: running time
Example 3: Find the minimum in non-empty array x [ n]
P: n ≥ 1 Q: m = min{x [j ]|j = 1, 2, n}n input: x [1..n]
line no. cost
iterations
1: m ← x [1]
2: for i ← 2, n do 1 2 3 4 1 1 2n
3: if x [i ] < m
then
n − 1 4:
m ← x [i ]
h(n)
5: end if
6: end for
7: return m
../pucitl ogo.jp

23. Example3: running time line no. cost iterations 1 2 2n 3 4 n − 1 h(n)
Example 3: Find the minimum in non-empty array x [ n]
P: n ≥ 1 Q: m = min{x [j ]|j = 1, 2, n}n input: x [1..n]
1: m ← x [1]
2: for i ← 2, n do
3: if x [i ] < m
then
line no.
cost
iterations 1 2 2n 3 4
n − 1
h(n) 4:
m ← x [i ]
5: end if
6: end for
T (n) = 1 + 2n + n − 1 + h(n)
7: return m

24. Example3: running time line no. cost iterations 1 2 2n 3 4 n − 1 h(n)
Example 3: Find the minimum in non-empty array x [ n]
P: n ≥ 1 Q: m = min{x [j ]|j = 1, 2, n}n input: x [1..n]
1: m ← x [1]
2: for i ← 2, n do
3: if x [i ] < m
then
line no.
cost
iterations 1 2 2n 3 4
n − 1
h(n) 4:
m ← x [i ]
5: end if
6: end for
T (n) = 1 + 2n + n − 1 + h(n)
T (n) = 3n + h(n)
7: return m

25. Example3: running time line no. cost iterations 1 2 2n 3 4 n − 1 h(n)
Example 3: Find the minimum in non-empty array x [ n]
P: n ≥ 1 Q: m = min{x [j ]|j = 1, 2, n}n input: x [1..n]
1: m ← x [1]
2: for i ← 2, n do
3: if x [i ] < m
then
line no.
cost
iterations 1 2 2n 3 4
n − 1
h(n) 4:
m ← x [i ]
5: end if
6: end for
T (n) = 1 + 2n + n − 1 + h(n)
T (n) = 3n + h(n)
7: return m
The running time depends not only on n but also on the properties
of input data
../pucitl ogo.jp

26. Best-case analysis and worst-case analysis
Whenever analysis of an algorithm not only depends on the input size but also on some property of input data then we have to perform analysis in more details
e Worst-case analysis: gives the longest running time for any input of size n.
e Best-case analysis: gives us the minimum running time for any input of size n.
) worst-case running time of an algorithm gives us an upper bound on the running time for any input.
) Knowing it provides a guarantee that the algorithm will never take any longer.
e best-case running time of an algorithm gives us lower bound on the running time for any input.
e Knowing it provides a guarantee that the algorithm will never tak..e/l more less time.

27. Example 4: sequential search
Preconditions: x [1..n], n >= 1, v a value Postconditions: found = TRUE when v ∈ x [1..n] Input: x [1..n], v
Algorithm 1 search 1: found ← true
2: i ← 1
3: while (found = false) and (i ≤ n) do
4: if x [i ] = v then 5:
found ←true
6: else 7:
i ← i + 1
8: end if
9: end while

28. Example 4: sequential search
Preconditions: x [1..n], n >= 1, v a value Postconditions: found = TRUE when v ∈ x [1..n] Input: x [1..n], v
Algorithm 2 search
line no
cost
1: found ← true
2: i ← 1 1 2 3 4 5 7 1
f (n) + 1
f (n)
g (n)
h(n)
3: while (found = false) and (i ≤ n) do
4: if x [i ] = v then 5:
found ←true
6: else 7:
i ← i + 1
T(n)=
3 + 2f (n) + g (n) + h(n)
8: end if
9: end while

29. Example 4: sequential search
Preconditions: x [1..n], n >= 1, v a value Postconditions: found = TRUE when v ∈ x [1..n] Input: x [1..n], v
Algorithm 3 search
line no
cost
1: found ← true
2: i ← 1 1 2 3 4 5 7 1
f (n) + 1
f (n)
g (n)
h(n)
3: while (found = false) and (i ≤ n) do
4: if x [i ] = v then 5:
found ←true
6: else 7:
i ← i + 1
T(n)=
3 + 2f (n) + g (n) + h(n)
8: end if
9: end while

30. Today’s Agenda Role of dominant term
Order of growth Asymptotic Analysis

31. Dominant term or Leading term
We used some simplifying abstractions to ease our analysis
The main aim of efficiency analysis is to find out how increases the running time when the problem size increases
e Running time is mostly affected by dominant term
e In most of the cases we do not require the detailed analysis of running time
e We need to just identify the dominant term, which helps to find:
) The order of growth of the running time
) The efficiency class to which an algorithm belongs

32. Identify the dominant term
In the expression of the running time one of the terms will become significantly larger than the other ones when n becomes large:this is the so-called dominant term
Running time Dominanat term
T1(n) = an + b an
T2(n) = alogn + b T3(n) = an2
alog n an2 an
T4(n) = an + bn + c(a > n)

33. What is the order of growth?
The order of growth expresses how increases the dominant term of the running time with the input size
Running time
Dominant term
Order of growth
T1(n) = an + b an
Linear
T2(n) = alogn + b
alogn
Logarithmic
T3(n) = an2
an2
Quadratic an
T4(n) = an + bn + c(a > n)
Exponential

34. Order of growth vs Input size?
Between two algorithms it is considered that the one having a smaller order of growth is more efficient
this is true only for large enough input sizes
Example:
T1(n) = 10n + 10 (linear order of growth)
T2(n) = n2 (quadratic order of growth)

35. Order of growth vs Input size?
Between two algorithms it is considered that the one having a smaller order of growth is more efficient
this is true only for large enough input sizes
Example:
T1(n) = 10n + 10 (linear order of growth)
T2(n) = n2 (quadratic order of growth) if n ≤ 10 then T1(n) > T2(n)
e In this case the order of growth is relevant only for n > 10
e For larger input,n the low terms in a function are relatively insignificant

36. Order of growth vs Input size?
Between two algorithms it is considered that the one having a smaller order of growth is more efficient
this is true only for large enough input sizes
Example:
T1(n) = 10n + 10 (linear order of growth)
T2(n) = n2 (quadratic order of growth) if n ≤ 10 then T1(n) > T2(n)
e In this case the order of growth is relevant only for n > 10
e For larger input,n the low terms in a function are relatively insignificant
e n n2 + 10n + 50 ≈ n4

37. n2 2n A comparison of the order of growth log n n nlogn 3.3 10 33 100
1024
6.6
664
10000
1030
1000
9965
10301 13
132877
103010

38. n2 2n A comparison of the order of growth log n n nlogn 3.3 10 33 100
1024
6.6
664
10000
1030
1000
9965
10301 13
132877
103010

39. Comparing order of growth
The order of growth of two running times T1(n) and T2(n) can be compared by computing the limit of T1(n)/T2(n) when n goes to infinity.
e If the limit is 0 then T1(n) has a smaller order of growth than
T2(n)
e If the limit is a finite constant c(c > 0) then T1(n) and T2(n) have the same order of growth
e If the limit is infinity then T1(n) has a larger order of growth than T2(n)
.l ogo.jp

40. Asymptotic Analysis
e While analysis of running time often extra precision is not required
e For large enough input, the multiplicative constants and lower order terms can be ignored
e When we look at input size large enough to make only the order of growth of the running time relevant, its called asymptotic efficiency of algorithms
e That is, we are concerned with how the running time of an algorithm increases with the size of the input in the limit, as the size of the input increases without bound.
..l ogo.jp

41. Asymptotic Analysis
e Asymptotic notation actually applies to functions.
) Recall that we characterized running time of a matrix initialization as an2 + bn + c.
) By writing writing running time of above function as O(n2), we have abstracted some details of the function.
e While applying asymptotic notations to running time, we need to understand which running times of algorithms.
) Sometimes we are interested in worst-case running time
) Often we wish to characterize running time no matter what the input
ogo.jp

42. Asymptotic Notations e Big O: asymptotic less than
) f (n) = O(g (n)) ⇒ f (n)≤g (n)
e Big Ω: asymptotic greater than
) f (n) = Ω(g (n)) ⇒ f (n)≥g (n)
e Big Θ: asymptotic equality
) f (n) = Θ(g (n)) ⇒ f (n)=g (n)
../pucitl ogo.jp

43. Big-O Notation
e We say f (n) = 30n + 8 is order n, or O(n) it is, at most, roughly proportional to n.
e We say g (n) = n2 + 1 is order n2, or O(n2) it is, at most, roughly proportional to n2.
../pucitl ogo.jp

44. Big-O Notation 104 1 f (n) g (n) 0.8 0.6 0.4 0.2 T (n) 20 40 60 80 100
../pucitl ogo.jp 20 40 60 80
100 n

45. Big-O: Example e 2n2 = O(n3): 2n2 ≤ cn3 ⇒ 2 ≤ cn ⇒ c = 1 and no = 2
../pucitl ogo.jp

46. Big-O: Example e 2n2 = O(n3): 2n2 ≤ cn3 ⇒ 2 ≤ cn ⇒ c = 1 and no = 2
e n2 = O(n2): n2 ≤ cn2 ⇒ 1 ≤ c ⇒ c = 1 and no = 1

47. Big-O: Example e 2n2 = O(n3): 2n2 ≤ cn3 ⇒ 2 ≤ cn ⇒ c = 1 and no = 2
e n2 = O(n2): n2 ≤ cn2 ⇒ 1 ≤ c ⇒ c = 1 and no = 1
e 1000n n = O(n2): 1000n n ≤ cn2 ⇒
1000n n ≤ 1001n2 ⇒ c = 1001 and no = 1000
. ogo.jp

48. Let c = 31, no = 8. Assume n > no then 30n + 8 ≤ 30n + n
Big-O: Formal Example
Show that 30n + 8 is O(n) For this we have to prove that
Proof:
∃c, no : 30n + 8 ≤ cn, ∀n > no
Let c = 31, no = 8. Assume n > no then 30n + 8 ≤ 30n + n
e There is no unique set of values for no and c in proving the asymptotic bounds
e Must find some constants c and no that satisfy the asymptotic notation relation
../pucitl ogo.jp

49. Big-Ω Notation
Ω(g (n)) is the set of functions with larger or same order of growth as g (n)
../pucitl ogo.jp

50. Example
5n2 = Ω(n)
∃c, no such that 0 ≤ cn ≤ 5n2 ⇒ cn ≤ 5n2 ⇒ c = 1 and
no = 1
100n + 5 ƒ= Ω(n2)
∃c, no such that 0 ≤ cn2 ≤ 100n n + 5 ≤ 100n + 5n(∀n ≥ 1) = 105n cn2 ≤ n ⇒ n(cn − 105) ≤ 0
since n is positive ⇒ cn − 105 ≤ 0 ⇒ n ≤ 105/c ⇒
contradiction: n cannot be smaller than constant.
../pucitl ogo.jp

51. Example
n2/2 − n/2 = Θ(n2)

52. Example
n2/2 − n/2 = Θ(n2)
First we prove that n2/2 − n/2 = O(n2), (∀n ≥ no )
n − n ≤ n 2
2 2
n − n ≤ n 2 2 2
0 ≤ n 1 2
0 ≤ n
Hence proved that n2/2 − n/2 = O(n2), (∀n ≥ no ) where
no = 0and c2 = 1 2

53. Example
n2/2 − n/2 = Θ(n2)
First we prove that n2/2 − n/2 = O(n2), (∀n ≥ no )
n − n ≤ n 2
2 2
n − n ≤ n 2 2 2
0 ≤ n 1 2
0 ≤ n
Hence proved that n2/2 − n/2 = O(n2), (∀n ≥ no ) where
no = 0and c2 = 1 2
Secondly we prove that n2/2 − n/2 = Ω(n2),(∀n ≥ no )
n − n ≥ n 2
2 4
n − n ≥ n 2
4 2
n ≥ n
4 2
Hence∀n ≥ 2, (∀n ≥ no ) where no = 2 and c1 = 1/4 6n3 ƒ= Θ(n2)

54. Running Time Analysis Also, one
of the luxuries we will have in this course is to be able to assume that we are given a clean, fully-specified
mathematical description of the computational problem.

55. Running Time Analysis Also, one
of the luxuries we will have in this course is to be able to assume that we are given a clean, fully-specified
mathematical description of the computational problem.

56. Review: Running Time An Example: Insertion Sort
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
Also, one
of the luxuries we will have in this course is to be able to assume that we are given a clean, fully-specified
mathematical description of the computational problem.

57. Review: Running Time i =  j =  key =  A[j] =  A[j+1] =  30 10 4020
i =  j =  key =  A[j] =  A[j+1] =  1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
Also, one
of the luxuries we will have in this course is to be able to assume that we are given a clean, fully-specified
mathematical description of the computational problem.
David Luebke /8/2019

58. An Example: Insertion Sort30 10 40 20
i = 2 j = 1 key = 10 A[j] = 30 A[j+1] = 10 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

59. An Example: Insertion Sort30 30 40 20
i = 2 j = 1 key = 10 A[j] = 30 A[j+1] = 30 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

60. An Example: Insertion Sort30 30 40 20
i = 2 j = 1 key = 10 A[j] = 30 A[j+1] = 30 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

61. An Example: Insertion Sort30 30 40 20
i = 2 j = 0 key = 10 A[j] =  A[j+1] = 30 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

62. An Example: Insertion Sort30 30 40 20
i = 2 j = 0 key = 10 A[j] =  A[j+1] = 30 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

63. An Example: Insertion Sort10 30 40 20
i = 2 j = 0 key = 10 A[j] =  A[j+1] = 10 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

64. An Example: Insertion Sort10 30 40 20
i = 3 j = 0 key = 10 A[j] =  A[j+1] = 10 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

65. An Example: Insertion Sort10 30 40 20
i = 3 j = 0 key = 40 A[j] =  A[j+1] = 10 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

66. An Example: Insertion Sort10 30 40 20
i = 3 j = 0 key = 40 A[j] =  A[j+1] = 10 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

67. An Example: Insertion Sort10 30 40 20
i = 3 j = 2 key = 40 A[j] = 30 A[j+1] = 40 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

68. An Example: Insertion Sort10 30 40 20
i = 3 j = 2 key = 40 A[j] = 30 A[j+1] = 40 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

69. An Example: Insertion Sort10 30 40 20
i = 3 j = 2 key = 40 A[j] = 30 A[j+1] = 40 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

70. An Example: Insertion Sort10 30 40 20
i = 4 j = 2 key = 40 A[j] = 30 A[j+1] = 40 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

71. An Example: Insertion Sort10 30 40 20
i = 4 j = 2 key = 20 A[j] = 30 A[j+1] = 40 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

72. An Example: Insertion Sort10 30 40 20
i = 4 j = 2 key = 20 A[j] = 30 A[j+1] = 40 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

73. An Example: Insertion Sort10 30 40 20
i = 4 j = 3 key = 20 A[j] = 40 A[j+1] = 20 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

74. An Example: Insertion Sort10 30 40 20
i = 4 j = 3 key = 20 A[j] = 40 A[j+1] = 20 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

75. An Example: Insertion Sort10 30 40 40
i = 4 j = 3 key = 20 A[j] = 40 A[j+1] = 40 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

76. An Example: Insertion Sort10 30 40 40
i = 4 j = 3 key = 20 A[j] = 40 A[j+1] = 40 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

77. An Example: Insertion Sort10 30 40 40
i = 4 j = 3 key = 20 A[j] = 40 A[j+1] = 40 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

78. An Example: Insertion Sort10 30 40 40
i = 4 j = 2 key = 20 A[j] = 30 A[j+1] = 40 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

79. An Example: Insertion Sort10 30 40 40
i = 4 j = 2 key = 20 A[j] = 30 A[j+1] = 40 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

80. An Example: Insertion Sort10 30 30 40
i = 4 j = 2 key = 20 A[j] = 30 A[j+1] = 30 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

81. An Example: Insertion Sort10 30 30 40
i = 4 j = 2 key = 20 A[j] = 30 A[j+1] = 30 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

82. An Example: Insertion Sort10 30 30 40
i = 4 j = 1 key = 20 A[j] = 10 A[j+1] = 30 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

83. An Example: Insertion Sort10 30 30 40
i = 4 j = 1 key = 20 A[j] = 10 A[j+1] = 30 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

84. An Example: Insertion Sort10 20 30 40
i = 4 j = 1 key = 20 A[j] = 10 A[j+1] = 20 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
David Luebke /8/2019

85. An Example: Insertion Sort10 20 30 40
i = 4 j = 1 key = 20 A[j] = 10 A[j+1] = 20 1 2 3 4
InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }
Done!
David Luebke /8/2019

86. Asymptotic Notation What is an algorithm?
a step-by-step procedure to solve a problem
every program is the instantiation of some algorithm
CSCE 411, Spring 2013: Set 1

87. Model of Computation
An other Goal: Algorithm is independent as possible of the variations in machine, operating system, compiler, or programming language.
algorithms to be understood by the people and programs are understood by the machine.
Flexiblitiy -> low level detail may be omitted
For algoritm to be understandable, settle it on mathematical Model of Computation.
Mathematical model is abstraction of a standard generic single-processor machine. call this model a Random Access Machine( RAM)
RAM:
Infinite Memory
Instruction are executed in serlization
Instruction perform the basic Operation
Each basic operation takes the same constant time to execute.
Also, one
of the luxuries we will have in this course is to be able to assume that we are given a clean, fully-specified
mathematical description of the computational problem.