1. Discrete Mathematics CMP-101 Lecture 12 Sorting, Bubble Sort, Insertion Sort, Greedy Algorithms
Abdul Hameed
Lecture 11

2. Sorting: Definition
Sorting: an operation that segregates items into groups according to specified criterion.
A = { }
A = { }

3. Sorting Sorting = ordering.
Sorted = ordered based on a particular way. Generally, collections of data are presented in a sorted manner. Examples of Sorting:
Words in a dictionary are sorted (and case distinctions are ignored).
Files in a directory are often listed in sorted order.
The index of a book is sorted (and case distinctions are ignored).

4. Sorting: Cont’d
Many banks provide statements that list checks in increasing order (by check number).
In a newspaper, the calendar of events in a schedule is generally sorted by date.
Musical compact disks in a record store are generally sorted by recording artist.
Why?
Imagine finding the phone number of your friend in your mobile phone, but the phone book is not sorted.

5. Review of Complexity
Most of the primary sorting algorithms run on different space and time complexity.
Time Complexity is defined to be the time the computer takes to run a program (or algorithm in our case).
Space complexity is defined to be the amount of memory the computer needs to run a program.

6. Complexity (cont.)
Complexity in general, measures the algorithms efficiency in internal factors such as the time needed to run an algorithm.
External Factors (not related to complexity):
Size of the input of the algorithm
Speed of the Computer
Quality of the Compiler

7. Types of Sorting Algorithms
There are many, many different types of sorting algorithms, but the primary ones are:
Bubble Sort
Selection Sort
Insertion Sort
Merge Sort
Quick Sort
Shell Sort
Radix Sort
Swap Sort
Heap Sort

8. Bubble Sort: Idea Idea: bubble in water. How?
Bubble in water moves upward. Why?
How?
When a bubble moves upward, the water from above will move downward to fill in the space left by the bubble.

9. Bubble Sort Example
9, 6, 2, 12, 11, 9, 3, 7
6, 9, 2, 12, 11, 9, 3, 7
6, 2, 9, 12, 11, 9, 3, 7
6, 2, 9, 12, 11, 9, 3, 7
6, 2, 9, 11, 12, 9, 3, 7
6, 2, 9, 11, 9, 12, 3, 7
6, 2, 9, 11, 9, 3, 12, 7
6, 2, 9, 11, 9, 3, 7, 12

10. Bubble Sort Example
First Pass
6, 2, 9, 11, 9, 3, 7, 12
Second Pass
2, 6, 9, 9, 3, 7, 11, 12
2, 6, 9, 9, 3, 11, 7, 12
2, 6, 9, 11, 9, 3, 7, 12
2, 6, 9, 9, 11, 3, 7, 12
6, 2, 9, 11, 9, 3, 7, 12
Notice that this time we do not have to compare the last two numbers as we know the 12 is in position. This pass therefore only requires 6 comparisons.

11. Bubble Sort Example
First Pass
6, 2, 9, 11, 9, 3, 7, 12
Second Pass
2, 6, 9, 9, 3, 7, 11, 12
Third Pass
2, 6, 9, 9, 3, 7, 11, 12
2, 6, 9, 3, 7, 9, 11, 12
2, 6, 9, 3, 9, 7, 11, 12
This time the 11 and 12 are in position. This pass therefore only requires 5 comparisons.

12. Bubble Sort Example
First Pass
6, 2, 9, 11, 9, 3, 7, 12
Second Pass
2, 6, 9, 9, 3, 7, 11, 12
Third Pass
2, 6, 9, 3, 7, 9, 11, 12
Fourth Pass
2, 6, 3, 7, 9, 9, 11, 12
2, 6, 9, 3, 7, 9, 11, 12
2, 6, 3, 9, 7, 9, 11, 12
Each pass requires fewer comparisons. This time only 4 are needed.

13. Bubble Sort Example
First Pass
6, 2, 9, 11, 9, 3, 7, 12
Second Pass
2, 6, 9, 9, 3, 7, 11, 12
Third Pass
2, 6, 9, 3, 7, 9, 11, 12
Fourth Pass
2, 6, 3, 7, 9, 9, 11, 12
Fifth Pass
2, 6, 3, 7, 9, 9, 11, 12
2, 3, 6, 7, 9, 9, 11, 12
The list is now sorted but the algorithm does not know this until it completes a pass with no exchanges.

14. Bubble Sort Example
First Pass
6, 2, 9, 11, 9, 3, 7, 12
Second Pass
2, 6, 9, 9, 3, 7, 11, 12
Third Pass
2, 6, 9, 3, 7, 9, 11, 12
Fourth Pass
2, 6, 3, 7, 9, 9, 11, 12
Fifth Pass
This pass no exchanges are made so the algorithm knows the list is sorted. It can therefore save time by not doing the final pass. With other lists this check could save much more work.
2, 3, 6, 7, 9, 9, 11, 12
Sixth Pass
2, 3, 6, 7, 9, 9, 11, 12

15. Bubble Sort Example Quiz Time
Which number is definitely in its correct position at the end of the first pass?
Answer: The last number must be the largest.
How does the number of comparisons required change as the pass number increases?
Answer: Each pass requires one fewer comparison than the last.
How does the algorithm know when the list is sorted?
Answer: When a pass with no exchanges occurs.
What is the maximum number of comparisons required for a list of 10 numbers?
Answer: 9 comparisons, then 8, 7, 6, 5, 4, 3, 2, 1 so total 45

16. Bubble Sort: Example 40 2 1 43 3 65 -1 58 42 4 65 2 1 40 3 43 -1 58 42-1 58 42 4 1 2 65 2 1 40 3 43 -1 58 42 4 1 2 3 40 -1 43 42 4 3 65 58 4 1 2 3 40 65 -1 43 58 42 4
Notice that at least one element will be in the correct position each iteration.

17. Bubble Sort: Example 1 2 3 -1 40 65 43 58 42 4 1 -1 3 2 65 43 58 42 403 -1 40 65 43 58 42 4 1 -1 3 2 65 43 58 42 40 4 6 -1 1 2 65 3 43 58 42 40 4 7 8 -1 1 2 65 3 43 58 42 40 4

18. Selection Sort: Idea We have two group of items:
sorted group, and
unsorted group
Initially, all items are in the unsorted group. The sorted group is empty.
We assume that items in the unsorted group unsorted.
We have to keep items in the sorted group sorted.

19. Selection Sort: Cont’d
Select the “best” (eg. largest) item from the unsorted group, then put the “best” item at the end of the sorted group.
Repeat the process until the unsorted group becomes empty.

20. Selection Sort 5 1 3 4 6 2
Comparison
Data Movement
Sorted

21. Selection Sort 5 1 3 4 6 2
Comparison
Data Movement
Sorted

22. Selection Sort 5 1 3 4 6 2
Comparison
Data Movement
Sorted

23. Selection Sort 5 1 3 4 6 2
Comparison
Data Movement
Sorted

24. Selection Sort 5 1 3 4 6 2
Comparison
Data Movement
Sorted

25. Selection Sort 5 1 3 4 6 2
Comparison
Data Movement
Sorted

26. Selection Sort 5 1 3 4 6 2
Comparison
Data Movement
Sorted

27. Selection Sort 5 1 3 4 6 2 
Largest
Comparison
Data Movement
Sorted

28. Selection Sort 5 1 3 4 2 6
Comparison
Data Movement
Sorted

29. Selection Sort 5 1 3 4 2 6
Comparison
Data Movement
Sorted

30. Selection Sort 5 1 3 4 2 6
Comparison
Data Movement
Sorted

31. Selection Sort 5 1 3 4 2 6
Comparison
Data Movement
Sorted

32. Selection Sort 5 1 3 4 2 6
Comparison
Data Movement
Sorted

33. Selection Sort 5 1 3 4 2 6
Comparison
Data Movement
Sorted

34. Selection Sort 5 1 3 4 2 6
Comparison
Data Movement
Sorted

35. Selection Sort 5 1 3 4 2 6 
Largest
Comparison
Data Movement
Sorted

36. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

37. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

38. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

39. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

40. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

41. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

42. Selection Sort 2 1 3 4 5 6 
Largest
Comparison
Data Movement
Sorted

43. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

44. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

45. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

46. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

47. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

48. Selection Sort 2 1 3 4 5 6 
Largest
Comparison
Data Movement
Sorted

49. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

50. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

51. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

52. Selection Sort 2 1 3 4 5 6
Comparison
Data Movement
Sorted

53. Selection Sort 2 1 3 4 5 6 
Largest
Comparison
Data Movement
Sorted

54. Selection Sort 1 2 3 4 5 6
Comparison
Data Movement
Sorted

55. Selection Sort 1 2 3 4 5 6
DONE!
Comparison
Data Movement
Sorted

56. Selection Sort: Example40 2 1 43 3 65 -1 58 42 4 40 2 1 43 3 4 -1 58 3 42 65 40 2 1 43 3 4 -1 42 3 58 65 40 2 1 3 3 4 -1 42 43 58 65

57. Selection Sort: Example40 2 1 3 3 4 -1 42 43 58 65 -1 2 1 3 3 4 40 42 43 58 65 -1 2 1 3 3 4 40 42 43 58 65 -1 2 1 3 3 4 40 42 43 58 65

58. Selection Sort: Example-1 2 1 3 3 4 40 42 43 58 65 -1 1 2 3 3 4 40 42 43 58 65 -1 1 2 3 3 4 40 42 43 58 65 -1 1 2 3 3 4 40 42 43 58 65 -1 1 2 3 3 4 40 42 43 58 65

59. Insertion Sort: Idea 8 | 5 9 2 6 3 5 8 | 9 2 6 3 5 8 9 | 2 6 3
Idea: sorting cards.
8 |
5 8 |
5 8 9 | 2 6 3
| 6 3
| 3 |

60. Insertion Sort: Idea We have two group of items:
sorted group, and
unsorted group
Initially, all items in the unsorted group and the sorted group is empty.
We assume that items in the unsorted group unsorted.
We have to keep items in the sorted group sorted.
Pick any item from, then insert the item at the right position in the sorted group to maintain sorted property.
Repeat the process until the unsorted group becomes empty.

61. Insertion Sort: Example40 40 2 1 43 3 65 -1 58 42 4 2 40 1 43 3 65 -1 58 3 42 4 1 2 40 43 3 65 -1 58 3 42 4

62. Insertion Sort: Example1 2 40 43 3 65 -1 58 3 42 4 1 2 3 40 43 65 -1 58 3 42 4 1 2 3 40 43 65 -1 58 3 42 4

63. Insertion Sort: Example1 2 3 40 43 65 -1 58 3 42 4 1 2 3 40 43 65 -1 58 3 42 4 -1 1 1 2 2 3 3 40 43 40 43 65 65 58 3 42 4

64. Insertion Sort: Example-1 1 1 2 2 3 3 40 40 43 43 65 58 65 3 42 4 -1 1 2 1 2 3 3 40 3 43 40 43 65 43 58 58 65 65 42 4 -1 1 1 2 2 3 3 40 3 43 40 65 43 42 65 43 58 65 4 1 2 3 40 43 65 42 -1 58 4

65. A Lower Bound
Bubble Sort, Selection Sort, Insertion Sort all have worst case of O(N2).
Turns out, for any algorithm that exchanges adjacent items, this is the best worst case: Ω(N2)
In other words, this is a lower bound!

66. Greedy Algorithm
Many algorithms are designed to solve optimization problems.
The goal of such problems is to find a solution to the given problem that either minimizes or maximizes the value of some parameter.
For Example: Finding a route between two cities with smallest total mileage,
Determining a way to encode messages using the fewest bits possible etc

67. Simplest approaches often leads to a solution of an optimization problem.
This approach selects the best choice at each step, instead of considering all sequences of steps that may lead to an optimal solution.
Algorithms that make what seems to be the "best“ choice at each step are called greedy algorithms.

68. Designing a Greedy Algorithm
Cast the problem so that we make a greedy (locally optimal) choice and are left with one subproblem
Prove there is always a (globally) optimal solution to the original problem that makes the greedy choice
Show that the choice together with an optimal solution to the subproblem gives an optimal solution to the original problem

69. Example
Consider the problem of making n cents change with quarters, dimes, nickels, and pennies, and using the least total number of coins.
We can devise a greedy algorithm for making change for n cents by making a locally optimal choice at each step;
that is, at each step we choose the coin of the largest denomination possible to add to the pile of change without exceeding n cents

70. For example, to make change for 67 cents, we first select a quarter (leaving 42 cents). We next select a second quarter (leaving 1 7 cents), followed by a dime (leaving 7 cents), followed by a nickel (leaving 2 cents), followed by a penny (leaving 1 cent), followed by a penny.