1. Chapter 7: Sorting (Insertion Sort, Shellsort)
CE 221
Data Structures and Algorithms
Chapter 7: Sorting (Insertion Sort, Shellsort)
Text: Read Weiss, § 7.1 – 7.4
Izmir University of Economics

2. Izmir University of Economics
Preliminaries
Main memory sorting algorithms
All algorithms are Interchangeable; an array containing the N elements will be passed.
“<“, “>” (comparison) and “=“ (assignment) are the only operations allowed on the input data : comparison-based sorting
Izmir University of Economics

3. Contents
Selection Sort
Insertion Sort
Shell Sort

4. CE 221
Selection Sort

5. Selection Sort
Given the input in array a[N] below, compare a[i] to a[min].

6. Selection Sort

7. Selection Sort

8. Selection Sort

9. Selection Sort

10. Selection Sort

11. Selection Sort

12. Selection Sort

13. Selection Sort

14. Selection Sort

15. Selection Sort

16. Selection Sort

17. Selection Sort

18. Selection Sort

19. Selection Sort

20. Selection Sort

21. Selection Sort

22. Selection Sort

23. Selection Sort

24. Time Analysis of Selection Sort
Comparisons:
Exchanges: N

25. CE 221
Insertion Sort

26. Insertion Sort

27. Insertion Sort

28. Insertion Sort

29. Insertion Sort

30. Insertion Sort

31. Insertion Sort

32. Insertion Sort

33. Insertion Sort

34. Insertion Sort

35. Insertion Sort

36. Insertion Sort

37. Insertion Sort

38. Insertion Sort

39. Time Analysis
Number of comparisions:
Number of excahnges:

40. Effective Algorithms
Shell Sort

41. Shell Sort
Move entries more than one position at a time by h–sorting the array
Start at L and look at every 4th element and sort it
Start at E and look at every 4th ekement and sort it
Start at A and look at every 4th ekement and sort it

42. Decreasing sequence of values of h

43. Example

44. Example: S O R T E X A M P L E
Result: A E E L M O P R S T X

45. Implementation

46. Time Analysis

47. Shell Sort vs Insertion Sort
Insertion sort compares every single item with all the rest elements of the list in order to find its place,
Shell sort compares items that lie far apart. This makes smaller elements to move faster to the front of the list.

48. Yes !!!

49. Insertion Sort Analysis
One of the simplest sorting algorithms
Consists of N-1 passes.
for pass p = 1 to N-1 (0 thru p-1 already known to be sorted)
elements in position 0 trough p (p+1 elements) are sorted by moving the element left until a smaller element is encountered.
Izmir University of Economics

50. Insertion Sort - Algorithm
N*N iterations, hence, time complexity = O(N2) in the worst case.This bound is tight (input in the reverse order). Number of element comparisons in the inner loop is p, summing up over all p = (1+2)+(1+3)+...+N-1 = Θ(N2). If the input is sorted O(N).
Izmir University of Economics

51. A Lower Bound for Simple Sorting Algorithms
An inversion in an array of numbers is any ordered pair (i, j) such that a[i] > a[j], here j < i !
In the example set we have I = 9 as the number of inversions,
(34,8),(34,32),(34,21),(64,51),(64,32),(64,21),(51,32),(51,21), and (32,21).
Swapping two adjacent elements that are out of place removes exactly one inversion and a sorted array has no inversions.
The running time of insertion sort O(I+N).
If the I = O(N), then the insertion sort runs in linear time.
Izmir University of Economics

52. Average Running Time for Simple Sorting - I
Theorem: The average number of inversions in an array with N distinct elements is the average number of inversions in a permutation, i.e., N(N-1)/4.
Proof: Consider a list L, which has a corresponding reverese ordered list LR that is in revers order (21,32,51,64,8,34).
The total number of inversion pairs in the list L ant its reverse LR is N(N-1)/2.Thus an average list has half of this amount, i.e.,
(N(N-1)/4)/2 = N(N-1)/2.
Izmir University of Economics

53. Average Running Time for Simple Sorting - II
Theorem: Any algorithm that sorts by exchanging adjacent elements requires Ω(N2) time on average.
Proof: Initially there exists N(N-1)/4 inversions on the average and each swap removes only one inversion, so Ω(N2) swaps are required.
This is valid for all types of sorting algorithms (including those undiscovered) that perform only adjacent exchanges.
Result: For a sorting algorithm to run subquadratic (o(N2)), it must exchange elements that are far apart (eliminating more than just one inversion per exchange).
Izmir University of Economics

54. Izmir University of Economics
Shellsort - I
Shellsort, invented by Donald Shell, works by comparing elements that are distant; the distance decreases as the algorithm runs until the last phase (diminishing increment sort)
Sequence h1, h2, ..., ht is called the increment sequence. h1 = 1 always. After a phase, using hk, for every i, a[i] ≤ a[i+hk]. The file is then said to be hk-sorted.
Izmir University of Economics

55. Izmir University of Economics
Shellsort - II
An hk-sorted file that is then hk-1-sorted remains hk-sorted.
To hk-sort, for each i in hk,hk+1,...,N-1, place the element in the correct spot among i, i-hk, i-2hk. This is equivalent to performing an insertion sort on hk independent subarrays.
Izmir University of Economics

56. Izmir University of Economics
Shellsort - III
Increment sequence by Shell: ht=floor(N/2), hk=floor(hk+1/2) (poor)
Izmir University of Economics

57. Worst-Case Analysis of Shellsort - I
Theorem: The worst case running time of Shellsort, using Shell’s increments, is Θ(N2).
Proof: part I: prove Ω(N2)= Why? smallest N/2 elements goes from position 2i-1 to i during the last pass. Previous passes all have even increments.
Izmir University of Economics

58. Worst-Case Analysis of Shellsort - II
Proof: part II: prove O(N2)
A pass with increment hk consists of hk insertion sorts of about N/hk elements. One pass,hence, is O(hk(N/hk)2). Summing over all passes which is O(N2).
Shell’s increments: pairs of increments are not relatively prime.
Hibbard’s increments: 1, 3, 7,..., 2k-1
Izmir University of Economics

59. Worst-Case Analysis of Shellsort - III
Theorem: The worst case running time of Shellsort, using Hibbard’s increments, is Θ(N3/2).
Proof: (results from additive number theory)
- for hk>N1/2 use the bound O(N2/hk) // hk=1, 3, 7,..., 2t-1
hk+2 done, hk+1done, hk now.
a[p-i] < a[p] if
but hk+2=2hk+1+1 hence gcd(hk+2, hk+1) = 1
Thus all can be expressed as such .
Therefore; innermost for loop executes O(hk) times for each N-hk positions. This gives a bound of O(Nhk) per pass.
OPTONAL
Izmir University of Economics

60. Izmir University of Economics
Homework Assignments
7.1, 7.2, 7.3, 7.4
You are requested to study and solve the exercises. Note that these are for you to practice only. You are not to deliver the results to me.
Izmir University of Economics