1. ME 171 Computer Programming Language
Partha Kumar Das
Lecturer,
Department of Mechanical Engineering, BUET
Lecture 6
Algorithms: Sorting

2. Algorithm of taking your lunch
Algorithms
In mathematics and computer science, an algorithm is a self-contained step-by-step set of operations to be performed for a given problem.
When an algorithm is developed to certain problem, then source code is written based on the algorithm.
Algorithm of taking your lunch
Find your best algorithm to take your lunch in a shortest steps.

3. Algorithms Lets Play A Game
A particular problem can be solved using different types of algorithms.
The most efficient algorithms can be defined based on the following factors:
1. Execution Time
2. Number of statements in the source code
3. Complexity in understanding and
4. Conformation of the solution
Lets Play A Game
I have chosen a particular number from the sorted list below. Guess the number by making guesses until you find the number. 1 2 3 4 5 6 7 8 9
A novice one would guess the first number(1) and then second and repeat it until he gets the chosen number (3).
But a good one would choose mid number at each turn so that he will need less turns to select the guessed number.

4. Sorting
Suppose you are a librarian whose task is to maintain the books of the same category in one row. Then you have to sort the books of the same categories, give them codes and again sort each book in ascending or descending order so that reader could easily find desired book.
Several algorithms have been developed to sort an array of numbers.
From the birth of computer knowledge, the algorithm for sorting achieved a great appeal due to its simplicity but potential and complexity to find out the most efficient algorithm to solve.
Some most common are:
Bubble Sort (Most inefficient)
Selection Sort
Insertion Sort
Shell Sort
Quick Sort
Bucket Sort
Merge Sort
Binary Sort
Tim Sort, etc.
Source; Wikipedia

5. Bubble Sort The most ancient sorting algorithm developed in 1956.
It is the most simple and inefficient and merely used for sorting an approximately fully sorted array in which some elements have accidentally changed position.
The algorithm works by comparing each item in the list with the item next to it, and swapping them if required.
In other words, the largest element has bubbled to the top of the array.
The algorithm repeats this process until it makes a pass all the way through the list without swapping any items.
Example: Sorting of 7, 5, 2, 4, 3, 9 using Bubble sort algorithm:
1st Pass
2nd Pass
3rd Pass
4th Pass
7, 5, 2, 4, 3, 9 5, 7, 2, 4, 3, 9 5, 2, 7, 4, 3, 9 5, 2, 4, 7, 3, 9 5, 2, 4, 3, 7, 9 5, 2, 4, 3, 7, 9
5, 2, 4, 3, 7, 9
2, 5, 4, 3, 7, 9 2, 4, 5, 3, 7, 9 2, 4, 3, 5, 7, 9 2, 4, 3, 5, 7, 9 2, 4, 3, 5, 7, 9
2, 4, 3, 5, 7, 9
2, 4, 3, 5, 7, 9 2, 3, 4, 5, 7, 9 2, 3, 4, 5, 7, 9 2, 3, 4, 5, 7, 9 2, 3, 4, 5, 7, 9
2, 3, 4, 5, 7, 9

6. #include <stdio.h>
int main() {
int i,j,temp,array[5];
printf("Input an array of 5 elements:\n");
for(i=0;i<5;i++)
scanf("%d",&array[i]);
for(i=0;i<5;i++) // O(i*i)
for(j=0;j<=i;j++) // O(i)
if(array[j]>array[i]) // O(1)
temp=array[j];
array[j]=array[i];
array[i]=temp; }
printf("The sorted array is:\n");
printf("%d\t",array[i]);
return 0;}
The worst-case runtime complexity is O(n2).

7. Insertion Sort
One of the simplest methods to sort an array is an insertion sort.
An example of an insertion sort occurs in everyday life while playing cards. To sort the cards in your hand you extract a card, shift the remaining cards, and then insert the extracted card in the correct place.
This process is repeated until all the cards are in the correct sequence.
Example: Sorting 26, 20, 73, 34, 64 using insertion sort algorithm:
29, 20, 73, 34, 64  1st turn 29, 20, 73, 34, 64  2nd turn 20, 29, 73, 34, 64  3rd turn 20, 29, 73, 34, 64  4th turn 20, 29, 34, 73, 64  5th turn 20, 29, 34, 64, 73  6th turn
we need 0 comparisons to insert the first element we need 1 comparison to insert the second element we need 2 comparisons to insert the third element ... we need (N-1) comparisons (at most) to insert the last element.
Totally, (N-1) = O(n2)
The worst-case runtim ecomplexity is O(n2).
The best-case runtime complexity is O(n). 

8. Shell Sort
Shell sort, developed by Donald L. Shell improves on the efficiency of insertion sort by quickly shifting values to their destination.
Average sort time is O(n^7/6), while worst-case time is O(n^4/3)
Insertion Sort
Shell Sort
Total number of shift in insertion sort: 2+2+1=5
Total number of shift in shell sort: 1+1+1=3

9. Shell Sort
Shell sort begins by an insertion sort using a spacing of two. In the first frame we examine numbers 3-1. Extracting 1, we shift 3 down one slot for a shift count of 1. Next we examine numbers 5-2. We extract 2, shift 5 down, and then insert 2. After sorting with a spacing of two, a final pass is made with a spacing of one. This is simply the traditional insertion sort. The total shift count using shell sort is = 3. By using an initial spacing larger than one, we were able to quickly shift values to their proper destination.
Insertion Sort
Shell Sort
Total number of shift in insertion sort: 2+2+1=5
Total number of shift in shell sort: 1+1+1=3

10. Shell Sort Various spacings may be used to implement a shell sort.
Typically the array is sorted with a large spacing, the spacing reduced, and the array sorted again.
On the final sort, spacing is one.
Although the shell sort is easy to comprehend, formal analysis is difficult.
The question of deciding which spacing to use is difficult.
Every spacing that contains 1 yields a correct sort.
There numerous formulas of spacings published so far.
Knuth recommends a technique, due to Sedgewick, that determines spacing h based on the following formulas: hs = 9.2 𝑠 𝑠/2 + 1, if s is even hs = 8.2 𝑠 (𝑠+1)/2 + 1, if s is odd These calculations result in values (h0, h1, h2,…) = (1, 5, 19, 41, 109, 209,…). Calculate h until 3ht >=N, where N is the number of elements in the array. Then choose ht-1 for a starting value. For example, to sort 150 items, ht = 109 (cause, 3*109 >= 150), so the first spacing is ht-1, or 41. The second spacing is 19, then 5, and finally 1.
Another example of shell sort using Papernov & Stasevich spacing formula (1, 3, 5, 9, ……) is below: a1 a2 a3 a4 a5 a6 a7 a8 a9
a10
a11
a12
Input data 62 83 18 53 07 17 95 86 47 69 25 28
After 5-spacing
After 3-spacing
After 1-spacing

11. Quick Sort
Quick sort is an efficient sorting algorithm, serving as a systematic method for placing the elements of an array in order.
Developed by Tony Hoare in 1959, with his work published in 1961, it is still a commonly used algorithm for sorting.
Mathematical analysis of quicksort shows that, on average, the algorithm takes O(n log n) comparisons to sort n items.
In the worst case, it makes O(n^2) comparisons, though this behavior is rare.
ALGORITHM:
Pick an element, called a pivot, from the array. Picking this pivot element is the most important decision to be made in quick sort. Always try to pick the mid element as pivot. Picking corner element will result in worst case scenario on already sorted array.
Partitioning: Reorder the array so that all elements with values less than the pivot come before the pivot, while all elements with values greater than the pivot come after it (equal values can go either way). After this partitioning, the pivot is in its final position. This is called the partition operation.
Recursively apply the above steps to the sub-array of elements with smaller values of the previous pivot and to the sub-array of elements with greater values of the previous pivot separately.

12. Selection Sort
Selection sort is performs worse than the similar insertion sort.
Selection sort is noted for its simplicity, and it has performance advantages over more complicated algorithms in certain situations, particularly where auxiliary memory is limited.
ALGORITHM:
Find the smallest number. Swap it with the first place number.
Find the second-smallest number. Swap it with the second place number.
Find the third-smallest number. Swap it with the third place number.
Repeat finding the next-smallest number, and swapping it into the correct position until the array is sorted.
When you sort exam sheets after examination, what sorting algorithm do you use, Selection or Insertion? Write the C source code of your algorithm.

13. References:
Introduction to Algorithms, by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford Stein.
Sorting and searching algorithms, by Thomas Niemann.
The Art of Computer Programming vol.: 1, 2, 3, by Donald E. Knuth.