Sorting Sorting: –Task of rearranging data in an order. –Order can be: Ascending Order: –1,2,3,4,5,6,7,8,9 Descending Order: –9,8,7,6,5,4,3,2,1 Lexicographic Order: –Dictionary Order »ada, bat, cat, mat, max, may, min

Sorting Terminologies: Stable Sort Unsorted Sorted

Sorting Terminologies: Unsorted Array A Sorted Array B Unsorted Array A Sorted Array A Not In Place In Place Sorting

Sorting Terminologies: –Stable Sort: A list of unsorted data may contain two or more equal data. If a sorting method maintains, –The same relative position of their occurrences in the sorted list, then it is called: –Stable Sort –In Place Sort: Suppose a set of data to be stored is stored in an array A. If a sorting method takes place, –Within the array A only, that is, without using any other extra storage space, then it is called: –In Place Sorting Method »Memory efficient because it does not require extra memory space.

Sorting Methods of Sorting: –Various sorting methods are: Bubble Sort Selection Sort Insertion Sort Quick Sort Merge Sort Heap Sort

Bubble Sort Pass Pass Pass A[1] vs A[2] A[2] vs A[3] A[3] vs A[4] A[1] vs A[2] A[2] vs A[3] A[3] vs A[4] A[1] vs A[2] A[2] vs A[3] A[3] vs A[4] n = 4 Array A

Bubble Sort Pass-1 Pass-2 Pass-3 A[1] vs A[2] A[2] vs A[3] A[3] vs A[4] A[1] vs A[2] A[2] vs A[3] A[3] vs A[4] A[1] vs A[2] A[2] vs A[3] A[3] vs A[4] for i = 1 to n-1 do // Controls number of passes for j = 1 to n-1 do // Controls comparisons in each pass if(A[ j ] > A[ j+1 ]), then Swap(A[ j ], A[ j+1 ]) EndIf EndFor Stop Any optimization possible? n = 4

Bubble Sort Pass Pass Pass A[1] vs A[2] A[2] vs A[3] A[3] vs A[4] A[1] vs A[2] A[2] vs A[3] A[1] vs A[2]

Bubble Sort Pass-1 Pass-2 Pass-3 A[1] vs A[2] A[2] vs A[3] A[3] vs A[4] A[1] vs A[2] A[2] vs A[3] A[1] vs A[2] n = 4 for i = 1 to n-1 do //Controls number of passes for j = 1 to n-i do //Controls comparisons in each pass if(A[ j ] > A[ j+1 ]), then Swap(A[ j ], A[ j+1 ]) EndIf EndFor Stop i = 1 i = 2 i = 3 j = 1 to 3 j = 1 to 2 j = 1 to 1

Bubble Sort Algorithm: –BubbleSort Input: –Array A[1...n] where n is the number of elements. Output: –Array A with all elements in ascending sorted order. Data Structure: –Array A[1...n]

Bubble Sort Steps: for i = 1 to n-1 do //Controls the number of passes for j = 1 to n-i do //Controls the comparisons in each pass if(A[ j ] > A[ j+1 ]), then //Logic to swap temp = A[ j ] A[ j ] = A[ j+1 ] A[ j+1 ] = temp EndIf EndFor Stop

Tracing of Bubble Sort Pass Pass Pass A[1] vs A[2] A[2] vs A[3] A[3] vs A[4] A[1] vs A[2] A[2] vs A[3] A[1] vs A[2]

Tracing of Bubble Sort Pass Pass Pass A[1] vs A[2] A[2] vs A[3] A[3] vs A[4] A[1] vs A[2] A[2] vs A[3] A[1] vs A[2]

Tracing of Bubble Sort Pass Pass Pass A[1] vs A[2] A[2] vs A[3] A[3] vs A[4] A[1] vs A[2] A[2] vs A[3] A[1] vs A[2]

Tracing of Bubble Sort Exercise

Bubble Sort Analysis of Algorithm BubbleSort: –Performance of any sorting algorithm depends on: Number of comparisons Number of movements (Swappings)

Analysis of algorithm BubbleSort Summary CaseComplexity Best Case List already in sorted order. T(n) = n 2 Worst Case List sorted in reverse order. T(n) = n 2 Average Case List sorted in random order. T(n) = n 2

Selection Sort Input Array A min_val min_loc Pass-1 Assumption: Element at location 1 should be minimum Pass-2 Assumption: Element at location 2 should be minimum. min_val min_loc Minimum element should have been at location 1. Is it at same location? No SWAP Minimum element should have been at location 2. Is it at same location? No SWAP

Selection Sort Pass-3 Assumption: Element at location 3 should be minimum. min_val min_loc Minimum element should have been at location 3. Is it at same location? No SWAP Pass-4 Assumption: Element at location 4 should be minimum. min_val min_loc Minimum element should have been at location 4. Is it at same location? Yes NO SWAP Output

Selection Sort Input Array A Pass-1 Assumption: Element at location 1 should be minimum. Minimum element should have been at location 1. Is it at same location? No SWAP min_loc min_val min_loc min_val min_loc min_val min_loc min_val min_loc min_val Pass-2 Assumption: Element at location 2 should be minimum. Minimum element should have been at location 2. Is it at same location? No SWAP min_loc min_val min_loc min_val min_loc min_val

Pass-3 Assumption: Element at location 3 should be minimum. Minimum element should have been at location 3. Is it at same location? Yes NO SWAP min_loc min_val Selection Sort Pass-4 Assumption: Element at location 4 should be minimum. Minimum element should have been at location 4. Is it at same location? Yes NO SWAP min_loc min_val Output

Selection Sort For i = 1 to n-1, do min_val = A[ i ], min_loc = i For j = i+1 to n, do If(min_val > A[ j ]), then min_val = A[ j ] min_loc = j EndIf EndFor If(i != min_loc) Swap(A[ i ], A[min_loc]) EndIf EndFor Stop To sort 5 elements, 4 passes were required. To sort n elements, n-1 passes are required. In 1 st pass, value at location 1 will be smallest. In 2 nd pass, value at location 2 will be smallest. In i-th pass, value at location i will be smallest. In 1 st pass, we need to do comparisons from 2 nd value. In 2 nd pass, we need to do comparisons from 3 rd value. In ith pass, we need to start comparisons from (i+1)th value upto n.

Selection Sort Algorithm: –SelectionSort Input: –Array A[1...n] where n is the number of elements. Output: –Array A with all elements in ascending sorted order. Data Structure: –Array A[1...n]

Selection Sort Steps: For i = 1 to n-1, do //Controls the number of passes min_val = A[ i ], min_loc = i For j = i+1 to n, do //Controls the comparisons in each pass If(min_val > A[ j ]), then min_val = A[ j ] min_loc = j EndIf EndFor If(i != min_loc) //Swap A[ i ] and A[min_loc] temp = A[ i ] A[ i ] = A[min_loc] A[min_loc] = temp EndIf EndFor

Selection Sort Input Array A Pass-1 Assumption: Element at location 1 should be minimum. Minimum element should have been at location 1. Is it at same location? Yes NO SWAP min_loc min_val Pass-2 Assumption: Element at location 2 should be minimum. Minimum element should have been at location 2. Is it at same location? Yes NO SWAP min_loc min_val

Pass-3 Assumption: Element at location 3 should be minimum. Minimum element should have been at location 3. Is it at same location? Yes NO SWAP min_loc min_val Selection Sort Pass-4 Assumption: Element at location 4 should be minimum. Minimum element should have been at location 4. Is it at same location? Yes NO SWAP min_loc min_val Output

Tracing of Selection Sort Input Array A Pass Min Pass Min Pass Min Pass Min No Swap

Array A Tracing of Selection Sort

Analysis of algorithm SelectionSort Summary CaseComplexity Best Case List already in sorted order. T(n) = n 2 Worst Case List sorted in reverse order. T(n) = n 2 Average Case List sorted in random order. T(n) = n 2

Insertion Sort Insertion Sort: –Based on a method called: Bridge Player –The way Bridge Player sort their hands. –Picking up one card at a time, placing into its appropriate position.

Insertion Sort Array A Array B

Insertion Sort B[1] = A[1] For i = 2 to n, do //Pick the element KEY = A[ i ] //Find appropriate location by comparison location = i While(location > 1) AND (KEY < B[location-1]), do location = location - 1 EndWhile //Shift elements if required j = i While( j > location ) B[ j ] = B [ j – 1 ] j = j - 1 EndWhile //Place the element B[location] = KEY EndFor 1 st element - Direct For all other elements 1. Pick the element 2. Find appropriate position / location. 3. Shift existing elements if required. 4. Place the element.

Insertion Sort Algorithm: –InsertionSort Input: –Array A[1...n] where n is the number of elements. Output: –Array B[1...n] with elements sorted in ascending order. Data Structure: –Array A[1...n] and B[1...n]

B[1] = A[1] For i = 2 to n, do //Pick the element KEY = A[ i ] //Find appropriate location by comparison location = i While(location > 1) AND (KEY < B[location-1]), do location = location - 1 EndWhile //Shift elements if required j = i While( j > location ) B[ j ] = B [ j – 1 ] j = j - 1 EndWhile //Place the element B[location] = KEY EndFor Steps of InsertionSort

Tracing of Insertion Sort Array A Array B Iteration

Tracing of Insertion Sort Array A

Analysis of algorithm InsertionSort Summary CaseComplexity Best Case List already in sorted order. T(n) = n Worst Case List sorted in reverse order. T(n) = n 2 Average Case List sorted in random order. T(n) = n 2

Sorting Divide & Conquer Break all the sticks into 2 parts and arrange them in order. Solution: 1) Divide: Separate the sticks from the bundle of sticks so that it could be broken. 2) Conquer (Solve / Win): Break each and every stick into 2 parts (which can be easily done) 3) Combine: Arrange all the parts in an order.

Quick Sort Quick Sort: –To sort an array, it uses the concept of: Divide & Conquer –Process: Divide a large list/array into a number of smaller lists/arrays. Sort them separately. Combine the results to get the sorted list.

Quick Sort Divide & Conquer Problem Divide P1P1 P2P2 P3P3 PnPn.... Combine Solution Solve

Quick Sort left loc Scan from right to left right right loc left Scan from left to right left loc right X X While(A[loc] <= A[right])While(A[loc] >= A[left]) right = right - 1left = left + 1

Quick Sort leftright loc Scan from right to left right While(A[loc] <= A[right]) right = right - 1 AND (loc < right) Scan from left to right While(A[loc] >= A[left]) left = left + 1 AND (loc > left)

Quick Sort leftright loc Scan from right to left right loc left Scan from left to right loc right loc left While (A[loc] <= A[right]) AND loc < right right = right - 1 While (A[loc] >= A[left]) AND loc > left left = left + 1

Quick Sort Partition Algorithm Initialize loc to left loc = left While(left < right), do //Scan from right to left While(A[loc] <= A[right]) AND (loc < right), do right = right - 1 EndWhile If(A[loc] > A[right]), then Swap(A[loc], A[right]) loc = right left = left + 1 EndIf //Scan from left to right While(A[loc] >= A[left]) AND (loc > left), do left = left + 1 EndWhile If(A[loc] < A[left]), then Swap(A[loc], A[left]) loc = left right = right - 1 EndIf EndWhile Scan from right to left Check as to why the scanning stopped. Repeat the scannings till left and right do not meet. Scan from left to right

Quick Sort leftright loc Scan from right to left right loc left Scan from left to right loc right loc left While (A[loc] <= A[right]) AND loc < right right = right - 1 While (A[loc] >= A[left]) AND loc > left left = left + 1

Quick Sort leftright loc leftright loc left 22 1 leftright loc right 99 5 leftright loc

Quick Sort Algorithm: –QuickSort Input: –L: Lower Bound of Array A. –U: Upper Bound of Array A. Output: –Array A with elements sorted in ascending order. Data Structure: –Array.

Steps: left = L, right = U if(left < right), then loc = Partition(left, right) QuickSort(left, loc – 1) QuickSort(loc + 1, right) EndIf Stop Quick Sort

Algorithm: –Partition Input: –left: Index of the leftmost element in Array A. –right: Index of the rightmost element in Array A. Output: –loc: Final position of the pivot element. Data Structure: –Array. Remark: –Element located at left is taken as the pivot element.

loc = left While(left < right), do //Scan from right to left While(A[loc] <= A[right]) AND (loc < right), do right = right - 1 EndWhile If(A[loc] > A[right]), then Swap(A[loc], A[right]) loc = right left = left + 1 EndIf //Scan from left to right While(A[loc] >= A[left]) AND (loc > left), do left = left + 1 EndWhile If(A[loc] < A[left]), then Swap(A[loc], A[left]) loc = left right = right - 1 EndIf EndWhile

Tracing of Quick Sort

Quick Sort Which is the best case of QuickSort out of above 3 options? Best Case would be the case in which equal partitions are done.

Analysis of algorithm QuickSort Summary CaseComplexity Worst Case List already in sorted order. T(n) = n 2 Worst Case List sorted in reverse order. T(n) = n 2 Best Case / Average Case List sorted in random order. T(n) = n log 2 n

Tracing of Quick Sort

Merge Sort Merge Sort: –Like Quick Sort, it also works on the basic principle of: Divide & Conquer –It uses a concept / technique called: Merging

in1jn2k Sorted Array A Sorted Array B Array C jkikjkikjki 4 kjkjkjk 7 10

Merge Sort Merging Algorithm Initialize i, j, k to 1 i = 1, j = 1, k = 1 While( i <= n1 ) AND ( j <= n2 ), do If ( A[ i ] <= B [ j ] ) C[ k ] = A[ i ] i = i + 1, k = k + 1 Else C[ k ] = B[ j ] j = j + 1, k = k + 1 EndIf EndWhile If ( i > n1 ), then While( j <= n2 ) C[ k ] = B[ j ] j = j + 1, k = k + 1 EndWhile Else If ( j > n2 ), then While( i <= n1 ) C[ k ] = A[ i ] i = i + 1, k = k + 1 EndWhile EndIf Compare elements from both the arrays until 1 array finishes. Now check which array has finished. Move the elements of other array one by one.

Merge Sort LRmid LmidRL R LmidRLRL RLR LRLRLRLR Array A

Merge Sort Array A Output

Merge Sort Algorithm: –MergeSort Input: –L: Lower Bound of Array A. –R: Upper Bound of Array A. Output: –Array A with elements sorted in ascending order. Data Structure: –Array.

Steps: if(L< R), then mid = floor((L+R) / 2) MergeSort(L, mid) MergeSort(mid + 1, R) Merge(L, mid, R) EndIf Stop MergeSort

Algorithm: –Merge Input: –L: Lower Bound of 1 st sub-array of Array A. –mid: Upper Bound of 1 st sub-array of Array A. mid + 1 will be lower bound of 2 nd sub-array of Array A. –R: Upper Bound of 2 nd sub-array of Array A. Output: –Two sub-arrays are merged and sorted in the array A. Data Structure: –Array. Assumption: –Extra storage space of array C.

Merge Algorithm Steps: i = L, j = mid + 1, k = L While( i <= mid ) AND ( j <= R ), do If ( A[ i ] <= A [ j ] ) C[ k ] = A[ i ] i = i + 1, k = k + 1 Else C[ k ] = A[ j ] j = j + 1, k = k + 1 EndIf EndWhile If ( i > mid ), then While( j <= R ) C[ k ] = A[ j ] j = j + 1, k = k + 1 EndWhile Else If ( j > R ), then While( i <= mid ) C[ k ] = A[ i ] i = i + 1, k = k + 1 EndWhile EndIf For m = L to k – 1, do A[ m ] = C[ m ] m = m + 1 EndFor

Quick Sort v/s Merge Sort if(left < right), then loc = Partition(left, right) QuickSort(left, loc – 1) QuickSort(loc + 1, right) //Not required EndIf Stop if(L< R), then mid = floor((L+R) / 2) MergeSort(L, mid) MergeSort(mid + 1, R) Merge(L, mid, R) EndIf Stop Quick Sort Merge Sort Divide Conquer Combine HARD DIVISION, EASY COMBINATION EASY DIVISION, HARD COMBINATION Equal sub-division is not always guaranteed. Both the sub-problems are of almost equal size always.