2. Sorting
Sorting places data in ascending or descending order, based on one or more sort keys
E.g.
A list of names could be sorted alphabetically,
bank accounts could be sorted by account number,
employee payroll records could be sorted by social security number
How do you sort a list/array of data?
A sorting algorithm is said to be in-place if it requires very little additional space besides the initial array holding the elements that are to be sorted. Normally “very little” is taken to mean that for sorting nnn elements, O⁢(log⁡n)OnO(\log n) extra space is required

3. Popular sorting algorithms:
Selection sort
Insertion sort
Merge sort
Bubble sort
Quick sort
The end result (sorted array) is the same no matter which algorithm you use to sort the array.
The choice of algorithm affects only the run time and memory use of the program.
Non-Comparison Based Sorting Algorithms
In the previous lessons we discussed three basic sorting algorithms, bubble sort, insertion sort and selection sort, each of which is easy to implement, but takes O(n2) steps in average case to sort a list of size n. More advanced recursive algorithms like Quick Sort and Merge Sort are hard to implement, but runs more efficiently on a set of data, typically in O(n log n) steps. All five algorithms assume comparison of the whole key to make the decision to sort. There are simpler algorithms that can sort a list using partial information about the keys. There are two algorithms we discussed in this section called Bucket Sort and radix Sort.
Bucket Sort
The idea of the bucket sort is to place items into various buckets based on a key or partial information about a key. For example, if we have a list that contains only numbers from 0.. 99, then we can take 10 buckets and place all elements that are between 0..9 into first bucket, into second bucket etc. This is a kind of sorting that only requires O(n) steps. A more general sorting algorithm is the radix sort that works as follows.
Radix Sort
Suppose we are able to compare parts of the data to make a decision. Say we have a bunch of numbers we need to sort. Assume we have the following numbers.
12, 67, 45, 42, 18, 60
Assume we have 10 buckets we can work with. First we place each item in the bucket based on the least significant digit (that is 1 place). So we have the numbers in buckets as follows.
                        42
60                    12                                45                    67        18       
 0         1          2          3          4          5          6          7          8          9
Next we sort the numbers again by second digit (that is 10 place) and place them in buckets. In the process we need to make sure that we are not changing the order of the elements.
            18                                45                    67
            12                                42                    60                               
Now note that if we just collect the numbers from left to right, and bottom to top in each bucket, we have a sorted list of numbers. This is called the Radix Sort and complexity of the Radix Sort is O(nb) where b is the number of digits in the largest number.
Note radix sort can be applied to data that can be compared partially like integers of characters. However, Radix sort cannot be applied to data that needs to be compared as a whole (eg: nodes)

4. Selection Sort Simple sorting algorithm First iteration:
selects the smallest element in the array and swaps it with the first element.
Second iteration:
selects the second-smallest item and swaps it with the second element.
After the ith iteration:
the smallest i items of the array will be sorted into increasing order in the first i elements of the array.
Selection sort sorts an array by repeatedly finding the smallest element of the unsorted tail region and moving it to the front.
Example: sorting an array of integers

5. Sorting an Array of Integers
Find the smallest and swap it with the first element
Find the next smallest. It is already in the correct place
Find the next smallest and swap it with first element of unsorted portion
Repeat
When the unsorted portion is of length 1, we are done

6. Selection sort pseudocode:
swap(A(i), A(iMin)):
temp = A(i)
A(i) = A(iMin)
A(iMin) = temp
For i = 0 to n-1 do:
iMin = i
For j = i + 1 to n-1 do:
If A(j) < A(iMin )
iMin = j
End-If
End-For
swap(A(i), A(iMin))

7. Programming Question
Implement class SelectionSorter . Method sort should accepts an integer array as parameter and perform selection sort on it.
Program template in next slide

8. public class SelectionSorter{
public static void sort(int[] a) {
//TODO: implement selection sort }
public static int[] randomIntArray(int length, int n) {
Random generator = new Random();
int[] a = new int[length];
for (int i = 0; i < a.length; i++)
a[i] = generator.nextInt(n);
return a;
public static void main(String[] args) {
int[] a = randomIntArray(20, 100);
System.out.println("Before: "+ Arrays.toString(a));
sort(a);
System.out.println("After: "+ Arrays.toString(a));

9. Answer SelectionSorter.java import java.util.Arrays;
import java.util.Random;
public class SelectionSorter{
public static void sort(int[] a) {
for (int i = 0; i < a.length - 1; i++) {
int minIndex = i;
//find index of minimum value element (minIndex) from indices i+1 to n-1 (unsorted section)
for (int j = i + 1; j < a.length; j++)
if (a[j] < a[minIndex])
minIndex = j;
//swap element at indices i and minPos
int tmp = a[i];
a[i] = a[minIndex];
a[minIndex] = tmp; }
public static int[] randomIntArray(int length, int n) {
Random generator = new Random();
int[] a = new int[length];
for (int i = 0; i < a.length; i++)
a[i] = generator.nextInt(n);
return a;
public static void main(String[] args) {
int[] a = randomIntArray(20, 100);
System.out.println("Before: "+ Arrays.toString(a));
sort(a);
System.out.println("After: "+ Arrays.toString(a));

10. Analyzing the Performance of the Selection Sort Algorithm
In an array of size n, count how many times an array element is visited:
To find the smallest, visit n elements + 2 visits for the swap
To find the next smallest, visit (n - 1) elements + 2 visits for the swap
The last term is 2 elements visited to find the smallest + 2 visits for the swap
For i = 0 to n-1 do:
iMin = i
For j = i + 1 to n-1 do:
If A(j) < A(iMin )
iMin = j
End-If
End-For
swap(A(i), A(iMin))
public static void main(String[] args) {
int[] a = randomIntArray(3000, 100);
System.out.println("Before: "+ Arrays.toString(a));
long start = System.currentTimeMillis();
sort(a);
long end = System.currentTimeMillis();
System.out.println("After: "+ Arrays.toString(a));
System.out.println("tme taken (ms): "+ (end-start) ); }

11. Analyzing the Performance of the Selection Sort Algorithm
The number of visits:
n (n - 1) (n - 2)
This can be simplified to n2 /2  +  5n/2  - 3
5n/2 - 3 is small compared to n2 /2 – so let's ignore it
Also ignore the 1/2 – it cancels out when comparing ratios

12. Analyzing the Performance of the Selection Sort Algorithm
The number of visits is of the order n2 .
Computer scientists use the big-Oh notation to describe the growth rate of a function.
Using big-Oh notation: The number of visits is O(n2).
To convert to big-Oh notation: locate fastest-growing term, and ignore constant coefficient.

13. Programming Question
Experimentally measure time it takes selection sort to sort an array for following array sizes :
1000, 2000, 3000, …., 10000
Then plot time(y axis) against the array size(n)
long start = System.currentTimeMillis();
sort(a);
long end = System.currentTimeMillis();
System.out.println("tme taken (ms): "+ (end-start) );

14. Answer O(n2) pattern public static void main(String[] args) {
for (int i=1000;i<=10000;i+=1000) {
int[] a = randomIntArray(i, 100);
long start = System.currentTimeMillis();
sort(a);
long end = System.currentTimeMillis();
System.out.println("tme taken (ms): for size "+i+" = "+ (end-start) ); }
O(n2) pattern

15. Common Big-Oh Growth Rates
Growth rate increases
downwards

16. Insertion Sort
Like selection sort, maintains a sorted portion(left) and a unsorted portion (right)in array
Fist iteration starts with second element in array and insert it into correct position in sorted portion in array
The second iteration looks at the third element and inserts it into the correct position with respect to the first two, so all three elements are in order.
and so on….

17. Insertion Sort
Assume initial sequence a[0] a[k] is sorted (k = 0):
Add a[1]; element needs to be inserted before 11
Add a[2]
Add a[3]
Finally, add a[4]

18. Algorithm: for(i=1;i<a.length;i++)//elements in unsorted section {
//shift all elements in sorted section> a[i] one position right to make room for a[i]
//insert a[i] }
Consider : | 4 2 1
Sorted:
Unsorted:
Next to insert: 4
Dry run code
Temp = a[3] = 4
j=2
a[2]=6>4  a[3] = a[2] = 6
J-- ; j=1
a[1]=5>4  a[2] = a[1] = 5
a[0]=3>4 X
Copy temp to a[1] = a[j++]

19. Programming Question
Implement InsertionSorter.java. You can write your sorting code in sort method that accepts an int array and call it from main.
Program template in next slide

20. public class InsertionSorter{
public static void sort(int[] a) {
//TODO: implement insertion sort }
public static int[] randomIntArray(int length, int n) {
Random generator = new Random();
int[] a = new int[length];
for (int i = 0; i < a.length; i++)
a[i] = generator.nextInt(n);
return a;
public static void main(String[] args) {
int[] a = randomIntArray(20, 100);
System.out.println("Before: "+ Arrays.toString(a));
sort(a);
System.out.println("After: "+ Arrays.toString(a));

21. Answer import java.util.Arrays; import java.util.Random;
public class InsertionSorter{
public static void sort(int[] a) {
for (int i = 1; i < a.length; i++) {
int temp = a[i]; //element in unsorted section to be inserted in sorted section
int j;
for(j = i-1;j>=0 && temp<a[j] ;j--) //for each element larger than temp in sorted section {
a[j+1]= a[j]; //shift one position right }
a[j+1] = temp; //insert temp in right position
public static int[] randomIntArray(int length, int n) {
Random generator = new Random();
int[] a = new int[length];
for (int i = 0; i < a.length; i++)
a[i] = generator.nextInt(n);
return a;
public static void main(String[] args) {
int[] a = randomIntArray(100, 100);
System.out.println("Before: "+ Arrays.toString(a));
sort(a);
System.out.println("After: "+ Arrays.toString(a));

22. Insertion Sort
Insertion sort is the method that many people use to sort playing cards. Pick up one card at a time and insert it so that the cards stay sorted.
Insertion sort is an O(n2) algorithm.
for (int i = 1; i < a.length; i++) {
int temp = a[i];
int j;
for(j = i-1;j>=0 && temp<a[j] ;j--)
a[j+1]= a[j]; }
a[j+1] = temp;

23. Merge Sort Sorts an array by
Cutting the array in half
Recursively sorting each half
Merging the sorted halves
Dramatically faster than the selection sort In merge sort, one sorts each half, then merges the sorted halves.

24. Merge Sort Example Divide an array in half and sort each half
Merge the two sorted arrays into a single sorted array

25. Programming Question
Implement MergeSorter.java. You can write your sorting code in sort method that accepts an int array and call it from main.
Note that the base case is an array with only one element and it simply return the same array as sorted array.
Steps:
Break array into halves/two sub arrays first, second
Sort first half
Sort second half
Merge first and second half
Implement a method merge for this. Method should accept 3 parameters (first, second, and array into which you want to merge first and second):
public static void sort(int[] a)
private static void merge(int[] first, int[] second, int[] a)

26. Merge Sort
public static void sort(int[] a) { if (a.length <= 1) { return; } int[] first = new int[a.length / 2]; int[] second = new int[a.length - first.length]; // Copy the first half of a into first, the second half into second sort(first); sort(second); merge(first, second, a); }
[0] [1] [2] [3] [4] [5] [6] [7]
First =[2,3,1,48] second = [8, 5, 36, 6]
Sort(first) = [1,2,3,48] //result in further dividing sub array and merging
Sort(second) = [5, 6,8,36] //same
merge(first, second) = [1,2,3,5,6,8,36,48] //not how pointers move in merge code

27. section_4/MergeSorter.java Continued
1 /** 2 The sort method of this class sorts an array, using the merge 3 sort algorithm. 4 */ 5 public class MergeSorter 6 { 7 /** 8 Sorts an array, using merge sort. a the array to sort 10 */ 11 public static void sort(int[] a) 12 { 13 if (a.length <= 1) { return; } 14 int[] first = new int[a.length / 2]; 15 int[] second = new int[a.length - first.length]; 16 // Copy the first half of a into first, the second half into second 17 for (int i = 0; i < first.length; i++) 18 { 19 first[i] = a[i]; 20 } 21 for (int i = 0; i < second.length; i++) 22 { 23 second[i] = a[first.length + i]; 24 } 25 sort(first); 26 sort(second); 27 merge(first, second, a); 28 } 29
Continued

28. section_4/MergeSorter.java Continued
30 /** 31 Merges two sorted arrays into an array first the first sorted array second the second sorted array a the array into which to merge first and second 35 */ 36 private static void merge(int[] first, int[] second, int[] a) 37 { 38 int iFirst = 0; // Next element to consider in the first array 39 int iSecond = 0; // Next element to consider in the second array 40 int j = 0; // Next open position in a // As long as neither iFirst nor iSecond is past the end, move 43 // the smaller element into a 44 while (iFirst < first.length && iSecond < second.length) 45 { 46 if (first[iFirst] < second[iSecond]) 47 { 48 a[j] = first[iFirst]; 49 iFirst++; 50 } 51 else 52 { 53 a[j] = second[iSecond]; 54 iSecond++; 55 } 56 j++; 57 } 58
Continued

29. section_4/MergeSorter.java
59 // Note that only one of the two loops below copies entries 60 // Copy any remaining entries of the first array 61 while (iFirst < first.length) 62 { 63 a[j] = first[iFirst]; 64 iFirst++; j++; 65 } 66 // Copy any remaining entries of the second half 67 while (iSecond < second.length) 68 { 69 a[j] = second[iSecond]; 70 iSecond++; j++; 71 } 72 } 73 }

30. section_4/MergeSortDemo.java Typical Program Run:
1 import java.util.Arrays; 2 3 /** 4 This program demonstrates the merge sort algorithm by 5 sorting an array that is filled with random numbers. 6 */ 7 public class MergeSortDemo 8 { 9 public static void main(String[] args) 10 { 11 int[] a = ArrayUtil.randomIntArray(20, 100); 12 System.out.println(Arrays.toString(a)); MergeSorter.sort(a); System.out.println(Arrays.toString(a)); 17 } 18 } 19
Typical Program Run:
[8, 81, 48, 53, 46, 70, 98, 42, 27, 76, 33, 24, 2, 76, 62, 89, 90, 5, 13, 21]
[2, 5, 8, 13, 21, 24, 27, 33, 42, 46, 48, 53, 62, 70, 76, 76, 81, 89, 90, 98]

31. Merge Sort Vs Selection Sort
Selection sort is an O(n2) algorithm.
Merge sort is an O(n log(n)) algorithm.
The n log(n) function grows much more slowly than n2.

32. Merge Sort Timing vs. Selection Sort

33. Sort Animator:
Comparison Sorting
Bubble Sort
Selection Sort
Insertion Sort
Shell Sort
Merge Sort
Quick Sort
Bucket Sort
Counting Sort
Radix Sort
Heap Sort
From:

34. Searching Linear search: also called sequential search
Examines all values in an array until it finds a match or reaches the end
Number of visits for a linear search of an array of n elements:
The average search visits n/2 elements
The maximum visits is n
A linear search locates a value in an array in O(n) steps

35. Programming Question
Implement the class LinearSearch. Implement the method search that performs linear search. The method should accept as parameters an int array and search value. The method should return the index of the value if found in array or -1 otherwise.
Then implement the tester class LinearSearchDemo that test sort method of LinearSearch class.
A sample output is shown:
Program Run:
[46, 99, 45, 57, 64, 95, 81, 69, 11, 97, 6, 85, 61, 88, 29, 65, 83, 88, 45, 88]
Enter number to search for, -1 to quit: 12
Found in position -1
Enter number to search for, -1 to quit: -1

36. Answer LinearSearch.java
1 /** 2 A class for executing linear searches in an array. 3 */ 4 public class LinearSearcher 5 { 6 /** 7 Finds a value in an array, using the linear search 8 algorithm. a the array to search value the value to find the index at which the value occurs, or if it does not occur in the array 13 */ 14 public static int search(int[] a, int value) 15 { 16 for (int i = 0; i < a.length; i++) 17 { 18 if (a[i] == value) { return i; } 19 } 20 return -1; 21 } 22 }

37. LinearSearchDemo.java
1 import java.util.Arrays; 2 import java.util.Scanner; 3 4 /** 5 This program demonstrates the linear search algorithm. 6 */ 7 public class LinearSearchDemo 8 { 9 public static void main(String[] args) 10 { 11 int[] a = ArrayUtil.randomIntArray(20, 100); 12 System.out.println(Arrays.toString(a)); 13 Scanner in = new Scanner(System.in); boolean done = false; 16 while (!done) 17 { 18 System.out.print("Enter number to search for, -1 to quit: "); 19 int n = in.nextInt(); 20 if (n == -1) 21 { 22 done = true; 23 } 24 else 25 { 26 int pos = LinearSearcher.search(a, n); 27 System.out.println("Found in position " + pos); 28 } 29 } 30 } 31 }

38. Question
If your array is sorted can you do a faster search than linear search?

39. Binary Search A binary search locates a value in a sorted array by:
Get middle element in array
If middle==searhckey return success
Else
Divide array into two halves.
Determining whether the value occurs in the first or second half
Then repeating the search in one of the halves
The size of the search is cut in half with each step.
Animator:

40. Binary Search Searching for 15 in this array
The last value in the first half is 9
So look in the second (darker colored) half
The last value of the first half of this sequence is 17
Look in the darker colored sequence

41. Binary Search
The last value of the first half of this very short sequence is 12 (<15),
This is smaller than the value that we are searching,
so we must look in the second half
15 ≠ 17: we don't have a match

42. Programming Question
Implement BinarySearcher class. The search method should implement binary search. The method should return the index of the value if found in array or -1 otherwise.
public static int search(int[] a, int low, int high, int value)
Original array
Index range to search
Search key

43. import java.util.Arrays;
import java.util.Random;
import java.util.Scanner;
public class BinarySearcher{
public static int search(int[] a, int low, int high, int value) {
//TODO : Implement recursive binary search }
public static int[] randomIntArray(int length, int n) {
Random generator = new Random();
int[] a = new int[length];
for (int i = 0; i < a.length; i++)
a[i] = generator.nextInt(n);
return a;
public static void main(String[] args) {
int[] a = randomIntArray(20, 100);
Arrays.sort(a);
System.out.println(Arrays.toString(a));
Scanner in = new Scanner(System.in);
System.out.print ("Enter number to search : ");
int n = in.nextInt();
int pos = BinarySearcher.search(a, 0, a.length - 1, n);
System.out.println("Found in position " + pos);

44. Answer
import java.util.Arrays; import java.util.Random; import java.util.Scanner; public class BinarySearcher{ public static int search(int[] a, int low, int high, int value) { if (low <= high) { int mid = (low + high) / 2; if (a[mid] == value) return mid; else if (a[mid] < value ) return search(a, mid + 1, high, value); else return search(a, low, mid - 1, value); } else { return -1; public static int[] randomIntArray(int length, int n) { Random generator = new Random(); int[] a = new int[length]; for (int i = 0; i < a.length; i++) a[i] = generator.nextInt(n); return a; //main: omitted

45. Binary Search Assume n is a power of 2, n = 2m where m = log2(n)
Then: T(n) = 1 + log2(n)
A binary search locates a value in a sorted array in O(log(n)) steps.

46. Question
Suppose you need to look through 1,000,000 records to find a telephone number. How many records do you expect to search (linear) before finding the number?

47. Answer
Answer: On average, you’d make 500,000 comparisons.

48. Question
Suppose you need to look through a sorted array with 1,000,000 elements to find a value. Using the binary search algorithm, how many records do you expect to search before finding the value?

49. Answer
Answer: You would search about 20. (The binary log of 1,024 is 10.)

50. Sorting and Searching in the Java Library - Sorting
You do not need to write sorting and searching algorithms
Use methods in the Arrays and Collections classes
The Arrays class contains static sort methods.
To sort an array of integers:
int[] a = ;
Arrays.sort(a);
That sort method uses the Quicksort
To sort an ArrayList, use Collections.sort
ArrayList<String> names = . . .;
Collections.sort(names);
Uses merge sort algorithm

51. Comparing Objects
Arrays.sort sorts objects of classes that implement Comparable interface:
public interface Comparable {
int compareTo(Object otherObject); }
The call a.compareTo(b) returns
A negative number if a should come before b
0 if a and b are the same
A positive number otherwise

52. Comparing Objects Some java library classes implement Comparable.
e.g. String and Date
You can implement Comparable interface for your own classes.
The Country class could implement Comparable:
public class Country implements Comparable {
public int compareTo(Object otherObject)
Country other = (Country) otherObject;
if (area < other.area) { return -1; }
else if (area == other.area) { return 0; }
else { return 1; } }

53. Comparing Objects You could pass an array of countries to Arrays.sort
Country[] countries = new Country[n];
Arrays.sort(countries); // Sorts by increasing area