ICS 220 – Data Structures and Algorithms

Slides:



Advertisements
Similar presentations
Introduction to Algorithms Quicksort
Advertisements

David Luebke 1 4/22/2015 CS 332: Algorithms Quicksort.
Algorithms Analysis Lecture 6 Quicksort. Quick Sort Divide and Conquer.
Sorting Comparison-based algorithm review –You should know most of the algorithms –We will concentrate on their analyses –Special emphasis: Heapsort Lower.
Sorting A fundamental operation in computer science (many programs need to sort as an intermediate step). Many sorting algorithms have been developed Choose.
Quick Sort, Shell Sort, Counting Sort, Radix Sort AND Bucket Sort
Liang, Introduction to Java Programming, Eighth Edition, (c) 2011 Pearson Education, Inc. All rights reserved Chapter 24 Sorting.
Dr. Ahmad R. Hadaegh A.R. Hadaegh California State University San Marcos (CSUSM) Page 1 Sorting.
Data Structures Data Structures Topic #13. Today’s Agenda Sorting Algorithms: Recursive –mergesort –quicksort As we learn about each sorting algorithm,
Sorting Algorithms and Average Case Time Complexity
Chapter 19: Searching and Sorting Algorithms
1 Sorting Problem: Given a sequence of elements, find a permutation such that the resulting sequence is sorted in some order. We have already seen: –Insertion.
Section 8.8 Heapsort.  Merge sort time is O(n log n) but still requires, temporarily, n extra storage locations  Heapsort does not require any additional.
Sorting21 Recursive sorting algorithms Oh no, not again!
Heapsort. 2 Why study Heapsort? It is a well-known, traditional sorting algorithm you will be expected to know Heapsort is always O(n log n) Quicksort.
Version TCSS 342, Winter 2006 Lecture Notes Priority Queues Heaps.
TDDB56 DALGOPT-D DALG-C Lecture 8 – Sorting (part I) Jan Maluszynski - HT Sorting: –Intro: aspects of sorting, different strategies –Insertion.
TTIT33 Algorithms and Optimization – Dalg Lecture 2 HT TTIT33 Algorithms and optimization Lecture 2 Algorithms Sorting [GT] 3.1.2, 11 [LD] ,
Sorting CS-212 Dick Steflik. Exchange Sorting Method : make n-1 passes across the data, on each pass compare adjacent items, swapping as necessary (n-1.
Insertion Sort & Shellsort By: Andy Le CS146 – Dr. Sin Min Lee Spring 2004.
1 Data Structures and Algorithms Sorting. 2  Sorting is the process of arranging a list of items into a particular order  There must be some value on.
ICS 220 – Data Structures and Algorithms Week 7 Dr. Ken Cosh.
1 Time Analysis Analyzing an algorithm = estimating the resources it requires. Time How long will it take to execute? Impossible to find exact value Depends.
Sorting HKOI Training Team (Advanced)
Computer Science 101 Fast Searching and Sorting. Improving Efficiency We got a better best case by tweaking the selection sort and the bubble sort We.
C++ Programming: Program Design Including Data Structures, Fourth Edition Chapter 19: Searching and Sorting Algorithms.
Merge Sort. What Is Sorting? To arrange a collection of items in some specified order. Numerical order Lexicographical order Input: sequence of numbers.
Heapsort CSC Why study Heapsort? It is a well-known, traditional sorting algorithm you will be expected to know Heapsort is always O(n log n)
Elementary Sorting Algorithms Many of the slides are from Prof. Plaisted’s resources at University of North Carolina at Chapel Hill.
The Selection Problem. 2 Median and Order Statistics In this section, we will study algorithms for finding the i th smallest element in a set of n elements.
1 Joe Meehean.  Problem arrange comparable items in list into sorted order  Most sorting algorithms involve comparing item values  We assume items.
Heapsort. Heapsort is a comparison-based sorting algorithm, and is part of the selection sort family. Although somewhat slower in practice on most machines.
Sorting. Pseudocode of Insertion Sort Insertion Sort To sort array A[0..n-1], sort A[0..n-2] recursively and then insert A[n-1] in its proper place among.
Data Structures Using C++ 2E Chapter 10 Sorting Algorithms.
Chapter 18: Searching and Sorting Algorithms. Objectives In this chapter, you will: Learn the various search algorithms Implement sequential and binary.
Priority Queues and Heaps. October 2004John Edgar2  A queue should implement at least the first two of these operations:  insert – insert item at the.
Chapter 8 Sorting and Searching Goals: 1.Java implementation of sorting algorithms 2.Selection and Insertion Sorts 3.Recursive Sorts: Mergesort and Quicksort.
Review 1 Selection Sort Selection Sort Algorithm Time Complexity Best case Average case Worst case Examples.
Sorting – Part II CS 367 – Introduction to Data Structures.
Chapter 9 Sorting 1. The efficiency of data handling can often be increased if the data are sorted according to some criteria of order. The first step.
Chapter 9 Sorting. The efficiency of data handling can often be increased if the data are sorted according to some criteria of order. The first step is.
Data Structures - CSCI 102 Selection Sort Keep the list separated into sorted and unsorted sections Start by finding the minimum & put it at the front.
FALL 2005CENG 213 Data Structures1 Priority Queues (Heaps) Reference: Chapter 7.
Searching and Sorting Searching: Sequential, Binary Sorting: Selection, Insertion, Shell.
AVL Trees and Heaps. AVL Trees So far balancing the tree was done globally Basically every node was involved in the balance operation Tree balancing can.
ICS201 Lecture 21 : Sorting King Fahd University of Petroleum & Minerals College of Computer Science & Engineering Information & Computer Science Department.
Sorting Fundamental Data Structures and Algorithms Aleks Nanevski February 17, 2004.
Chapter 9 sorting. Insertion Sort I The list is assumed to be broken into a sorted portion and an unsorted portion The list is assumed to be broken into.
CSC317 1 So far so good, but can we do better? Yes, cheaper by halves... orkbook/cheaperbyhalf.html.
Chapter 4, Part II Sorting Algorithms. 2 Heap Details A heap is a tree structure where for each subtree the value stored at the root is larger than all.
PREVIOUS SORTING ALGORITHMS  BUBBLE SORT –Time Complexity: O(n 2 ) For each item, make (n –1) comparisons Gives: Comparisons = (n –1) + (n – 2)
Week 13 - Wednesday.  What did we talk about last time?  NP-completeness.
Sorting Ordering data. Design and Analysis of Sorting Assumptions –sorting will be internal (in memory) –sorting will be done on an array of elements.
Data Structures and Algorithms Instructor: Tesfaye Guta [M.Sc.] Haramaya University.
CSE 250 – Data Structures. Today’s Goals  First review the easy, simple sorting algorithms  Compare while inserting value into place in the vector 
Sorting and Runtime Complexity CS255. Sorting Different ways to sort: –Bubble –Exchange –Insertion –Merge –Quick –more…
Liang, Introduction to Java Programming, Tenth Edition, (c) 2013 Pearson Education, Inc. All rights reserved. 1 Chapter 23 Sorting.
Dr. Bernard Chen Ph.D. University of Central Arkansas Fall 2008
Sorting by Tammy Bailey
Description Given a linear collection of items x1, x2, x3,….,xn
Bubble Sort Bubble sort is one way to sort an array of numbers. Adjacent values are swapped until the array is completely sorted. This algorithm gets its.
Binary Search Back in the days when phone numbers weren’t stored in cell phones, you might have actually had to look them up in a phonebook. How did you.
CSS 342 Data Structures, Algorithms, and Discrete Mathematics I
8/04/2009 Many thanks to David Sun for some of the included slides!
Sub-Quadratic Sorting Algorithms
Simple Sorting Methods: Bubble, Selection, Insertion, Shell
Analysis of Algorithms
CSE 373 Data Structures and Algorithms
CS 615: Design & Analysis of Algorithms
Algorithm Course Algorithms Lecture 3 Sorting Algorithm-1
Presentation transcript:

ICS 220 – Data Structures and Algorithms Dr. Ken Cosh Lecture 9

Review Graphs Types Representations Traversal Shortest Path Dijkstra’s Algorithm Minimum Spanning Trees Networks Eulerian Graphs Hamiltonian Graphs The Travelling Salesman

This week Sorting Algorithms The efficiency of many algorithms depends on the way data is stored. Most data structures can be improved by ordering the data structure in some way. This week we will investigate different approaches to sorting data.

Sorting Criteria The first step is to decide what to sort by; numerical alphabetical ASCII codes

Algorithm Measures Best Case Worst Case Average Case Often when data already sorted Worst Case Data completely disorganised, or in reverse order. Average Case Random order Some sorting algorithms are the same for all three cases, others can be tailored to suit certain cases.

Sorting Algorithms Sorting Algorithms are generally comprised of ‘comparisons’ and ‘movements’. If the data being sorted is complicated (strings for example), then the comparison part can be more expensive. If the data items are large, then the movement part can be more expensive.

Elementary Sorting Algorithms First lets investigate 3 elementary sorting algorithms; Insertion Sort Selection Sort Bubble Sort

Insertion Sort Insertion sort starts with the first two elements, and compares them; a[0] < a[1] ? If necessary it will swap them. Next the 3rd element is compared and positioned, and then the 4th etc.

Insertion Sort template<class T> void insertionsort(T data[], int n) { for (int i = 1, j; i < n; i++) { T tmp = data[i]; for (j=i; j<0 && tmp < data[j-1]; j--) data[j] = data[j-1]; data[j] = tmp; }

Insertion Sort The insertion method only sorts the part of the array that needs to be sorted – if the array is already sorted, less processing is required. Best Case when data is already in order, 1 comparison per position (n-1), or O(n). Worst Case is when data is in reverse order and ever, in which case comparisons are required for both the outer and the inner loop, O(n2) How about Average Case? The result should be somewhere between O(n) and O(n2), but which is it closer to?

Insertion Sort – Average Case The number of iterations of the outer loop depends on how far the number is away from where it should be; it could loop 0 times if it is where it should be, or i times if it needs to be inserted at the start. Therefore on average it should be; 0+1+2+…+i / i = (i+1)/2 This is the average per case, which needs to be multiplied by the number of numbers (n), leaving O(n2), sadly closer to the worst case than best.

Selection Sort Selection Sort takes each value and moves it to its final place. i.e. it first searches for the smallest value and moves it to the start, then the next smallest and moves that to position 2.

Selection Sort template<class T> void selectionsort(T data[], int n) { for (int i = 0,j,least; i<n-1; i++) { for (j = i+1, least = i; j < n; j++) if (data[j] < data[least]) least = j; swap(data[least], data[i]); } Once again this algorithm relies on a double loop, so O(n2)

BubbleSort Imagine a glass of beer… The bubbles slowly rise to the surface. Now imagine those bubbles are the data to be sorted, with the lowest values rising to the surface. That’s how Bubblesort works!

BubbleSort Starting with the other end of the dataset, a[n-1], each value is compared with its predecessor, a[n-2], and if it is less it is swapped. Next a[n-2] is compared with a[n-3], and if necessary swapped, and so on. Therefore at the end of the first loop through the array, the smallest value is in position a[0], but many of the other values have been bubbled towards their final location. The next loop through will move values closer still.

BubbleSort template<class T> void bubblesort(T data[], int n) { for (int i=0; i<n-1; i++) for (int j=n-1; j>I; --j) if (data[j] < data[j-1]) swap(data[j], data[j-1]); }

BubbleSort With Bubble Sort there are the same number of comparisons in every case (best, worst and average) Sadly that number of comparisons is; (n(n-1))/2, or O(n2)

Cocktail Sort A variation of the BubbleSort is called Cocktail Sort. Here the bubbles move in both directions alternatingly. As though a cocktail was being shaken. First lower values are bubbled towards the start, then higher values are bubbled towards the end.

Elementary Sorting Algorithms Sorry… but O(n2) just isn’t good enough anymore!

Efficient Sorting Algorithms Shell Sort Heap Sort Quick Sort Merge Sort Radix Sort

ShellSort One problem with the basic sorting algorithms is that their runtime grows faster than the size of the array. Therefore one way to solve this problem is to perform the sort on smaller / limited subarray parts of the array. ShellSort (named after Donald Shell), breaks the array into subarrays, sorts them, and then sorts the whole array.

ShellSort Shellsort is actually slightly more complex than that, because decisions need to be made about the size of each subarray. Normally the Sort is performed on a decreasing number of sub arrays, with the final sort being performed on the whole array (the sub arrays growing in size each Sort). Note that the actual Sort part can use any of the Elementary Sorting Algorithms already discussed.

Shell Sort Sample Original Data Set 5 subarray Sort 10, 15, 2, 9, 12, 4, 19, 5, 8, 0, 11 5 subarray Sort 10, 4, 11  4,10,11 15, 19  15, 19 2, 5  2, 5 9, 8  8, 9 12, 0  0, 12 After 5 subarray Sort 4, 15, 2, 8, 0, 10, 19, 5, 9, 12, 11

Shell Sort Sample cont. 3 subarray Sort 4, 15, 2, 8, 0, 10, 19, 5, 9, 12, 11 3 subarray Sort 4, 8, 19, 12  4, 8, 12, 19 15, 0, 5, 11  0, 5, 11, 15 2, 10, 9  2, 9, 10 After 3 subarray Sort 4, 0, 2, 8, 5, 9, 12, 11, 10, 19, 15 Final Sort 0, 2, 4, 5, 8, 9, 10, 11, 12, 15, 19

Shell Sort In the Sample, I arbitrarily chose sub array sizes of 1, 3 and 5. In reality that increment isn’t ideal, but then it isn’t clear what increment is. From empirical studies, a suggested increment follows this pattern; h1 = 1 hi+1 = 3hi+1 until ht+2 ≥n What sequence will this give?

Shell Sort Shell Sort is shown to be better than O(n2), but how much depends on the increments. With Shell Sort different Elementary Sorting Algorithms can affect the efficiency of the ShellSort, but its not agreed on how. A favourite is surely the “Cocktail Shaker Sort”, which uses iterations of the bubble sort, finishing off with a insertion sort – as the cherry is inserted into the cocktail!

Heap Sort To investigate how Heap Sort works, we need to investigate a Heap further. A heap is like a binary tree, with 2 different properties; First the value of each node is not less than the values in its children (i.e. the highest value is the root). Second the tree is perfectly balanced.

Heap Sort Heap Sort makes use of a heap, and is a variation of Selection Sort. The weakness of selection Sort was the number of comparisons required – because of the improved searchability of a tree, heap sort reduces the comparisons made during selection sort. First a heap is created, then the root is moved to the end of the dataset, before a new heap is created. Therefore the complexity of the algorithm mirrors the complexity of creating a heap.

Heap Sample Remove root node, and make new heap from 2 heaps 15 12 6 11 10 2 3 1 8

Heap Sort Rather than moving the root to a new array, heapsort uses the same memory space and swaps the root value with the last leaf node (8). 8 is then be moved down the tree, swapping with the larger child until it reaches its new base (i.e. swap with 12 and then 11). The new root can then be swapped with the next lowest node (1). And the process repeated, which in the worst case is O(n lg n).

QuickSort Quicksort uses a recursive approach, which is similar to the approach we used when guessing a number between 1 and 10. Quicksort shares with Shellsort the ideal of breaking the problem down into subproblems that are easier to solve. Quicksort divides the values into 2 subarrays – those lower than value ‘x’ and those higher than value ‘x’. The 2 subarrays can then be sorted – guess what… using QuickSort.

QuickSort The problem remains which value to chose as the pivot or bound value? Obviously if we choose a value too near to either end of the dataset we will have an unbalanced split. We could arbitrarily choose the first value, but some arrays are already partly sorted, therefore we could arbitrarily choose the middle value.

Quick Sort {10, 15, 2, 9, 12, 4, 19, 5, 8, 0, 11} {2, 0} 4 {10, 15, 9, 12, 19, 5, 8, 11} {0, 2} 4 {10, 9, 5, 8, 11} 12 {15, 19} 0, 2, 4, {}5{10, 9, 8, 11} 12 {15,19} 0, 2, 4, 5 {8}9{10, 11} 12, 15, 19 1, 2, 4, 5, 8, 9, 10, 11, 12, 15, 19

QuickSort Pseudocode Quicksort(array[]) if length(array)>1 choose bound; while there are elements in array include element in either subarray1 or subarray2; quicksort(subarray1); quicksort(subarray2);

QuickSort Best Case – when the array divides into even size parts; n + 2n/2 + 4n/4 + 8n/8 + … …+nn/n = O(n lg n) Worst Case – when one array is always size 1; O(n2) Average case is in between. Ideally if we can avoid the worst case, but to do that we need to pick a good pivot value; A better approach is to choose the median of the first, last and middle values

MergeSort The problem with QuickSort is still picking a good pivot value. MergeSort combats this by concentrating on merging two already sorted subarrays. The merge process can be quite straight forward assuming the two subarrays are already sorted. We can sort the subarrays recursively using MergeSort!

Merge Sort Mergesort(data) If data has at least 2 elements Mergesort(left half); Mergesort(right half); merge(both halves);

merge merge(array1, array2, array3) int i1 = 0, i2 = 0, i3 = 0; while both array2 and array 3 have elements if array2[i2] < array3[i3] array1[i1++] = array2[i2++]; else array1[i1++] = array3[i3++]; add the remains of either array2 or array3 into array1;

MergeSort {10, 15, 2, 9, 12, 4, 19, 5, 8, 0, 11} {10, 15, 2, 9, 12, 4} {19, 5, 8, 0, 11} {10, 15, 2}{9, 12, 4} {19, 5, 8}{0, 11} {10,15}{2}{9,12}{4} {19,5}{8}{0,11} {2, 10, 15}{4, 9, 12} {5, 8, 19}{0, 11} {2, 4, 9, 10, 12, 15} {0, 5, 8, 11, 19} {0, 2, 4, 5, 8, 9, 10, 11, 12, 15, 19}

MergeSort The MergeSort algorithm is O(n lg n), but has a major drawback due to the space required to store multiple arrays (in the run time stack). An iterative version while visually more complex is a little more efficient space wise. Also using a linked list rather than an array can counter the space issue.

Radix Sorting Radix Sorting mirrors a real life approach to sorting. Given a selection of books, we might first make a pile of all the ‘A’ author books, all the ‘B’ author books etc. Then we would sort each pile. Or, if given a series of 10 digit numbers, we might first sort according to the first digit, and then according to the second digit etc.

Comparing approaches Take a look at the table on page 511 to see how the different sorting algorithms compare.