Introduction to Programming (in C++) Sorting Jordi Cortadella, Ricard Gavaldà, Fernando Orejas Dept. of Computer Science, UPC.

Slides:



Advertisements
Similar presentations
Garfield AP Computer Science
Advertisements

SEARCHING AND SORTING HINT AT ASYMPTOTIC COMPLEXITY Lecture 9 CS2110 – Spring 2015 We may not cover all this material.
Introduction to Programming (in C++) Subprograms: procedures and functions Jordi Cortadella, Ricard Gavaldà, Fernando Orejas Dept. of Computer Science,
Sorting A fundamental operation in computer science (many programs need to sort as an intermediate step). Many sorting algorithms have been developed Choose.
Stephen P. Carl - CS 2421 Recursive Sorting Algorithms Reading: Chapter 5.
DIVIDE AND CONQUER APPROACH. General Method Works on the approach of dividing a given problem into smaller sub problems (ideally of same size).  Divide.
Computability Start complexity. Motivation by thinking about sorting. Homework: Finish examples.
Sorting Algorithms. Motivation Example: Phone Book Searching Example: Phone Book Searching If the phone book was in random order, we would probably never.
CS203 Programming with Data Structures Sorting California State University, Los Angeles.
CS 171: Introduction to Computer Science II Quicksort.
the fourth iteration of this loop is shown here
CSE 373: Data Structures and Algorithms
Lecture 25 Selection sort, reviewed Insertion sort, reviewed Merge sort Running time of merge sort, 2 ways to look at it Quicksort Course evaluations.
Simple Sorting Algorithms
CS 202, Spring 2003 Fundamental Structures of Computer Science II Bilkent University1 Sorting CS 202 – Fundamental Structures of Computer Science II Bilkent.
Sorting II/ Slide 1 Lecture 24 May 15, 2011 l merge-sorting l quick-sorting.
CS 202, Spring 2003 Fundamental Structures of Computer Science II Bilkent University1 Sorting - 3 CS 202 – Fundamental Structures of Computer Science II.
Introduction to Programming (in C++) Vectors Jordi Cortadella, Ricard Gavaldà, Fernando Orejas Dept. of Computer Science, UPC.
Simple Sorting Algorithms. 2 Bubble sort Compare each element (except the last one) with its neighbor to the right If they are out of order, swap them.
Advance Data Structure 1 College Of Mathematic & Computer Sciences 1 Computer Sciences Department م. م علي عبد الكريم حبيب.
Describing algorithms in pseudo code To describe algorithms we need a language which is: – less formal than programming languages (implementation details.
Introduction to Programming (in C++) Data and statements Jordi Cortadella, Ricard Gavaldà, Fernando Orejas Dept. of Computer Science, UPC.
Week 11 Introduction to Computer Science and Object-Oriented Programming COMP 111 George Basham.
Reasoning with invariants Jordi Cortadella Department of Computer Science.
1 Time Analysis Analyzing an algorithm = estimating the resources it requires. Time How long will it take to execute? Impossible to find exact value Depends.
Matrices Jordi Cortadella Department of Computer Science.
Prime numbers Jordi Cortadella Department of Computer Science.
Chapter 12 Recursion, Complexity, and Searching and Sorting
Computer Science Searching & Sorting.
Introduction to Programming (in C++) Algorithms on sequences. Reasoning about loops: Invariants. Jordi Cortadella, Ricard Gavaldà, Fernando Orejas Dept.
CompSci 100e 11.1 Sorting: From Theory to Practice l Why do we study sorting?  Because we have to  Because sorting is beautiful  Example of algorithm.
Sequences Jordi Cortadella Department of Computer Science.
C++ Programming: From Problem Analysis to Program Design, Second Edition Chapter 19: Searching and Sorting.
Search algorithms for vectors Jordi Cortadella Department of Computer Science.
Introduction to Programming (in C++) Loops Jordi Cortadella, Ricard Gavaldà, Fernando Orejas Dept. of Computer Science, UPC.
CS 61B Data Structures and Programming Methodology July 28, 2008 David Sun.
CSE 373: Data Structures and Algorithms Lecture 6: Sorting 1.
1 Joe Meehean.  Problem arrange comparable items in list into sorted order  Most sorting algorithms involve comparing item values  We assume items.
BUBBLE SORT. Introduction Bubble sort, also known as sinking sort, is a simple sorting algorithm that works by repeatedly stepping through the list to.
12. Sorting Intro Programming in C++ Computer Science Dept Va Tech August, 2002 © Barnette ND & McQuain WD 1 Sorting Many computer applications.
Introduction to Programming (in C++) Numerical algorithms Jordi Cortadella, Ricard Gavaldà, Fernando Orejas Dept. of Computer Science, UPC.
Sorting. Sorting Terminology Sort Key –each element to be sorted must be associated with a sort key which can be compared with other keys e.g. for any.
3 – SIMPLE SORTING ALGORITHMS
1 Searching and Sorting Searching algorithms with simple arrays Sorting algorithms with simple arrays –Selection Sort –Insertion Sort –Bubble Sort –Quick.
Introduction to Programming (in C++) Multi-dimensional vectors Jordi Cortadella, Ricard Gavaldà, Fernando Orejas Dept. of Computer Science, UPC.
Chapter 9 Sorting. The efficiency of data handling can often be increased if the data are sorted according to some criteria of order. The first step is.
Liang, Introduction to Java Programming, Sixth Edition, (c) 2007 Pearson Education, Inc. All rights reserved Chapter 23 Algorithm Efficiency.
HEAPS. Review: what are the requirements of the abstract data type: priority queue? Quick removal of item with highest priority (highest or lowest key.
Searching and Sorting Searching: Sequential, Binary Sorting: Selection, Insertion, Shell.
ICS201 Lecture 21 : Sorting King Fahd University of Petroleum & Minerals College of Computer Science & Engineering Information & Computer Science Department.
C++ How to Program, 7/e © by Pearson Education, Inc. All Rights Reserved.
Vectors Jordi Cortadella Department of Computer Science.
Computer Science 1620 Sorting. cases exist where we would like our data to be in ascending (descending order) binary searching printing purposes selection.
0 Introduction to asymptotic complexity Search algorithms You are responsible for: Weiss, chapter 5, as follows: 5.1 What is algorithmic analysis? 5.2.
Sorting – Lecture 3 More about Merge Sort, Quick Sort.
12. Searching/Sorting Programming in C++ Computer Science Dept Va Tech August, 2000 © Barnette ND, McQuain WD, Keenan MA 1 Simple Searching Many.
Sorting and Runtime Complexity CS255. Sorting Different ways to sort: –Bubble –Exchange –Insertion –Merge –Quick –more…
Sort Algorithm.
The Bubble Sort Mr. Dave Clausen La Cañada High School
Reasoning with invariants
Jordi Cortadella and Jordi Petit Department of Computer Science
Jordi Cortadella Department of Computer Science
Objectives At the end of the class, students are expected to be able to do the following: Understand the purpose of sorting technique as operations on.
Selection Sort Sorted Unsorted Swap
Quicksort analysis Bubble sort
Data Structures Review Session
IT 4043 Data Structures and Algorithms
Jordi Cortadella and Jordi Petit Department of Computer Science
Jordi Cortadella Department of Computer Science
Searching and Sorting Hint at Asymptotic Complexity
Presentation transcript:

Introduction to Programming (in C++) Sorting Jordi Cortadella, Ricard Gavaldà, Fernando Orejas Dept. of Computer Science, UPC

Sorting Let elem be a type with a  operation, which is a total order A vector v is (increasingly) sorted if for all i with 0  i  v.size()-1, v[i]  v[i+1] Equivalently: if i  j then v[i]  v[j] A fundamental, very common problem: sort v Order the elements in v and leave the result in v Introduction to Programming© Dept. CS, UPC2

Sorting Introduction to Programming© Dept. CS, UPC Another common task: sort v[a..b] a b a b

Sorting We will look at four sorting algorithms: – Selection Sort – Insertion Sort – Bubble Sort – Merge Sort Let us consider a vector v of n elems (n = v.size()) – Insertion, Selection and Bubble Sort make a number of operations on elems proportional to n 2 – Merge Sort is proportional to n·log 2 n: faster except for very small vectors Introduction to Programming© Dept. CS, UPC4

Selection Sort Observation: in the sorted vector, v[0] is the smallest element in v The second smallest element in v must go to v[1]… … and so on At the i-th iteration, select the i-th smallest element and place it in v[i] Introduction to Programming© Dept. CS, UPC5

Selection Sort Introduction to Programming© Dept. CS, UPC6 From

Selection Sort Selection sort keeps this invariant: Introduction to Programming© Dept. CS, UPC ?????? ii-1 this is sorted and contains the i-1 smallest elements this may not be sorted… but all elements here are larger than or equal to the elements in the sorted part

Selection Sort // Pre: -- // Post: v is now increasingly sorted void selection_sort(vector & v) { int last = v.size() - 1; for (int i = 0; i < last; ++i) { int k = pos_min(v, i, last); swap(v[k], v[i]); } } Introduction to Programming© Dept. CS, UPC8 // Invariant: v[0..i-1] is sorted and // if a < i <= b then v[a] <= v[b] Note: when i=v.size()-1, v[i] is necessarily the largest element. Nothing to do.

Selection Sort Introduction to Programming© Dept. CS, UPC9 // Pre: 0 <= left <= right < v.size() // Returns pos such that left <= pos <= right // and v[pos] is smallest in v[left..right] int pos_min(const vector & v, int left, int right) { int pos = left; for (int i = left + 1; i <= right; ++i) { if (v[i] < v[pos]) pos = i; } return pos; }

Selection Sort At the i-th iteration, Selection Sort makes – up to v.size()-1-i comparisons among elems – 1 swap (=3 elem assignments) per iteration The total number of comparisons for a vector of size n is: (n-1)+(n-2)+…+1= n(n-1)/2 ≈ n 2 /2 The total number of assignments is 3(n-1). Introduction to Programming© Dept. CS, UPC10

Insertion Sort Let us use induction: – If we know how to sort arrays of size n-1, – do we know how to sort arrays of size n? Introduction to Programming© Dept. CS, UPC n-1n

Insertion Sort Insert x=v[n-1] in the right place in v[0..n-1] Two ways: -Find the right place, then shift the elements -Shift the elements to the right until one ≤ x is found Introduction to Programming© Dept. CS, UPC12

Insertion Sort Introduction to Programming© Dept. CS, UPC13 This is sorted This may not be sorted and we have no idea of what may be here Insertion sort keeps this invariant: ?????? ii-1

Insertion Sort Introduction to Programming© Dept. CS, UPC14 From

Insertion Sort // Pre: -- // Post: v is now increasingly sorted void insertion_sort(vector & v) { for (int i = 1; i 0 and v[j - 1] > x) { v[j] = v[j - 1]; --j; } v[j] = x; } } Introduction to Programming© Dept. CS, UPC15 // Invariant: v[0..i-1] is sorted in ascending order

Insertion Sort At the i-th iteration, Insertion Sort makes up to i comparisons and up to i+2 assignments of type elem The total number of comparisons for a vector of size n is, at most: … + (n-1) = n(n-1)/2 ≈ n 2 /2 At the most, n 2 /2 assignments But about n 2 /4 in typical cases Introduction to Programming© Dept. CS, UPC16

Selection Sort vs. Insertion Sort Introduction to Programming© Dept. CS, UPC

Selection Sort vs. Insertion Sort Introduction to Programming© Dept. CS, UPC18

Evaluation of complex conditions void insertion_sort(vector & v) { for (int i = 1; i 0 and v[j - 1] > x) { v[j] = v[j - 1]; --j; } v[j] = x; } } How about: while (v[j – 1] > x and j > 0) ? Consider the case for j = 0  evaluation of v[-1] (error !) How are complex conditions really evaluated? Introduction to Programming© Dept. CS, UPC19

Evaluation of complex conditions Many languages (C, C++, Java, PHP, Python) use the short-circuit evaluation (also called minimal or lazy evaluation) for Boolean operators. For the evaluation of the Boolean expression expr1 op expr2 expr2 is only evaluated if expr1 does not suffice to determine the value of the expression. Example: (j > 0 and v[j-1] > x) v[j-1] is only evaluated when j>0 Introduction to Programming© Dept. CS, UPC20

Evaluation of complex conditions In the following examples: n != 0 and sum/n > avg n == 0 or sum/n > avg sum/n will never execute a division by zero. Not all languages have short-circuit evaluation. Some of them have eager evaluation (all the operands are evaluated) and some of them have both. The previous examples could potentially generate a runtime error (division by zero) when eager evaluation is used. Tip: short-circuit evaluation helps us to write more efficient programs, but cannot be used in all programming languages. Introduction to Programming© Dept. CS, UPC21

Bubble Sort A simple idea: traverse the vector many times, swapping adjacent elements when they are in the wrong order. The algorithm terminates when no changes occur in one of the traversals. Introduction to Programming© Dept. CS, UPC22

Bubble Sort Introduction to Programming© Dept. CS, UPC The largest element is well-positioned after the first iteration. The second largest element is well-positioned after the second iteration. The vector is sorted when no changes occur during one of the iterations.

Bubble Sort Introduction to Programming© Dept. CS, UPC24 From

Bubble Sort void bubble_sort(vector & v) { bool sorted = false; int last = v.size() – 1; while (not sorted) { // Stop when no changes sorted = true; for (int i = 0; i v[i + 1]) { swap(v[i], v[i + 1]); sorted = false; } } // The largest element falls to the bottom --last; } Introduction to Programming© Dept. CS, UPC25 Observation: at each pass of the algorithm, all elements after the last swap are sorted.

Bubble Sort void bubble_sort(vector & v) { int last = v.size() – 1; while (last > 0) { int last_swap = 0; // Last swap at each iteration for (int i = 0; i v[i + 1]) { swap(v[i], v[i + 1]); last_swap = i; } } last = last_swap; // Skip the sorted tail } Introduction to Programming© Dept. CS, UPC26

Bubble Sort Worst-case analysis: – The first pass makes n-1 swaps – The second pass makes n-2 swaps – … – The last pass makes 1 swap The worst number of swaps: … + (n-1) = n(n-1)/2 ≈ n 2 /2 It may be efficient for nearly-sorted vectors. In general, bubble sort is one of the least efficient algorithms. It is not practical when the vector is large. Introduction to Programming© Dept. CS, UPC27

Merge Sort Recall our induction for Insertion Sort: – suppose we can sort vectors of size n-1, – can we now sort vectors of size n? What about the following: – suppose we can sort vectors of size n/2, – can we now sort vectors of size n? Introduction to Programming© Dept. CS, UPC28

Merge Sort Introduction to Programming© Dept. CS, UPC Induction! Induction! How do we do this?

Merge Sort Introduction to Programming© Dept. CS, UPC30 From

Merge Sort We have seen almost what we need! // Pre: A and B are sorted in ascending order // Returns the sorted fusion of A and B vector merge(const vector & A, const vector & B); Now, v[0..n/2-1] and v[n/2..n-1] are sorted in ascending order. Merge them into an auxiliary vector of size n, then copy back to v. Introduction to Programming© Dept. CS, UPC31

Merge Sort Introduction to Programming© Dept. CS, UPC Split Merge Merge Sort

// Pre: 0 <= left <= right < v.size() // Post: v[left..right] has been sorted increasingly void merge_sort(vector & v, int left, int right) { if (left < right) { int m = (left + right)/2; merge_sort(v, left, m); merge_sort(v, m + 1, right); merge(v, left, m, right); } Introduction to Programming© Dept. CS, UPC33

Merge Sort – merge procedure // Pre: 0 & v, int left, int mid, int right) { int n = right - left + 1; vector aux(n); int i = left; int j = mid + 1; int k = 0; while (i <= mid and j <= right) { if (v[i] <= v[j]) { aux[k] = v[i]; ++i; } else { aux[k] = v[j]; ++j; } ++k; } while (i <= mid) { aux[k] = v[i]; ++k; ++i; } while (j <= right) { aux[k] = v[j]; ++k; ++j; } for (k = 0; k < n; ++k) v[left+k] = aux[k]; } Introduction to Programming© Dept. CS, UPC34

Merge Sort Introduction to Programming© Dept. CS, UPC : merge_sort : merge

Merge Sort Introduction to Programming© Dept. CS, UPC36

Comparison of sorting algorithms Introduction to Programming© Dept. CS, UPC37 Selection Insertion Bubble Merge

Comparison of sorting algorithms Approximate number of comparisons: Note: it is known that every general sorting algorithm must do at least n·log 2 n comparisons. Introduction to Programming© Dept. CS, UPC38 n = v.size()101001,00010,000100,000 Insertion, Selection and Bubble Sort (  n 2 /2) 505,000500,00050,000,0005,000,000,000 Merge Sort (  n·log 2 n) 671,35020,000266,0003,322,000

Comparison of sorting algorithms Introduction to Programming© Dept. CS, UPC39 Vector size Execution time (µs) For small vectors

Comparison of sorting algorithms Introduction to Programming© Dept. CS, UPC40 Vector size Execution time (ms) For medium vectors

Comparison of sorting algorithms Introduction to Programming© Dept. CS, UPC41 Vector size Execution time (secs) For large vectors

Other sorting algorithms There are many other sorting algorithms. The most efficient algorithm for general sorting is quick sort (C.A.R. Hoare). – The worst case is proportional to n 2 – The average case is proportional to n·log 2 n, but it usually runs faster than all the other algorithms – It does not use any auxiliary vectors Quick sort will not be covered in this course. Introduction to Programming© Dept. CS, UPC42

Sorting with the C++ library A sorting procedure is available in the C++ library It probably uses a quicksort algorithm To use it, include: #include To increasingly sort a vector v (of int’s, double’s, string’s, etc.), call: sort(v.begin(), v.end()); Introduction to Programming© Dept. CS, UPC43

Sorting with the C++ library To sort with a different comparison criteria, call sort(v.begin(), v.end(), comp); For example, to sort int’s decreasingly, define: bool comp(int a, int b) { return a > b; } To sort people by age, then by name: bool comp(const Person& a, const Person& b) { if (a.age == b.age) return a.name < b.name; else return a.age < b.age; } Introduction to Programming© Dept. CS, UPC44

Sorting is not always a good idea… Example: to find the min value of a vector min = v[0]; for (int i=1; i < v.size(); ++i) if (v[i] < min) min = v[i]; sort(v); min = v[0]; Efficiency analysis: – Option (1): n iterations (visit all elements). – Option (2): 2n∙log 2 n moves with a good sorting algorithm (e.g., merge sort) Introduction to Programming© Dept. CS, UPC45 (1) (2)