Discrete Structures Lecture 11: Algorithms Miss, Yanyan,Ji United International College Thanks to Professor Michael Hvidsten

Objective: Construct and analysis algorithm Outline:  Algorithm definition and properties  Using Pseudocode describe sorting and search algorithm  Algorithm analysis: Big oh, Big Theta, Big Omega Objective and Outline

Algorithms Definition: An algorithm is a finite set of precise instructions for performing a computation or for solving a problem. Example: Find the largest (maximum) number in a list. Algorithm: (in Pseudocde) procedure max(a 1, a 2, a 3, …, a n : integers) max = a 1 for i = 2 to n if max < a i then max = a i return max

Algorithm Properties All algorithms share these properties: – Input: The set of input value(s) for the algorithm – Output: Value(s) that form the solution to the problem – Well-Defined: The steps of the algorithm must be precise – Correctness: An algorithm should produce the correct input for a given set of input values – Finiteness: An algorithm should produce an output after a finite number of steps – Effectiveness and Generality: The algorithm should be applicable for all possible input values.

Algorithms Example: Find the maximum number in a list. procedure max(a 1, a 2, a 3, …, a n : integers) max = a 1 for i = 2 to n if max < a i then max = a i return max Does this algorithm have all of the properties listed above?

Solution Input :a sequence of integers Output: the largest integer in the sequence Definiteness: only assignments, a finite loop, and condition statements occur. Correctness: initial value of max is the first term of the sequence, as successive terms of the sequence are examined. max is updated to the value of a term if the term exceeds the maximum of the terms previously examined. Finiteness: it terminates after all the integers in the sequence have been examined. Effectiveness: the algorithm can be carried out in a finite amount of time since each step is either a comparison or an assignment. Generality: it can be used to find the maximum of any finite sequence of integers.

Algorithm Properties Note: An algorithm can be considered as a function that takes an input and returns an output. Pseudocode: Provides an intermediate step between an English language description and a implementation in a programming language. We want to use instructions resembling those of a programming language. We don’t use any particular language (like C or Java) to describe the steps in an algorithm, since we want to be able to implement the algorithm in every language.

Search Algorithms General Problem: Locate an element x in a list of elements a 1, a 2, a 3, …, a n. Example: Locate a word in a dictionary. Linear Search Algorithm: Compare x with a 1. If x = a 1, return 1. Otherwise, compare x with a 2. If x = a 2, return 2. Continue in this process until we either find x, then the solution will be the location of x or we exhaust the list (in which case we return 0).

Linear Search Linear Search Algorithm: (Pseudocode) procedure linear search (x: integer, a 1,a 2,a 3,…, a n : distinct integers) i = 1 while (i ≤ n and x ≠ a i ) i = i+1 if i ≤ n then return i else return 0

Binary Search Binary Search Algorithm: If the terms in the list are in order of increasing size, a 1 ≤ a 2 ≤ a 3 ≤ … ≤ a n we can use the following procedure: Compare x to the middle term of the list a m where m = ⌊ (n+1)/2 ⌋. If x > a m, continue searching in the second half of the list. Otherwise, continue searching in the first half of the list. In either case, we now have a new list to search of half the size as the original. Use the same procedure on this new list, on so on.

Binary Search Binary Search Algorithm: i = left endpoint of search interval j = right endpoint of search interval Start: i = 1, j = n, x = (term to find) While i<j do the following: Compute middle index of list, m = ⌊ (i+j)/2 ⌋ If x > a m, search in second half of list (a m+1, a m+2, … a j ) and set i = m+1 Else search in first half of list (a i, a i+1, … a m ) and set j = m

Binary Search Example: Find index of 13 in (1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16) Start: i=1, j=12, x = 13 (1) m = ⌊ (i+j)/2 =13/2 ⌋ = 6 and a m = 7. Since x > 7, we set i = m+1 = 7 and continue searching in ( ). [i=7, j=12] (2) m = ⌊ (i+j)/2 ⌋ = 9 and a m = 12. Since x > 12, we set i = m+1 = 10 and continue searching in ( ). [i=10, j=12] (3)m = ⌊ 22/2 ⌋ = 11 and a m = 15. Since x < 15, set j= m = 11 and continue searching in (13 15). [i=10, j=11] (4)m = ⌊ 21/2 ⌋ =10 and a m = 13. Since x = 13, set j=m=10. We are done, since i=j=10. Return 10.

Binary Search Binary Search Algorithm: (Pseudocode) procedure Binary search(x: integer, a 1,a 2,a 3,…, a n : increasing integers) i = 1 (left endpoint of search interval) j = 1 (right endpoint of search interval) while (i < j) begin m = ⌊ (i+j)/2 ⌋ if x > a m then i = m+1 else j = m end if x = a i return i else return 0

public class BinarySearch { public static int search(int x, int[] list) { int i=0, j = list.length-1, m =0; while (i < j) { m = (int)Math.floor( ((double)(i+j)) / 2.0); System.out.println("i = " + i + ", j = " + j + ", m = " + m); System.out.println(" x = " + x + ", list[m]= " + list[m]); if(x > list[m]) i = m+1; else j = m; } System.out.println("last step: i = " + i + ", j = " + j + ", list[i] = " + list[i]); if(x == list[i]) return i; else return 0; } public static void main(String arg[]) { int list[] = {1,2,3,5,6,7,8,10,12,13,15,16}; int x = 13; int m = BinarySearch.search(x,list); System.out.println("index of x in list = " + m); }

Third, we would type “javac BinarySearch.java” to compile our java program. If successful, a file called “BinarySearch.class” is created. This is the bytecode for our program. Lastly, to run the program type “java BinarySearch” Java compiling

Java Compiling

Question Class exercise: List all the steps used to search for 9 in the sequence 1,3,4,5,6,8,9,11 using (a)a linear search (b)a binary search Answer: (a) i=1,i=2,i=3,i=4,i=5,i=6,i=7 So Location=7 (b) i=1,j=8,m=4,i=5,j=8,m=6,i=7,j=8,m=7,j=7,location=7 Question

Input sequence Output sequence such that: a1’<=a2’<=a3’…<=an’ or a1’>=a2’>=a3’…>=an’ Sorting

Bubble sort is one of the simplest sorting algorithms, but not one of the most efficient. The smaller elements “bubble” to the top,the larger elements “sink” to the bottom. The Bubble sort algorithm: Procedure bubblesort(a1,…,an :real numbers with n>=2) for i:=1 to n-1 for j=1 to n-I if aj > aj+1 then interchange aj and aj+1 {a1,…,an is in increasing order}

Sorting Use the bubble sort to put 3,2,4,1,5 into increasing order.

Insertion sort is a simple sorting algorithm, a comparison sort in which the sorted array is built one entry at a time. The insertion sort algorithm: Procedure insertion sort(A[1:n]) for j:=2 to n do key=A [j] i=j-1; while i>0 and A [i]>key do A [i+1]:=A [i] i:=i-1 end A [i+1]:=key End {A [n] are sorted} Sorting keyi j

Insertion Example: done Sorting

Class Exercise: Sorting the sequence numbers: 6,2,3,1,5,4 using (a)Bubble sort (b)Insertion sort Question

Algorithm Analysis Why need algorithm analysis ?  Writing a working program is not good enough  The program may be inefficient!  If the program is run on a large data set, then the running time becomes an issue

Without worry about the hardware and software used to implement an algorithm. Assume the different operations used in an algorithm take the same time, which simplifier the analysis. Determine whether it is practical to use a particular algorithm to solve a problem as the size of the input increase Compare two algorithms to determine which is more efficient as the size of input grows. The growth of functions

Big-O notation is used extensively to estimate the number of operations an algorithm as its input grows. Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers.  c, n 0 > 0 such that f(N)  c g(N) when N  n 0 g(N) is an upper bound on f(N) f(x) is big –O of g(x) Big-O Notation

F(x)=x 2 +2x+1 is O(x 2 ) We observe that we can readily estimate of f(x) when x>1 because x 1. x 2 <=x 2 +2x+1<=x 2 +2x 2 +x 2 =4x 2 Consequently, we can take c=4 and n 0 =1 Big-O Notation

Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers.  c, n 0 > 0 such that f(N)  c g(N) when N  n 0 g(N) is an lower bound on f(N) f(x) is big –  of g(x) Big-Omega

Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. f(N) =  (g(N)) iff f(N) = O(g(N)) and f(N) =  (g(N)) g(N) is both an upper and a lower bound on the size of f(N) Big-Theta

Show that 3x 2 +8x log x is  (x 2 ) 0  8x log x  8 x 2, it follows that 3x 2 +8x log x  11 x 2 for x>1. Consequently, 3x 2 +8x log x is O(X 2 ). Clearly. X 2 is O(3x 2 +8x log x ). So, 3x 2 +8x log x is  (x 2 ) Big-Theta

Class Exercise: Using the definition of Big-O notation to proof: (1)Suppose that f 1 (x) is O(g 1 (x)) and f 2 x is O(g 2 (x)). Then (f 1 +f 2 )(x) is O(max(|g 1 (x)|,|g 2 (x)|). (2)Suppose that f 1 (x) is O(g 1 (x)) and f 2 x is O(g 2 (x)). Then (f 1 f 2 )(x) is O(g 1 (x)g 2 (x)). Exercise

Best Case – constraints on the input, other than size, resulting in the fastest possible running time. Worst Case – constraints on the input, other than size, resulting in the slowest possible running time. Average Case – average running time over every possible type of input (usually involve probabilities of different types of input). Complexity of Algorithm

Three Analyses of Insertion Sorting Best Case A[1] ≤ A[2] ≤ A[3] ≤ ··· ≤ A[n] The number of comparisons needed is equal to Worst Case A[1] ≥ A[2] ≥ A[3] ≥ ··· ≥ A[n] The number of comparisons needed is equal to Average Case Θ(n2) assuming that each of the n! instances are equally likely.

Complexity of Algorithm Commonly Used Terminology for the Complexity of Algorithm Complexity Terminology Θ(1)Constant Complexity Θ(log n)Logarithmic Complexity Θ(n)Linear Complexity Θ(n log n)N log N complexity Θ(n b )Polynomial Complexity Θ(b n )Exponential Complexity Θ(n!)Factorial Complexity