CS 2430 Day 30

Announcements Quiz #5: 4/19

Agenda Big O Searching –Linear search

Computational complexity and big O notation

Algorithm efficiency Need a way to determine how “efficient” an algorithm so we can compare algorithms Could try timings, but they can vary a lot depending on the input and/or hardware Another approach is to count the number of steps required to process N elements

Critical steps To get a feel for the behavior of the algorithm, we just count the dominant instructions –for a sort, it would be the comparison of array elements –for a search, it would be the comparison of an array element to the item being searched for The count will be some function of N, i.e., the number of items being processed

Big O The “Big O” of an algorithm is the order of the fastest growing term of the number of critical steps performed (as a function of N) Examples: –2N / = O(N) –42 = 42N 0 = O(1) –100N 2 + 7N / = O(N 2 ) –Log N + 31 = O(Log 2 N), where “Log 2 N” is the base-2 logarithm of N

Big O examples Search an unsorted array? O(N) Push onto a Stack ? O(1) Remove from a Bag ? O(N)

Matrix multiplication

public class Matrix { private double[][] data; // matrix entries private int n; // dimension of matrix, assume square... public Matrix times(Matrix rhs) { Matrix lhs = this; Matrix res = new Matrix(lhs.n); // creates empty square Matrix for (int i = 0; i < res.n; i++) for (int j = 0; j < res.n; j++) for (int k = 0; k < lhs.n; k++) res.data[i][j] += lhs.data[i][k] * rhs.data[k][j]; return res; } } Let N denote the matrix dimension, i.e., n What is the big O of times() ? O(N 3 )

Is it possible to do better than O(N 3 ) for matrix multplication?

Yes! Middle school matrix multiplication algorithmO(N 3 ) Strassen algorithm (1969)O(N ) Coppersmith–Winograd algorithm (1987, 1990) O(N ) Williams algorithm (2011)O(N ) (20??)O(N 2 ) – conjectured to be possible!

Bigger big O and beyond! Agrawal–Kayal–Saxena primality test algorithm (2002) O(N 10 ) –The exponent continues to decrease Traveling Salesman problem (NP-complete) O(N!) – brute force O(2 N ) – “Dynamic programming” Halting problem (undecidable)Cannot be solved by any computer given any amount of time. No big O! Cannot even be solved by humans (in general)

Searching

Linear search private static int lsearch(Comparable a[], int n, E x) { for (int i = 0; i < n; i++) if (a[i].compareTo(x) == 0) return i; return -1; } Can put this method in any container class that needs it What is the worst case big O? O(N)

Linear search on average What is the average case efficiency of linear search? Given a randomly sorted array that contains the target element X, how many comparisons needed? Answer = 1*Pr[X is in position 0] + 2*Pr[X is in position 1] + ∙ ∙ ∙ + N*Pr[X is in position N – 1] = ???

How to get the answer! ???= 1*Pr[X is in position 0] + 2*Pr[X is in position 1] + ∙ ∙ ∙ + N*Pr[X is in position N – 1] = 1 / N + 2 / N + ∙ ∙ ∙ + N / N = (1 / N) * N*(N+1) / 2 = (N + 1) / 2 = O(N)

Can we do better if the array is sorted?