Week 12: Running Time and Performance 1.  Most of the problems you have written in this class run in a few seconds or less  Some kinds of programs can.

Slides:



Advertisements
Similar presentations
MATH 224 – Discrete Mathematics
Advertisements

CHAPTER 2 ALGORITHM ANALYSIS 【 Definition 】 An algorithm is a finite set of instructions that, if followed, accomplishes a particular task. In addition,
Introduction to Algorithms Rabie A. Ramadan rabieramadan.org 2 Some of the sides are exported from different sources.
HST 952 Computing for Biomedical Scientists Lecture 10.
Week 12 - Wednesday.  What did we talk about last time?  Hunters and prey  Class variables  Big Oh notation.
 The amount of time it takes a computer to solve a particular problem depends on:  The hardware capabilities of the computer  The efficiency of the.
Algorithm Analysis (Big O) CS-341 Dick Steflik. Complexity In examining algorithm efficiency we must understand the idea of complexity –Space complexity.
Fall 2006CENG 7071 Algorithm Analysis. Fall 2006CENG 7072 Algorithmic Performance There are two aspects of algorithmic performance: Time Instructions.
CSE332: Data Abstractions Lecture 2: Math Review; Algorithm Analysis Tyler Robison Summer
Introduction to Analysis of Algorithms
Complexity Analysis (Part I)
CS 307 Fundamentals of Computer Science 1 Asymptotic Analysis of Algorithms (how to evaluate your programming power tools) based on presentation material.
© 2006 Pearson Addison-Wesley. All rights reserved6-1 More on Recursion.
1 Data Structures A program solves a problem. A program solves a problem. A solution consists of: A solution consists of:  a way to organize the data.
1 Algorithms and Analysis CS 2308 Foundations of CS II.
Analysis of Algorithms 7/2/2015CS202 - Fundamentals of Computer Science II1.
Analysis of Algorithms COMP171 Fall Analysis of Algorithms / Slide 2 Introduction * What is Algorithm? n a clearly specified set of simple instructions.
Design and Analysis of Algorithms Chapter Analysis of Algorithms Dr. Ying Lu August 28, 2012
Analysis of Algorithms Spring 2015CS202 - Fundamentals of Computer Science II1.
Algorithm Analysis (Big O)
Data Structures Introduction Phil Tayco Slide version 1.0 Jan 26, 2015.
COMPSCI 102 Introduction to Discrete Mathematics.
1 Complexity Lecture Ref. Handout p
Big Oh Algorithm Analysis with
CSC 201 Analysis and Design of Algorithms Lecture 03: Introduction to a CSC 201 Analysis and Design of Algorithms Lecture 03: Introduction to a lgorithms.
Week 2 CS 361: Advanced Data Structures and Algorithms
1 Chapter 24 Developing Efficient Algorithms. 2 Executing Time Suppose two algorithms perform the same task such as search (linear search vs. binary search)
1 Recursion Algorithm Analysis Standard Algorithms Chapter 7.
{ CS203 Lecture 7 John Hurley Cal State LA. 2 Execution Time Suppose two algorithms perform the same task such as search (linear search vs. binary search)
Lecture 2 Computational Complexity
Algorithm Efficiency CS 110: Data Structures and Algorithms First Semester,
Week 2 - Monday.  What did we talk about last time?  Exceptions  Threads  OOP  Interfaces.
Chapter 12 Recursion, Complexity, and Searching and Sorting
Object-Oriented Design CSC 212. Announcements Ask more questions!  Your fellow students have the same questions (remember, I grade the daily quizzes)
© 2011 Pearson Addison-Wesley. All rights reserved 10 A-1 Chapter 10 Algorithm Efficiency and Sorting.
Arrays Tonga Institute of Higher Education. Introduction An array is a data structure Definitions  Cell/Element – A box in which you can enter a piece.
Analysis of Algorithms These slides are a modified version of the slides used by Prof. Eltabakh in his offering of CS2223 in D term 2013.
Introduction to Analysis of Algorithms COMP171 Fall 2005.
Program Efficiency & Complexity Analysis. Algorithm Review An algorithm is a definite procedure for solving a problem in finite number of steps Algorithm.
Week 2 - Wednesday.  What did we talk about last time?  Running time  Big Oh notation.
Algorithm Analysis Part of slides are borrowed from UST.
Complexity & Computability. Limitations of computer science  Major reasons useful calculations cannot be done:  execution time of program is too long.
Week 13 - Monday.  What did we talk about last time?  B-trees  Hamiltonian tour  Traveling Salesman Problem.
Algorithm Analysis (Big O)
Week 12 - Friday.  What did we talk about last time?  Finished hunters and prey  Class variables  Constants  Class constants  Started Big Oh notation.
27-Jan-16 Analysis of Algorithms. 2 Time and space To analyze an algorithm means: developing a formula for predicting how fast an algorithm is, based.
CS 150: Analysis of Algorithms. Goals for this Unit Begin a focus on data structures and algorithms Understand the nature of the performance of algorithms.
Week 12 - Wednesday.  What did we talk about last time?  Hunters and prey.
Computer science is a field of study that deals with solving a variety of problems by using computers. To solve a given problem by using computers, you.
Week 12 - Monday.  What did we talk about last time?  Defining classes  Class practice  Lab 11.
CS216: Program and Data Representation University of Virginia Computer Science Spring 2006 David Evans Lecture 8: Crash Course in Computational Complexity.
Onlinedeeneislam.blogspot.com1 Design and Analysis of Algorithms Slide # 1 Download From
CSC 212 – Data Structures Lecture 15: Big-Oh Notation.
BITS Pilani Pilani Campus Data Structure and Algorithms Design Dr. Maheswari Karthikeyan Lecture1.
Algorithm Analysis with Big Oh ©Rick Mercer. Two Searching Algorithms  Objectives  Analyze the efficiency of algorithms  Analyze two classic algorithms.
Analysis of Algorithms Spring 2016CS202 - Fundamentals of Computer Science II1.
Chapter 15 Running Time Analysis. Topics Orders of Magnitude and Big-Oh Notation Running Time Analysis of Algorithms –Counting Statements –Evaluating.
CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis Linda Shapiro Winter 2015.
CSE 3358 NOTE SET 2 Data Structures and Algorithms 1.
Algorithmic Foundations COMP108 COMP108 Algorithmic Foundations Algorithm efficiency Prudence Wong
(Complexity) Analysis of Algorithms Algorithm Input Output 1Analysis of Algorithms.
Week 2 - Wednesday CS221.
Week 13: Searching and Sorting
Introduction to Algorithms
Week 2 - Friday CS221.
Week 11 - Monday CS221.
Algorithm Analysis (not included in any exams!)
CS 201 Fundamental Structures of Computer Science
Searching, Sorting, and Asymptotic Complexity
slides created by Ethan Apter
Presentation transcript:

Week 12: Running Time and Performance 1

 Most of the problems you have written in this class run in a few seconds or less  Some kinds of programs can take much longer:  Chess algorithms (Deep Blue)  Routing and scheduling algorithms (Traveling Sales Person, Purdue classroom scheduling)  Medical imaging (massive amounts of data)  Graphics processing (Pixar: Toy Story, Incredibles) 2

 You can time the program  What if it takes 1000 years to run?  You can look at the program  What are you looking for?  Does the input matter?  Sorting 100,000 numbers must take longer than sorting 10 numbers, right? 3

 A sequence of steps used to solve a problem  In CS, we take an algorithm and convert it into code -- Java in this class  The study of algorithms is intended to:  Make sure that the code you write gets the right answer  Make sure that the code runs as quickly as possible  Make sure that the code uses a reasonable amount of memory 4

 You want to find the definition of the word “rheostat” in the dictionary  The Oxford English Dictionary has 301,100 main entries  What if you search for the word “rheostat” by starting at the beginning and checking each word to see if it is the one we are interested in  Given a random word, you’ll have to check about 150,550 entries before you find it  Alternatively, you could jump to the correct letter of the alphabet, saving tons of time  Searching for “zither” takes forever otherwise 5

 Write a program to sum up the numbers between 1 and n  Easy, right?  Let us recall that a faster way might be to use the summation equation: 6

 Memory usage is a problem too  If you run out of memory, your program can crash  Memory usage can have serious performance consequences too 7

 Remember, there are multiple levels of memory on a computer  Each next level is on the order of 100 times larger and 100 times slower Cache Actually on the CPU Fast and expensive RAM Primary memory for a desktop computer Pretty fast and relatively expensive Hard Drive Secondary memory for a desktop computer Slow and cheap 100X SizeSpeed 8

 If you can do a lot of number crunching without leaving cache, that will be very fast  If you have to fetch data from RAM, that will slow things down  If you have to read and write data to the hard drive (unavoidable with large pieces of data like digital movies), you will slow things down a lot 9

 Memory can be easier to estimate than running time  Depending on your input, you will allocate a certain number of objects, arrays, and primitive data types  It is possible to count the storage for each item allocated  A reference to an object or an array costs an additional 4 bytes on top of the size of the object 10

 A graph is a set of nodes and edges  It’s a really simple way to represent almost any kind of relationship  The numbers on the edges could be distances, costs, time, whatever A A B B D D C C E E

 There’s a classic CS problem called the Traveling Salesperson Problem (TSP)  Given a graph, find the shortest path that visits every node and comes back to the starting point  Like a lazy traveling salesperson who wants to visit all possible clients and then return home as soon as possible 12

 Strategies:  Always visit the nearest neighbor next  Randomized approaches  Try every possible combination  How can you tell what’s the best solution? 13

 We are tempted to always take the closest neighbor, but there are pitfalls 14

 In a completely connected graph, we can try any sequence of nodes  If there are n nodes, there are (n – 1)! tours  For 30 cities, 29! =  If we can check 1,000,000,000 tours in one second, it will only take about 20,000 times the age of the universe to check them all  We will (eventually) get the best answer! 15

 This and similar problems are useful problems to solve for people who need to plan paths  UPS  Military commanders  Pizza delivery people  Internet routing algorithms  No one knows a solution that always works in a reasonable amount of time  We have been working on it for decades 16

 The problem is so hard that you can win money if you solve it  Clay Mathematics Institute has offered a $1,000,000 prize  You can do one of two things to collect it:  Find an efficient solution to the general problem  Prove that it is impossible to find an efficient solution in some cases 17

 How do we measure algorithm running time and memory usage in general?  We want to take into account the size of the problem (TSP with 4 cities is certainly easier than TSP with 4,000 cities)  Some algorithms might work well in certain cases and badly in others  How do we come up with a measuring scheme? 18

 Ideally, we want to classify both problems and algorithms  We want to figure out which problems take a really long time to solve  We want to figure out how we can compare one algorithm to another  We want to prove that some algorithm is the best possible algorithm to solve a particular problem 19

 Computer scientists use Big-Oh notation to give an indication of how much time (or memory) a program will take  Big-Oh notation is:  Asymptotic Focuses on the behavior as the size of input gets large  Worst-Case Focuses on the absolute longest the program can take 20

 Add up the operations done by the following code:  Initialization: 1 operation  Loop: 1 initialization + n checks + n increments + n additions to sum = 3n + 1  Output: 1 operation  Total: 3n + 3 int sum = 0; for( int i = 0; i < n; i++ ) sum += i; System.out.println(“Sum: “ + sum); 21

 We could express the time taken by the code on the previous slide as a function of n: f(n) = 3n + 3  This approach has a number of problems:  We assumed that each line takes 1 time unit to accomplish, but the output line takes much longer than an integer operation  This program is 4 lines long, a longer program is going to have a very messy running time function  We can get nit picky about details: Does sum += i; take one operation or two if we count the addition and the store separately? 22

 In short, this way of getting a running time function is almost useless because:  It cannot be used to give us an idea of how long the program really runs in seconds  It is complex and unwieldy  The most important thing about the analysis of the code that we did is learning that the growth of the function should be linear (3n+3)  A general description of how the running time grows as the input grows would be useful 23

 Enter Big Oh notation  Big Oh simplifies a complicated running time function into a simple statement about its worst case growth rate  All constant coefficients are ignored  All low order terms are ignored  3n + 3 is O(n)  Big Oh is a statement that a particular running time is no worse than some function, with appropriate constants 24

 147n 3 + 2n 2 + 5n is O(n3)O(n3)  15n 2 + 6n + 7log n is O(n2)O(n2)  log n is O(log n)  Note: In CS, we use log 2 unless stated otherwise 25

 How long does it take to do multiplication by hand? 123 x __  Let’s assume that the length of the numbers is n digits  (n multiplications + n carries) x n digits + (n + 1 digits) x n additions = 3n 2 + n  Running time: O(n 2 ) 26

 How do we find the largest element in an array?  Running time: O(n) if n is the length of the array  What if the array is sorted in ascending order?  Running time: O(1) int largest = array[0]; for( int i = 1; i < array.length; i++ ) if( array[i] > largest ) largest = array[i]; System.out.println(“Largest: “ + largest); System.out.println(“Largest: “ + array[array.length-1]); 27

 Given two n x n matrices A and B, the code to multiply them is:  Running time: O(n 3 )  Is there a faster way to multiply matrices?  Yes, but it’s complicated and has other problems double[][] c = new double[n][n]; for( int i = 0; i < n; i++ ) for( int j = 0; j < n; j++ ) for( int k = 0; k < n; k++ ) c[i][j] += a[i][k]*b[k][j]; 28

 Here is some code that sorts an array in ascending order  What is its running time?  Running time: O(n 2 ) for( int i = 0; i < array.length; i++ ) for( int j = 0; j < array.length - 1; j++ ) if( array[j] > array[j + 1] ) { int temp = array[j]; array[j] = array[j + 1]; array[j + 1] = temp; } 29

 There is nothing better than constant time (O(1))  Logarithmic time (O(log n)means that the problem can become much larger and only take a little longer  Linear time (O(n))means that time grows with the problem  Quadratic time (O(n 2 )) means that expanding the problem size significantly could make it impractical  Cubic time (O(n 3 )) is about the reasonable maximum if we expect the problem to grow  Exponential and factorial time mean that we cannot solve anything but the most trivial problem instances 30

 Looking at the code and determining the order of complexity is one technique  Sometimes this only captures worst-case behavior  The code can be complicated and difficult to understand  Another possibility is running experiments to see how the running time changes as the problem size increases 31

 The doubling hypothesis states that it is often possible to determine the order of the running time of a program by progressively doubling the input size and measuring the changes in time 32

 Let’s use the doubling hypothesis to test the running time of the program that finds the largest element in an unsorted array  Looks like a decent indication of O(n) SizeTimeRatio ms ms ms ms

 Let’s check matrix multiplication  A factor of 8 is expected for O(n 3 )  Not too bad SizeTimeRatio ms ms ms ms

 Let’s try bubble sort  Success! O(n 2 ) is supposed to give a ratio of 4  We should have done more trials to do this scientifically SizeTimeRatio ms ms ms ms