Performance Measurement

Slides:



Advertisements
Similar presentations
CHAPTER 2 ALGORITHM ANALYSIS 【 Definition 】 An algorithm is a finite set of instructions that, if followed, accomplishes a particular task. In addition,
Advertisements

Fundamentals of Python: From First Programs Through Data Structures
Introduction to Analysis of Algorithms
Performance Measurement Performance Analysis Paper and pencil. Don’t need a working computer program or even a computer.
Insertion Sort for (i = 1; i < n; i++) {/* insert a[i] into a[0:i-1] */ int t = a[i]; int j; for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j];
Lecture 3 Aug 31, 2011 Goals: Chapter 2 (algorithm analysis) Examples: Selection sorting rules for algorithm analysis discussion of lab – permutation generation.
Time Complexity s Sorting –Insertion sorting s Time complexity.
Lecture 3 Feb 7, 2011 Goals: Chapter 2 (algorithm analysis) Examples: Selection sorting rules for algorithm analysis Image representation Image processing.
1 Performance Measurement CSE, POSTECH 2 2 Program Performance Recall that the program performance is the amount of computer memory and time needed to.
Insertion Sort for (int i = 1; i < a.length; i++) {// insert a[i] into a[0:i-1] int t = a[i]; int j; for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1]
Analysis of Algorithms
Asymptotic Notation (O, Ω, )
Performance Measurement Performance Analysis Paper and pencil. Don’t need a working computer program or even a computer.
SNU IDB Lab. Ch4. Performance Measurement © copyright 2006 SNU IDB Lab.
Processor Structure and Function Chapter8:. CPU Structure  CPU must:  Fetch instructions –Read instruction from memory  Interpret instructions –Instruction.
ANALYSING COSTS COMP 103. RECAP  ArrayList: add(), ensuring capacity, iterator for ArrayList TODAY  Analysing Costs 2 RECAP-TODAY.
TAs Fardad Jalili: Aisharjya Sarkar: Soham Das:
1 Chapter 2 Program Performance. 2 Concepts Memory and time complexity of a program Measuring the time complexity using the operation count and step count.
LECTURE 9 CS203. Execution Time Suppose two algorithms perform the same task such as search (linear search vs. binary search) and sorting (selection sort.
Complexity Analysis (Part I)
CSC 427: Data Structures and Algorithm Analysis
May 17th – Comparison Sorts
Analysis of Algorithms
CS/COE 1541 (term 2174) Jarrett Billingsley
William Stallings Computer Organization and Architecture 8th Edition
Recitation 13 Searching and Sorting.
Introduction to complexity
Big-O notation.
Source: wikipedia
Introduction to Algorithms
Architecture Background
Lecture 6 Memory Hierarchy
Source:
Algorithm Analysis CSE 2011 Winter September 2018.
Computer Science 210 Computer Organization
Performance Measurement
CSCI1600: Embedded and Real Time Software
Big-Oh and Execution Time: A Review
Algorithm Analysis (not included in any exams!)
Building Java Programs
Computer Science 210 Computer Organization
CSE 143 Lecture 5 Binary search; complexity reading:
Insertion Sort for (int i = 1; i < a.length; i++)
Quicksort analysis Bubble sort
Algorithm An algorithm is a finite set of steps required to solve a problem. An algorithm must have following properties: Input: An algorithm must have.
CSC 413/513: Intro to Algorithms
Lesson 15: Processing Arrays
Performance Measurement
An Overview of Insertion Sort
Data Structures Review Session
CS 201 Fundamental Structures of Computer Science
Building Java Programs
Simple Sorting Methods: Bubble, Selection, Insertion, Shell
Sorting Rearrange a[0], a[1], …, a[n-1] into ascending order.
CSE 373 Data Structures and Algorithms
CSC 427: Data Structures and Algorithm Analysis
Insertion Sort for (int i = 1; i < n; i++)
Performance Measurement
Memory System Performance Chapter 3
Sum this up for me Let’s write a method to calculate the sum from 1 to some n public static int sum1(int n) { int sum = 0; for (int i = 1; i
Performance Measurement
CSE 373: Data Structures and Algorithms
Estimating Algorithm Performance
Insertion Sort for (int i = 1; i < n; i++)
CSE 332: Data Abstractions Memory Hierarchy
CSCI1600: Embedded and Real Time Software
Complexity Analysis (Part I)
Modern Collections Classes & Generics
Complexity Analysis (Part I)
Chapter 4 The Von Neumann Model
Presentation transcript:

Performance Measurement

Performance Analysis Paper and pencil. Don’t need a working computer program or even a computer. These could be considered some of the advantages of performance analysis over performance measurement.

Some Uses Of Performance Analysis determine practicality of algorithm predict run time on large instance compare 2 algorithms that have different asymptotic complexity e.g., O(n) and O(n2) If the complexity is O(n2), we may consider the algorithm practical even for large n. An O(2n) algorithm is practical only for small n, say n < 40. The worst-case run time of an O(n2) algorithm will quadruple with each doubling of n. So if the worst-case time is 10 sec when n = 100, it will be approximately 40 sec when n = 200.

Limitations of Analysis Doesn’t account for constant factors. but constant factor may dominate 1000n vs n2 and we are interested only in n < 1000

Limitations of Analysis Modern computers have a hierarchical memory organization with different access time for memory at different levels of the hierarchy.

Memory Hierarchy MAIN L2 L1 ALU R 8-32 32KB 512KB 512MB 1C 2C 10C 100C 1 cycle to add data that are in the registers 2 cycles to copy data from L1 cache to a register 10 cycles to copy from L2 to L1 and register L1 ALU R 8-32 32KB 512KB 512MB 1C 2C 10C 100C

Limitations of Analysis Our analysis doesn’t account for this difference in memory access times. Programs that do more work may take less time than those that do less work. A program that does 100 operations on the same data would take less time than a program that performs a single operation on say 50 different pieces of data (assuming, in a simple model) that each time data is fetched from main memory, only one piece of data is fetched. The first program would make one fetch at a cost of 100 cycles and perform 100 operations at a cost of 1 cycle each for a total of 200 cycles. The second program would need 50 * 100 cycles just to fetch the data.

Performance Measurement Measure actual time on an actual computer. What do we need?

Performance Measurement Needs programming language working program computer compiler and options to use

Performance Measurement Needs data to use for measurement worst-case data best-case data average-case data timing mechanism --- clock

Timing In C++ long start, stop; time(start); // set start to current time in // hundredths of a second // code to be timed comes here time(stop); // set stop to current time long runTime – stop – start; runTime is in hundredths of a second

Shortcoming Clock accuracy assume 1/100 second Repeat work many times to bring total time to be >= 1/10 second Preceding measurement code is acceptable only when the elapsed time is large relative to the accuracy of the clock.

Accurate Timing time(start); long counter; do { counter++; doSomething(); time(stop); } while (stop - start < 10) double elapsedTime = stop - start; double timeForTask = elapsedTime/counter;

Accuracy Now accuracy is 10%. first reading may be just about to change to start + 1 second reading may have just changed to stop so stop - start is off by 1 unit 1 unit = 1/100 sec

Accuracy first reading may have just changed to start second reading may be about to change to stop + 1 so stop - start is off by 1 unit

Accuracy Examining remaining cases, we get trueElapsedTime = stop - start +- 1 To ensure 10% accuracy, require elapsedTime = stop – start >= 10

What Went Wrong? time(start); long counter; do { counter++; insertionSort(a,n); time(stop); } while (stop - start < 10) double elapsedTime = (stop – start); double timeToSort = elapsedTime/counter; Suppose we are measuring worst-case run time. The array a initially is in descending order. After the first iteration, the data is sorted. Subsequent iterations no longer start with worst-case data. So the measured time is for one worst-case sort plus counter-1 best-case sorts!

The Fix time(start); long counter; do { counter++; // put code to initialize a here insertionSort(a,n); time(stop); } while (stop - start < 10) Now the measured time is for counter number of worst-case sorts plus counter number of initializes. We can measure the time to initialize separately and subtract. Or, if we are to compare with other sort methods, we could ignore the initialize time as it is there in the measurements for all sort methods. Or we could argue that for large n the worst case time to sort using insertion sort (O(n2)) overshadows the initialize time, which is O(n), and so we may ignore the initialize time.

Bad Way To Time elapsedTime += stop - start; do { counter++; time(start); doSomething(); time(stop) elapsedTime += stop - start; } while (elapsedTime < 10) Suppose the clock is updated every 1/100 sec and doSomething takes 1/99 sec and it takes another 1/100 sec to do everything else in the loop. If the clock is updated exactly at the start of the loop then it is not updated between the start = and elapsedTime += statements. So elapsedTime is always 0.