Prof. Hsin-Mu (Michael) Tsai ( 蔡欣穆 ) Department of Computer Science and Information Engineering National Taiwan University BASICS

WHAT IS AN ALGORITHM? A computable set of steps to achieve a desired result. All algorithm must satisfy the following criteria: Input Output Definiteness Finiteness Effectiveness DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

WHAT IS A DATA STRUCTURE? An organization of information, usually in memory, for better algorithm efficiency. Or, a way to store and organize data in order to facilitate access and modifications. DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

ALGORITHM SPECIFICATION DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

HOW DO WE DESCRIBE AN ALGORITHM? Human language (English, Chinese, …) Programming language A mix of the above DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE: SELECTION SORT DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL ㄅ 1 1 ㄆㄅ 2 1 ㄆㄅ 2

EXAMPLE: SELECTION SORT First attempt: for (i=0; i<n; ++i) { Examine list[i] to list[n-1] and suppose that the smallest integer is at list[min]; Interchange list[i] and list[min]; } Task 1 Task 2 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

void swap(int *x, int *y) { int temp = *x; *x=*y; *y=temp; } Or #define SWAP(x,y,t) ((t)=(x), (x)=(y), (y)=(t)) Task 2 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

min=i; for(j=i;j<n;++j) if (list[j]<list[min]) min=j; Task 1 Finally, see program 1.4 on p. 11 for a complete program of the selection sort we just develop. DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

HOW DO WE PROVE THAT IT IS CORRECT? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE: BINARY SEARCH DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE: searchnum=13; return middle; DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE: searchnum=13; left right middl e middle=(left+right)/2; left=middle+1; DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE: searchnum=5; return -1; DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

int binsearch(int list[], int searchnum, int left, int right) { int middle; while(left<=right) { middle=(left+right)/2; switch(COMPARE(list[middle], searchnum)) { case -1: left=middle+1; break; case 0: return middle; case 1: right=middle-1; } return -1; } DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

RECURSIVE ALGORITHMS What does “recursive” mean? A function which calls itself (direct recursion), or a function which calls other functions which call the calling function again (indirect recursion). Any function that we can write using assignment, if-else, and while statements can be written recursively. Why do we want to use recursive functions (or algorithms)? It allow us to express an otherwise complex process in very clear terms. Often the recursive function is easier to understand than its iterative counterpart. DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

THE STORY OF RECURSIVE FUNCTION Function blah Function blah (clone 1) Function blah (clone 2) Function blah (clone 2) ㄅ is too much for me. I’ll partition it, do part 1, and clone two copies of myself to do the rest. Yo. Do the work ㄅ. I’m ㄅ. The work to be done. Hey man. Do the work ㄅ part 2. Hi. Do the work ㄅ part 3. ㄅ ㄅ DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE 1 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE 2 int binsearch(int list[], int searchnum, int left, int right) { int middle; if (left<=right) { middle=(left+right)/2; switch (COMPARE(list[middle], searchnum)) { case -1: return binsearch(list, searchnum, middle+1, right); case 0: return middle; case 1: return binsearch(list, searchnum, left, middle-1); } 1. Termination condition 2. Recursive calls DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE 3 Output all permutations Input: {a,b,c} Output: (a,b,c),(a,c,b),(b,a,c),(b,c,a),(c,a,b),(c,b,a) How do we write this program recursively? Example: input={a,b,c,d}. Output = a followed by all permutations of {b,c,d} b followed by all permutations of {a,c,d} c followed by all permutations of {a,b,d} d followed by all permutations of {a,b,c} “all permutations of {…}”  recursive calls! Homework: read and understand program 1.9 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

DATA ABSTRACTION DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

DATA TYPE What is a data type? A data type is a collection of objects and a set of operations that act on those objects. Data types in C char, int, float, long, double (unsigned, signed, …) Array Struct struct { int a; int b; char str[16]; int * iptr; } blah; int iarray[16]; DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

DATA TYPE Operations +, -, *, /, %, == =, +=, -= ? : sizeof, - (negative) giligulu(int a, int b) DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

DATA TYPE Representation of the objects of the data type Example: char char blah=‘A’; (‘A’: ASCII code is 65(dec), or 0x41 (hex)) Q: The maximum number which can be represented with a char variable? A: 255. Homework: How about char, int, long, float? byte of memory: DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

DATA TYPE Q: Do we need to know about the representation of a data type? A: It depends. We can usually write algorithms which are more efficient if we make use of the knowledge about the representation. However!!! If the representation of the data type is changed, the program needs to be verified, revised, or completely re-written. 囧 Porting to a different platform (x86, ARM, embedded system, …) Changing the specification of a program or a library (ex. 16-bit int  32-bit long) DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

ABSTRACT DATA TYPE An “abstract data type” (ADT) is a data type that is organized in such a way that the specification of the objects and the specification of the operations on the objects is separated from the representation of the objects and the implementation of the operations. Representation and Implementation Specification (Interface) User DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

ABSTRACT DATA TYPE Specifications: Name of the function and the description of what the function does The type of the argument(s) The type of the result(s) (return value) Function categories: Creator/constructor Transformers Observer/reporter Homework: Read Example 1.5 on p. 20 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

PERFORMANCE ANALYSIS DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

HOW DO YOU MAKE EVALUATIVE JUDGMENT ABOUT PROGRAM? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Does the program meet the original specifications of the task? 2.Does it work correctly? 3.Does the program contain documentation that shows how to use it and how it works? 4.Does the program effectively use functions to create logical units? 5.Is the program’s code readable?

HOW DO YOU MAKE EVALUATIVE JUDGMENT ABOUT PROGRAM? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Does the program efficiently use primary and secondary storage? Primary storage: memory? Secondary storage: Hard drive, flash disk, etc. 7.Is the program’s running time acceptable for the task? Example: Network intrusion detection system (1) 99.8% detection rate, 50 minutes to finish analysis of a minute of traffic (2) 85% detection rate, 20 seconds to finish analysis of a minute of traffic

HOW DO YOU MAKE EVALUATIVE JUDGMENT ABOUT PROGRAM? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Does the program efficiently use primary and secondary storage? 7.Is the program’s running time acceptable for the task?

SPACE & TIME COMPLEXITY DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Space complexity of a program: The amount of memory that it needs to run to completion. Time complexity of a program: The amount of computer time that it needs to run to completion.

SPACE COMPLEXITY DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Is it good to use? (recursion) TypeNameNumber of bytes parameters: array pointerlist[]4 parameter: integern4 return address: (used internally) 4 TOTAL per recursive call12 Probably not for this problem.

TIME COMPLEXITY DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Method 1: Count all instances of all operations in the program. AddSubtractLoadStore ADD(n)SUB(n)LDA(n)STA(n) Is it good to use? (method 1)

DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Method 2: Separate the program into program steps whose execution time is independent of instance characteristics Count the number of steps Different program steps have different amounts of computing a=2; a=2*b+3*c/d-e+f/g/a/b/c; Count the number of steps needs to solve a particular instance

EXAMPLE 1 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL float sum(float list[], int n) { float tempsum = 0; count++; //assignment int i; for (i=0;i<n;i++) { count++; // for loop tempsum+=list[i]; count++; //assignment } count++; //last iteration of for count++; return tempsum; }

EXAMPLE 1 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL float sum(float list[], int n) { float tempsum = 0; int i; for (i=0;i<n;i++) { count+=2; } count+=3; return tempsum; } count = 2n+3 (steps)

EXAMPLE 2 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL float rsum(float list[], int n) { count++; // if if (n) { count++; //return return rsum(list,n-1)+list[n-1]; } count++; //return return list[0]; } count = 2n+2 (steps) 2n+2 < 2n+3. Does this mean rsum is faster than sum ? No!

EXAMPLE 3 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL void add(int a[][MAX_SIZE], int b[][MAX_SIZE], int c[][MAX_SIZE], int rows, int cols) { int i,j; for(i=0; i<rows; ++i) for(j=0;j<cols;++j) c[i][j]=a[i][j]+b[i][j]; }

EXAMPLE 3 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL void add(int a[][MAX_SIZE], int b[][MAX_SIZE], int c[][MAX_SIZE], int rows, int cols) { int i,j; for(i=0; i<rows; ++i) { count++; for(j=0;j<cols;++j) { count++; c[i][j]=a[i][j]+b[i][j]; count++; } count++; } count++; } count=((2*cols)+2)*rows+1 =2cols*rows+2*rows+1

EXAMPLE 1 REVISITED (TABULAR METHOD) DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Statements/eFrequencyTotal steps float sum(float list[], int n) { 000 float tempsum=0; 111 int i; 000 for (i=0;i<n;i++) 0n+1 tempsum+=list[i]; 1nn return tempsum; } 111 Please practice using the method for the other two examples.

SUMMARY DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Instance characteristics? The number of inputs The number of outputs The magnitude of the inputs We then calculate the number of steps which is independent of the characteristics we selected.

DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Sometimes, the time complexity does not depend on only these simple numbers. For example, binsearch. In that case, we are interested in the average case, the worst case, and the best case.

ASYMPTOTIC NOTATION DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

MOTIVATION DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Compare the time complexities of two programs that compute the same function. Predict the growth in run time as the instance characteristics change. “Steps” are not exact. “3n+3” faster than “3n+5” ? Usually “3n+3”, “7n+2”, or “2n+15” all runs for about the same time. Let’s go for a less accurate way to describe the run time…

EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Break even point No.

ASYMPTOTIC NOTATION – BIG OH DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

THE WORLD OF BIG OH DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Faster Slower

BIG OH IS AN UPPER BOUND DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

A USEFUL THEOREM DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

ASYMPTOTIC NOTATION – OMEGA DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

DISCUSSION DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Homework 1

ASYMPTOTIC NOTATION – THETA DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Theta Platform GMC Terrain

EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

DISCUSSION DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Homework 1

SO, WHAT NOW? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Asymptotic complexity can be determined quite easily Work on each statement first Then we add them up No need for the silly step counts anymore

EXAMPLE 1 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL StatementAsymptotic complexity void add(int a[][MAX_SIZE]…) {0 int i,j;0 for(i=0;i<rows;i++) for(j-0;j<cols;j++) c[i][j]=a[i][j]+b[i][j]; }0 Total

EXAMPLE 2 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL int binsearch(int list[], int searchnum, int left, int right) { int middle; while(left<=right) { middle=(left+right)/2; switch(COMPARE(list[middle], searchnum)) { case -1: left=middle+1; break; case 0: return middle; case 1: right=middle-1; } return -1; }

searchnum=13; left right middl e middle=(left+right)/2; left=middle+1; DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Half the searching window every iteration.

MORE EXAMPLES DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL Please take a look at example 1.20 and 1.21 (p )

PRACTICAL COMPLEXITY DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

兜基 ?? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

WHICH ONE IS BETTER? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL

ON A 1 BILLION-STEPS-PER-SEC COMPUTER DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL n

下課 ! DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 等等 ~~ Homework 1 is out today, the due date is two weeks from yesterday (5pm). Download the question sets from the course website your question to Or, come to our office hours. Have a nice weekend!