1. 1 Data Structures A program solves a problem. A program solves a problem. A solution consists of: A solution consists of:  a way to organize the data  sequence of steps to solve the problem. In other words: In other words:  programs = algorithms + data structures

2. 2 Data Structures From A to Z From A to Z  Problem specification  Analysis and Design  Break up the solution in well-defined modules t Specify pre/post-conditions t Specify how modules interact  IDEA: separate the purpose of a module from its implementation

3. 3 Data Structures  Decide how the data will be organized  Decide what operations can be performed on data  IDEA: separate the data organization and operations from the implementation  Implementation  Testing  Maintenance  Program must be easy to understand/modify

4. 4 Data Structures Key ideas: Key ideas:  Modularity  break problem into subproblems  Abstraction  hide non-relevant information from the user  Information hiding  hide data not needed by a module  Software reuse  do not reinvent the wheel

5. 5 Data Structures Abstract Data Type (ADT) = Abstract Data Type (ADT) =  a collection of data, and  a set of operations on the data. (typical ops?) The user does not know The user does not know  the internal representation  the implementations of the operations The user does know The user does know  a conceptual picture of the organization of the data  how to perform operations on it

6. 6 Data Structures Goals: Goals:  Program must be easy to understand and modify.  Design a data structure that can be reused in other programs.  Design (or select) the right data structure.  The program must be correct and efficient (memory/time).

7. 7 Mathematics Review Basic formulas for derive and reviews basic proof techniques Basic formulas for derive and reviews basic proof techniques Exponents Exponents

8. 8 Mathematics Review Logarithms Logarithms All logarithms are to the base 2 unless specified otherwise. All logarithms are to the base 2 unless specified otherwise. Definition: Definition:

9. 9 Mathematics Review b=2 b=e b=10

10. 10 Mathematics Review Series: Geometric series Series: Geometric series Derivation Let S = 1+A+A 2 +…… (1) where, 0<A<1 then AS = A+A 2 +A 3 +…(2) Subtracting (1) and (2), we get S-AS = 1, i.e.

11. 11 Mathematics Review Series: Arithmetic series Series: Arithmetic series Example: To find the sum 2+5+8+….+ (3k-1) = 3(1+2+3+…+k) - (1+1+1+….+1)

12. 12 Mathematics Review The P word The P word - to proof a false statement: proof by counter example - to proof a correct statement - proof by induction (1) proving a base case (2) inductive hypothesis - proof by contradiction (1) assume it is false (2) show that this assumption is false

13. 13 Algorithms A problem is a specification of an input- output relationship. A problem is a specification of an input- output relationship. Algorithm = Algorithm =  a well-defined computational procedure that takes a set of values as input and produces a set of values as output.  a tool for solving a well-specified computational problem.

14. 14 Algorithm characteristics Correctness Correctness  algorithm halts with correct output for every legal instance of the problem.  proof of correctness Generality Generality  must work for all legal data (eg. a sorting algorithm should work on both numbers and names)

15. 15 Algorithm characteristics Finiteness Finiteness  won’t take forever to run Efficiency Efficiency  making efficient use of resources (time and space)

16. 16 Algorithm characteristics Pseudocode = an English-like language with limited vocabulary that we use to describe algorithms in an easy to understand manner. Pseudocode = an English-like language with limited vocabulary that we use to describe algorithms in an easy to understand manner. Example: Linear Search Example: Linear Search Input : array A, key x Output: index of array element equal to x, -1 if x is not in A for i = 1 to length[A] if A[i]==x then return i endif endfor return -1

17. 17 Algorithm characteristics Is the Linear Search algorithm... Is the Linear Search algorithm...  correct?  efficient?  general?  finite?

18. 18 Algorithm analysis Analyzing an algorithm = estimating the resources it requires Analyzing an algorithm = estimating the resources it requires Time Time  Number of steps/operations executed  Each operation has a cost  A function of problem (input) size Space Space  Amount of temporary storage required.  We usually don’t count the input.

19. 19 Algorithm analysis Best case analysis Best case analysis  Given the algorithm and input of size n that makes it run fastest (compared to all other possible inputs of size n), what is the running time? Worst case analysis Worst case analysis  Given the algorithm and input of size n that makes it run slowest (compared to all other possible inputs of size n), what is the running time?

20. 20 Algorithm analysis Average case analysis Average case analysis  Given the algorithm and a typical, average input of size n, what is the running time?

21. 21 Insertion Sort One of the simplest methods to sort an array is an insertion sort. An example of an insertion sort occurs in everyday life while playing cards. To sort the cards in your hand you extract a card, shift the remaining cards, and then insert the extracted card in the correct place. This process is repeated until all the cards are in the correct sequence. Both average and worst-case time is O(n 2 ). One of the simplest methods to sort an array is an insertion sort. An example of an insertion sort occurs in everyday life while playing cards. To sort the cards in your hand you extract a card, shift the remaining cards, and then insert the extracted card in the correct place. This process is repeated until all the cards are in the correct sequence. Both average and worst-case time is O(n 2 ). Assuming there are n elements in the array, we must index through n - 1 entries. For each entry, we may need to examine and shift up to n - 1 other entries, resulting in a O(n 2 ) algorithm. The insertion sort is an in- place sort. That is, we sort the array in-place. No extra memory is required. The insertion sort is also a stable sort. Stable sorts retain the original ordering of keys when identical keys are present in the input data. Assuming there are n elements in the array, we must index through n - 1 entries. For each entry, we may need to examine and shift up to n - 1 other entries, resulting in a O(n 2 ) algorithm. The insertion sort is an in- place sort. That is, we sort the array in-place. No extra memory is required. The insertion sort is also a stable sort. Stable sorts retain the original ordering of keys when identical keys are present in the input data.

22. 22 Starting near the top of the array in Figure (a), we extract the 3. Then the above elements are shifted down until we find the correct place to insert the 3. This process repeats in Figure (b) with the next number. Finally, in Figure (c), we complete the sort by inserting 2 in the correct place. Starting near the top of the array in Figure (a), we extract the 3. Then the above elements are shifted down until we find the correct place to insert the 3. This process repeats in Figure (b) with the next number. Finally, in Figure (c), we complete the sort by inserting 2 in the correct place.

23. 23 Algorithm Analysis InsertionSort(A) for i=2 to length(A) item = A[i] j = i - 1 while (j > 0 and A[j] > item) A[j+1] = A[j] j = j - 1 A[j+1] = item cost  times executed c 1  n c 2  n-1 c 3  n-1 c 4   t i c 5   (t i -1) c 6   (t i -1) c 7  n-1

24. 24 Algorithm Analysis Observations: Observations:  the constants are “unimportant” details  some terms “grow” faster than others as the size of the input increases. Let’s simplify this... Let’s simplify this...

25. 25 Algorithm Analysis Consider: Consider:  n 2 + 100n + 1000  How does this change as n becomes larger?  Is this a “good” running time? Compare: Compare:  n 2 /3 to 3n  Do the constants really make a difference?

26. 26 Algorithm Analysis Idea: Idea:  comparing algorithms:  upper bounds  lower bounds  tight bounds  we are interested in the growth rate of an algorithm: how its running time changes proportional to the size of the input.

27. 27 Algorithm Analysis: Big Oh A function f(n) is O(g(n)) if there exist constants c, n 0 >0 such that f(n)  c·g(n) for all n  n 0 A function f(n) is O(g(n)) if there exist constants c, n 0 >0 such that f(n)  c·g(n) for all n  n 0 What does this mean in English? What does this mean in English?  c·g(n) is an upper bound of f(n) for large n Examples: Examples:  f(n) = 3n 2 +2n+1 is O(n 2 )  f(n) = 2n is O(n 2 ) (it is also O(n) )  f(n) = 1000n 3 is O(n 10 )

28. 28 Algorithm Analysis: Omega A function f(n) is  (g(n)) if there exist constants c, n 0 >0 such that f(n)  c·g(n) for all n  n 0 A function f(n) is  (g(n)) if there exist constants c, n 0 >0 such that f(n)  c·g(n) for all n  n 0 What does this mean in English? What does this mean in English?  c·g(n) is a lower bound of f(n) for large n Example: Example:  f(n) = n 2 is  (n)

29. 29 Algorithm Analysis: Theta A function f(n) is  (g(n)) if it is both O(g(n)) and  (g(n)), in other words: A function f(n) is  (g(n)) if it is both O(g(n)) and  (g(n)), in other words:  There exist constants c1,c2,n>0 s.t. 0  c 1 g(n)  f(n)  c 2 g(n) for all n  n 0 What does this mean in English? What does this mean in English?  f(n) and g(n) have the same order of growth   indicates a tight bound. Example: Example:  f(n) = 2n+1 is  (n)

30. 30 Algorithm Analysis O, ,  are transitive and reflexive O, ,  are transitive and reflexive  is symmetric  is symmetric f(n) = O(g(n)) iff g(n) =  (f(n)) f(n) = O(g(n)) iff g(n) =  (f(n)) analogy: analogy:  f(n) = O(g(n))  a  b  f(n) =  (g(n))  a  b  f(n) =  (g(n))  a = b  Note: this doesn’t mean that all functions are asymptotically comparable

31. 31 Growth of functions A way to describe behavior of functions in the limit -- asymptotic efficiency. Growth of functions. Focus on what’s important by abstracting away low-order terms and constant factors. How to indicate running times of algorithms? A way to compare “sizes” of functions: O  ≤   ≥   = o 

32. 32

33. 33 Standard notations and common functions Monotonicity: Monotonicity:  f (n) is monotonically increasing if m ≤ n ⇒ f (m) ≤ f (n).  f (n) is monotonically decreasing if m  n ⇒ f (m) ≥ f (n).  f (n) is strictly increasing if m < n ⇒ f (m) < f (n).  f (n) is strictly decreasing if m  n ⇒ f (m) > f (n). Floor and Ceilings: x – 1 <  x   x   x  < x+1 Floor and Ceilings: x – 1 <  x   x   x  < x+1 Modular arithmetic: a mod n = a -  a/n  n Modular arithmetic: a mod n = a -  a/n  n -- refer to the textbook (p36-38). -- refer to the textbook (p36-38). Polynomials: Polynomials: Exponentials: Exponentials: Logarithms: Logarithms: Factorials: Factorials:

34. 34 Algorithm Analysis What if a constant is too large? What if a constant is too large?  Consider 10 10 n as compared to n 2

35. 35 Algorithm analysis Input: Collection of size n Output: The smallest difference between any two different numbers in the collection Algorithm: List all pairs of different numbers in the collection estimate=difference between values in first pair. for each remaining pair compute the difference between the two values if it is less than estimate set estimate to that difference return estimate

36. 36 Algorithm analysis Input: Collection of size n Output: The smallest difference between any two different numbers in the collection Algorithm: estimate = infinity for each value v in the collection for each value u  v if |u-v| < estimate estimate = |u-v| return estimate