1. 1 Chapter (1) – Introduction Objectives Consistency and importance of the performance of a program for inputs of Different sizes. Basic mathematical background. Review Recursion. Review some important features of C++. _____________________________________________________

2. 2 How important is to know which algorithm is better for solving a problem? How do we determine which method algorithm is better? Example (1): Suppose we are given a group on n numbers, how do pick the kth largest? For demonstration purposes let’s use the set of n=10 numbers {12, 34, 5, 20, 11, 32, 14, 8, 17, 13} and find the 4 th largest number among them. Method 1: Sort the numbers and save them in an array, a[ ], of size n: A[0]= 34, A[1]=32, A[2]=20, A[3]=17, A[4]=14, A[5]=13, A[6]=12, A[7]=11, A[8]=8, A[9]=5.

3. 3 Example (1) - Method 2: Step 1: Take the first 4 numbers and sort them in an array of size 4: A[0]=34, A[1]=20, A[2]=12, A[3]=5. Step 2: Read the rest of numbers one by one and compare them with the last number in the array A. If the number is smaller than the last element of the array A, ignore it and get the next number and repeat step 2. If the number is larger than the last member, A then insert it in proper place in the array and take the last element out. Reading 11, 11 > 5, 11 should go after 12, and 5 should be deleted, now the sequence looks like this: A[0]=34, A[1]=20,A[2]=12, A[3]=11.

4. 4 Reading 32, 32>11, 32 should go after 34, and 11 should be deleted, thus we will have: A[0]=34, A[1]=32, A[2]=20, A[3]=12. Reading 14, 14>12, 14 should go after 20, and 12 should be deleted, thus we will have: A[0]=34, A[1]=32, A[2]=20, A[3]=14. Reading 8, 8<14, ignore it and get the next number. Reading 17, 17>14, 17 should go after 20, and 14 should be deleted, thus we will have: A[0]=34, A[1]=32, A[2]=20, A[3]=17. Reading 13, 13<17, ignore it. No more number left, the 4 th largest element is in A[3], i.e.; A[0]=34, A[1]=32, A[2]=20, A[3]=17.

5. 5 Which method do you prefer to use? Both algorithms are simple to implement. But when n is very large and k is also a large number, none of these two algorithms is efficient, i.e., They do not finish in a reasonable amount of time. Example (2): Word puzzle. Find the words in the puzzle. These words may be horizontal, vertical, or diagonal. TDGF4 GHAO3 STAW2 SIHT1 4321

6. 6 Method (1) : Take from the list of words one word at a time and check the strings in the table in all possible combinations (rows, columns, diagonals)to see if that word exists in the table. Method (2): Take all possible combinations of letters from the table (rows, columns, diagonals, number of characters) one by one, and compare the string with the words in the list of words to see if that string exists as a word. Once again, both algorithms are relatively easy to implement, but they both suffer from the same problem as the one mentioned in the previous example. They do not finish in reasonable amount of time.

7. 7 Things to remember before we move on: Writing a working program is not good enough. Writing a program that runs efficiently is the very important. How do we compare the running time of different algorithms without writing the codes for them? In this course we learn some of the techniques: to improve the running time, and to determine the bottlenecks in a code. In the following sections, we review some of the mathematical concepts that we need for this purpose.

8. 8 Mathematical Review: Exponents: Logarithms: Note: In computer science, all logarithms are in base 2, unless specified otherwise. Definition: (1) If and only if (iff)log x b=a. log 10 100=2, i.e. 10 2 =100

9. 9 THEOREM 1.1: PROOF: Let x=log c b, y=log c a, and z=log a b. Using definition (1), we can write: c x = b, c y =a, and a z =b. Using these we can write: a z = (c y ) z = c x or c yz = c x which means : x = yz. By replacing x,y, and z with their logarithmic equivalent we will get : log c b = log c a. log a b. And from this we can find:

10. 10 THEOREM 1.2: log c ab = log c a + log c b PROOF: Let x=log c b, y=log c a, and z=log c ab. Using definition (1), we can write: c x = b, c y =a, and c z =ab. Using these we can write: c z = ab=c y. c x or c z = c y+x which means : z = x+y. By replacing x,y, and z with their logarithmic equivalent we will get: log c ab = log c a + log a b.

11. 11 Can you show that: 1) log c (a/b) = log c a – log c b, 2) log c (a b ) = blog c a, 3) logx 0, and 4) log 1= 0, log 2= 1, log 1,024= 10, log 1,048,576 = 20 ? Hints: Do the same thing as we did for Theorem 1.2. Write a b as a.a.a….a and use Theorem 1.2. Look at the definition of logarithm … Write the numbers in power of 2 and use (2).

12. 12 Series: Formulas to remember: Note: See how a number between 0 and 1 behaves at the higher powers. One can see that if n   then the sum will approach 1/(1- a). These are the “geometric formula”. We can prove the last one: Let S = 1 + a + a 2 + a 3 + … Then aS = a + a 2 + a 3 + …, and S – aS = 1, which implies that:

13. 13 Use the technique in the previous page to compute: We can write: We multiply S by 2 to get: And of course 2S – S = S, thus, And this approaches to 2 for large number of terms, which means that: S=2.

14. 14 Arithmetic Series: Basic formula: Example: 2+5+8+…+(3k-1) = ? This can be written as; (3-1) + (6-1) + (9-1) + … + (3k-1) = (3.1-1)+(3.2-1)+(3.3-1)+…+(3k-1)= (3.1)+(3.2)+(3.3)+…(3.k)-(1+1+1…+1)= 3(1+2+3+…+k) – (1+1+1+…+1)=(3k(k+1)/2)-k = k(3k+1)/2. Or we could do it differently as: 2 3 8 …. (3k-2) (3k-1) + (3k-1) 3k-2 3k-3 …. 3 2 ----------------------------------------------------- 3k+1 3k+1 3k+1 …. 3k+1 3k+1 = k(3k+1) But this is twice as many terms as we need, so we will divide it by 2 to get: k(3k+1)/2.

15. 15 Modular Arithmetic We say that a is congruent to b modulo n, written a  b (mod n), iff n divides a - b. What does this mean ? It means that the remainder is the same when either a or b is divided by n. Example: 81  61  1 (mod 10). Rule : if a  b (mod n), then a + c  b + c (mod n), and ad  bd (mod n). The P Word In data structure analysis are proof by induction and proof by contradiction.

16. 16 Proof by Induction Proof by Induction (PBI) has two standard parts: 1st) providing a base case, that is, establishing that the theorem is true for some small value(s). This step is almost always trivial. 2nd) assuming an inductive hypothesis, That is we assume that the theorem is true for all cases up to some limit k. Using this assumption, the theorem is then shown to be true for the next value, k+1. Note that k is finite.

17. 17 Example (1) : Prove that the Fibonacci series, F 0 = 1, F 1 = 1, F 2 = 2, F 3 = 3, F 4 = 5, …,F i = F i-1 + F i-2, satisfy : F i 1. 1st) Base case: F 1 = 1 < (5/3) and F 2 = 2 < (5/3) 2 = (25/9). 2nd) Inductive hypothesis: We assume that the theorem is true for i = 1,2,…,k. Thus, F k-1 <(5/3) k-1 and F k <(5/3) k are also true. We want to prove that: F k+1 < (5/3) k+1. From the definition of the Fibonacci series we have: F k+1 = F k + F k-1. From the Inductive hypothesis we have: F k+1 = F k-1 + F k < (5/3) k-1 + (5/3) k < (3/5)(5/3 )k+1 + (3/5) 2 (5/3 )k+1 <[(3/5) + (9/25)](5/3) k+1 <(24/25) (5/3) k+1 since (24/25) < 1, then< (5/3) k+1.

18. 18 Proof by Induction Example (2) : THEOREM: PROOF: 1st) Base case: F 1 = 1 2 = [1(1+1)(2.1+1)]/6 = 6/6. 2nd) Inductive hypothesis: We assume that the theorem is true for Thus, is also true. We will prove that:

19. 19 We can write the sum as:

20. 20 Proof by Counterexample In this case we bring an example that is against the assumption. Example: Prove that F k > k 2 is false. The easiest way to prove this is to bring a counter example. F 11 = 144 > 11 2.

21. 21 Proof by Contradiction In this case we assume that the theorem is false and then we show that this assumption implies that some known property(ies) is false, hence the original assumption was erroneous. Example: Proof that there is an infinite number of primes. 1st) We assume that theorem is false, so that there is some largest prime p k. Now we write the serious of prime numbers from p 1 to p k in order as; p 1, p 2, p 3, …, p k. Now consider a number N = p 1 p 2 p 3 …p k + 1, Clearly N is larger than p k so by assumption N is not a prime. However, none of p 1, p 2, p 3, …, p k divide N exactly, because there is always a remainder of 1. This is a contradiction, since every number is either a prime or a product of primes. Thus, the assumption that p k is the largest prime is false, which implies that the theorem is true.

22. 22 Introduction to Recursion Most mathematical functions are described in simple formula and can be easily translated to their C++ equivalents. Example: Formula for converting degree Fahrenheit to Celsius: C = 5(F-32)/9. In some cases, a function is defined in terms of itself: Example: f(x) = 2f(x-1) + x 2. Looking at this statement, it is evident that we cannot start our calculations of f(x) unless we know f(x-1). Also, we need to know the value of f at some starting point to find the f at the next point. Assume that f(0) = 0, then we can compute: f(1) = 2f(0) + 1 2 = 1, f(2) = 2f(1) + 2 2 = 6, f(3) = 2f(2) + 3 2 = 21, ….

23. 23 /* 1 */ int /* 2 */F( const int X) /* 3 */{ /* 4 */if( X = 0 ) /* 5 */return 0; /* 6 */else /* 7 */return 2 * F( X - 1) + X * X; /* 8 */}

24. 24 Common Errors in Recursive Calls There are two common errors that usually are made: 1) The base is defined such that it will not be reached. Example: If we want to compute f(-7) in the previous example we never get to the base that is defined at x = 0, i.e, we never get to f(0). Here is why; f(-7) = 2*f(-8) + (-7) 2. As you can see to compute f(-7) we need f(-8) and to find f(-8) we will need f(-9), and so on. We will never use the base f(0) = 0 and the answer will never be determined.

25. 25 2) The base in not defined. Example: /* 1 */int /* 2 */Bad( const int N ) /* 3 */{ /* 4 */if( N == 0) /* 5 */return 0; /* 6 */else /* 7 */return Bad( N/3 + 1) + N -1; /* 8 */} We make call to Bad(1) but Bad(0) will never be reached. We get stock in Bad(1). Thus the program never resolves.

26. 26 Example: Print out integer numbers assuming the I/O routine takes single- digit numbers and output it to the terminal. For example for 76234 it takes 4, leave 7623 out. Then takes 3, leave 762 out. It continues until it gets to 7 which will the base case. /* 1 */void /* 2 */Print_Out( const unsigned int N ) /* 3 */{ /* 4 */if ( N < 10 ) /* 5 */Print_Digit( N ); /* 6 */else /* 7 */{ /* 8 */Print_Out( N/10 ); /* 9 */Print_Digit( N%10); /* 10 */} /* 11 */}

27. 27 We wanted to print a number 76234. We need to print: 7623 then 4 To print 4 we use Print_Digit(N%10). To print 7623 we use Print_Out(N/10). This will work fine if the base is defined properly. To stop the program we just need to distinguish the numbers less than 10. Using induction can you prove that the above program works?

28. 28 Four Basic Rules to Remember When Writing a Recursive Program: Base Cases : You must always have some base cases, which can be solved without recursion. You have to tell the program to stop somewhere. Making Progress: For the cases that are to be solved recursively, the recursive call must always be to a case that makes progress toward a base case. The program should continuously obtain new values. Design Rule: Assume that all the recursive calls work. All recursive calls must work. Compound Interest Rule: Never duplicate work by solving the same instance of a problem in separate recursive calls. Do not use recursion to evaluate simple mathematical functions.

29. 29 C++ at a Glance C++ is a strongly typed language. Thus, every object must belong to a type. An object of one type cannot assigned to or compared with one another type except by a well-defined type conversion. Call by reference is supported in C++. This removes some of the difficult pointer code. A direct result is that any large structure should be passed by reference. The compiler directive const can be used in many places to imply that some object is unalterable. Large structures can safely be passed as const and by reference, rather than using a pointer,

30. 30 Integers that are passed by value can be specified as cosnt to prevent accidental modification. The use of #define for symbolic constants is considered bad practice in C++. Instead const is used for the same purpose. Preprocessor macros are obsolete in C++, because inline functions are supported. I/O has been revamped in C++ to use streams. Thus, it is not a good idea to use printf and scanf. Malloc and free have been replaced by new and delete. These are syntactic improvements to C.

31. 31 ClassesOverloadingTemplatesInheritance Example: A class that supports string operations. In C: we use strcmp and strcpy defined in to do string operations. These operate on pointers to characters. These routines are available in C++. Limitations: Copying and comparing strings looks different from copying and comparing integers. Thus, “=“ and “= =“ means different. If applied on strings, it results to errors. This can be fixed by overloading the comparison and assignment operators. Storage issue. When a string is dynamically allocated the user has to remember to free the memory when the string is no longer needed. Error handling. If the target of strcpy is not pointing to a sufficiently large array, disaster will happen. An array index out of bounds will corrupt another variable.

32. 32 Example: Lack of safety. char *S = strdup(“Hello”); We can modify this S currently holding word “Hello”: S[0] = ‘J’; Or in a very different way; char *T = S; T[0] = ‘M’; This alters S without S knowing about it. How ?

33. 33 #include /** * A class for simulating an integer memory cell. */ class IntCell { public: /** * Construct the IntCell. * Initial value is 0. */ IntCell( ) { storedValue = 0; } /** * Construct the IntCell. * Initial value is initialValue. */ IntCell( int initialValue ) { storedValue = initialValue; } /** * Return the stored value. */ int read( ) { return storedValue; } /** * Change the stored value to x. */ void write( int x ) { storedValue = x; } private: int storedValue; }; int main( ) { IntCell m; m.write( 5 ); cout << "Cell contents: " << m.read( ) << endl; return 0; } 1.5

34. 34 #include /** * A class for simulating an integer memory cell. */ class IntCell { public: /* 1*/ explicit IntCell( int initialValue = 0 ) /* 2*/ : storedValue( initialValue ) { } /* 3*/ int read( ) const /* 4*/ { return storedValue; } /* 5*/ void write( int x ) /* 6*/ { storedValue = x; } private: /* 7*/ int storedValue; }; int main( ) { IntCell m; m.write( 5 ); cout << "Cell contents: " << m.read( ) << endl; return 0; } 1.6

35. 35 #ifndef IntCell_H_ #define IntCell_H_ /** * A class for simulating an integer memory cell. */ class IntCell { public: explicit IntCell( int initialValue = 0 ); int read( ) const; void write( int x ); private: int storedValue; }; #endif IntCell.h

36. 36 #include "IntCell.h" /** * Construct the IntCell with initialValue */ IntCell::IntCell( int initialValue ) : storedValue( initialValue ) { } /** * Return the stored value. */ int IntCell::read( ) const { return storedValue; } /** * Store x. */ void IntCell::write( int x ) { storedValue = x; } IntCell.cpp

37. 37 #include "IntCell.h" #include using namespace std; int main() { IntCell m; //Or, IntCell m (0); but not IntCell m(); m.write( 5 ); cout << "Cell contents: " << m.read() << endl; return 0; } IntCellMain.cpp

38. 38 Pointers #include #include "IntCell.h" int main( ) { /* 1*/ IntCell *m; /* 2*/ m = new IntCell( 0 ); /* 3*/ m->write( 5 ); /* 4*/ cout read( ) << endl; /* 5*/ delete m; /* 6*/ return 0; } IntCellMain.cpp

39. 39 Parameter Passing Call by value – void MyFunction(int x) Call by constant reference – void MyFunction(const int x) Call by reference – void MyFunction(int& x) void MyFunction(int x, int& y, const int z); int main( ) { int x = 8, y = 20, z = 100; MyFunction(x, y, z); cout << “x = “ << x << “, y = “ << y << “, z = “ << z << endl; return 0; } void MyFunction(int x, int& y, const int z) { x = y = z; } What would be the output of this program?

40. 40 Return Passing Find the maximum string const string & findMax( const vector & a) { int maxIndex = 0; for( int i = 1; i < a.size( ); i++) if( a[maxIndex] < a [i] ) maxIndex = i; return a[maxIndex]; } const string & findMax( const vector & a) { string maxValue = a[0]; for( int i = 1; i < a.size( ); i++) if( maxValue < a [i] ) maxValue = a[i]; return maxValue; } Correct/Incorrect?

41. 41 Destructor, Copy Constructor, and Operator = IntCell::~IntCell( ) { // Does nothing } IntCell::IntCell(const IntCell & rhs) : storedValue( rhs.storedValue) { } const IntCell & IntCell::operator =(const IntCell & rhs) { /* 1 */ if( this != &rhs) //standard alias test /* 2 */storedValue = rhs.storedValue; /* 3 */return *this; }

42. 42 #include class IntCell { public: explicit IntCell( int initialValue = 0 ) { storedValue = new int( initialValue ); } int read( ) const { return *storedValue; } void write( int x ) { *storedValue = x; } private: int *storedValue; }; int f( ) { IntCell a( 2 ); IntCell b = a; IntCell c; c = b; a.write( 4 ); cout << a.read( ) << endl << b.read( ) << endl << c.read( ) << endl; return 0; } int main( ) { f( ); return 0; } When default do not work Output: 4 What is the problem?

43. 43 #include class IntCell { public: explicit IntCell( int initialValue = 0 ); IntCell( const IntCell & rhs ); ~IntCell( ); const IntCell & operator=( const IntCell & rhs ); int read( ) const; void write( int x ); private: int *storedValue; }; IntCell::IntCell( int initialValue ) { storedValue = new int( initialValue ); } IntCell::IntCell( const IntCell & rhs ) { storedValue = new int( *rhs.storedValue ); }

44. 44 IntCell::~IntCell( ) { delete storedValue; } const IntCell & IntCell::operator=( const IntCell & rhs ) { if( this != &rhs ) *storedValue = *rhs.storedValue; return *this; } int IntCell::read( ) const { return *storedValue; } void IntCell::write( int x ) { *storedValue = x; }

45. 45 int f( ) { IntCell a( 2 ); IntCell b = a; IntCell c; c = b; a.write( 4 ); cout << a.read( ) << endl << b.read( ) << endl << c.read( ) << endl; return 0; } int main( ) { f( ); return 0; }

46. 46 Templates /** * Return the maximum item in array a. * Assumes a.size( ) > 0. * Comparable objects must provide * copy constructor, operator<, operator= */ template const Comparable & findMax( const vector & a ) { /* 1*/ int maxIndex = 0; /* 2*/ for( int i = 1; i < a.size( ); i++ ) /* 3*/ if( a[ maxIndex ] < a[ i ] ) /* 4*/ maxIndex = i; /* 5*/ return a[ maxIndex ]; }

47. 47 int main( ) { vector v1( 37 ); vector v2( 40 ); vector v3( 80 ); // vector v4( 75 ); // Additional code to fill in the vectors cout << findMax( v1 ) << endl; // OK: Comparable = int cout << findMax( v2 ) << endl; // OK: Comparable = double cout << findMax( v3 ) << endl; // OK: Comparable = string // cout << findMax( v4 ) << endl; // Illegal; operator< undefined return 0; }; Programs: g++ findMax.cpp IntCell.cpp vector.cpp string.cpp

48. 48 Class Templates /** * A class for simulating a memory cell. */ template class MemoryCell { public: explicit MemoryCell( const Object & initialValue = Object( ) ) : storedValue( initialValue ) { } const Object & read( ) const { return storedValue; } void write( const Object & x ) { storedValue = x; } private: Object storedValue; }; MemoryCell template class

49. 49 int main( ) { MemoryCell m1; MemoryCell m2( "hello" ); m1.write( 37 ); m2.write( m2.read( ) + " world" ); cout << m1.read( ) << endl << m2.read( ) << endl; return 0; } Program that uses MemoryCell template class Program: g++ Fig01_19.cpp string.cpp

50. 50 /** * Return the maximum item in array a. * Assumes a.size( ) > 0. * Comparable objects must provide * copy constructor, operator<, operator= */ template const Comparable & findMax( const vector & a ) { /* 1*/ int maxIndex = 0; /* 2*/ for( int i = 1; i < a.size( ); i++ ) /* 3*/ if( a[ maxIndex ] < a[ i ] ) /* 4*/ maxIndex = i; /* 5*/ return a[ maxIndex ]; } Object and Comparable

51. 51 class Employee { public: void setValue( const string & n, double s ) { name = n; salary = s; } void print( ostream & out ) const { out << name << " (" << salary << ")"; } bool operator< ( const Employee & rhs ) const { return salary < rhs.salary; } // Other general accessors and mutators, not shown private: string name; double salary; }; // Define an output operator for Employee ostream & operator<< ( ostream & out, const Employee & rhs ) { rhs.print( out ); return out; }

52. 52 int main( ) { vector v( 3 ); v[0].setValue( "Bill Clinton", 200000.00 ); v[1].setValue( "Bill Gates", 2000000000.00 ); v[2].setValue( "Billy the Marlin", 60000.00 ); cout << findMax( v ) << endl; return 0; }

53. 53 Matrix Class template class matrix { public: matrix( int rows, int cols ) : array( rows ) { for( int i = 0; i < rows; i++ ) array[ i ].resize( cols ); } matrix( const matrix & rhs ) : array( rhs.array ) { } const vector & operator[]( int row ) const { return array[ row ]; } vector & operator[]( int row ) { return array[ row ]; } int numrows( ) const { return array.size( ); } int numcols( ) const { return numrows( ) ? array[ 0 ].size( ) : 0; } private: vector > array; };