Data Structure Assistant Prof. Chih-Chia Yao（ C. –C. Yao.

Data Structure Assistant Prof. Chih-Chia Yao（ C. –C. Yao

Text & Reference books (Textbook)：
Ellis Horowitz, Sartaj Sahni, and Dinesh P. Mehta, “Fundamentals of Data Structures in C++”, 2nd Ed., Silicon Press, 2007. C. –C. Yao

Algorithm Definition: An algorithm is a finite set of instructions that, if followed, accomplishes a particular task. In addition, all algorithms must satisfy the following criteria: Input. Zero more quantities are externally supplied. Output. At least one quantity is produced. Definiteness. Each instruction is clear and unambiguous. Finiteness. If we trace out the instructions of an algorithm, then for all cases, the algorithm terminates after a finite number of steps. Effectiveness. Every instruction must be basic enough to be carried out, in principle, by a person using only pencil and paper. It is not enough that each operation be definite as in 3) it also must be feasible. C. –C. Yao

Example: Selection Sort
Suppose we must devise a program that sorts a collection of n ≥ 1 integers. From those integers that are currently unsorted, find the smallest and place it next in the sorted list. Problem in the above statement Does not describe where and how the integers are initially sorted. Does not indicate where to place the result. C. –C. Yao

C++ Program for Selection Sort
void sort (int *a, const int n) // sort the n integers a[0] to a[n-1] into non-decreasing order for (int i = 0; i < n; i++) { int j = i; // find smallest integer in a[i] to a[n-1] for (int k = i + 1; k < n; k++) if (a[k] < a[j]) j = k; // interchange int temp = a[i]; a[i] = a[j]; a[j] = temp; } C. –C. Yao

Selection Sort (Cont.) Theorem 1.1: sort(a, n) correctly sorts a set of n ≥ 1 integers; the result remains in a[0] … a[n-1] such that a[0] ≤ a[1] ≤ … ≤ a[n–1]. C. –C. Yao

Example:Binary Search
Assume that we have n ≥ 1 distinct integers that are already sorted and stored in the array a[0] … a[n-1]. Our task is to determine if the integer x is present and if so to return j such that x = a[j]; otherwise return -1. By making use of the fact that the set is sorted, we conceive the following efficient method: Let left and right, respectively, denote the left and right ends of the list to be searched. Initially, left = 0 and right = n – 1. Let middle = (left + right) / 2 be the middle position in the list. If we compare a[middle] with x, we obtain one of the three results: (1) x < a[middle]. In this case, if x is present, it must be in the positions between 0 and middle – 1. Therefore, we set right to middle – 1. (2) x == a[middle]. In this case, we return middle. (3) x > a[middle]. In this case, if x is present, it must be in the positions between middle+1 and n-1. So, we set left to middle+1. C. –C. Yao

Algorithm for Binary Search
int BinarySearch (int *a, const int x, const int n) // Search the sorted array a[0], … , a[n-1] for x { for (initialize left and right; while there are more elements;) let middle be the middle element; switch (compare (x, a[middle])) { case ‘>’: set left to middle+1; break; case ‘<‘: set right to middle -1; break; case ‘=‘: found x; } // end of switch } // end of for not found; } // end of BinarySearch C. –C. Yao

C++ Program for Binary Search
char compare (int x, int y) { if (x > y) return ‘>’; else if ( x < y) return ‘<‘; else return ‘=‘; } // end of compare C. –C. Yao

Algorithm for Binary Search (Cont.)
int BinarySearch (int *a, const int x, const int n) // Search the sorted array a[0], … , a[n-1] for x { for (int left = 0, right = n - 1; left <= right;) int middle = (left + right) / 2; switch (compare (x, a[middle])) { case ‘>’: left = middle+1; break; // x > a[middle] case ‘<‘: right = middle -1; break; // x < a[middle] case ‘=‘: return middle; // x == a[middle] } // end of switch } // end of for return -1; } // end of BinarySearch C. –C. Yao

Recursive Algorithms int BinarySearch (int *a, const int x, const int left, const int right) { if (left <= right) int middle = (left + right) / 2; if (x < a[middle]) return BinarySearch(a,x,left,middle-1); else if (x < a[middle]) return BinarySearch(a,x,left,middle-1); return middle; } // end if return -1; } // end of BinarySearch C. –C. Yao

Recursive Algorithms(cont.)
Recursive program: int main() { int n=10; printf(“%d”, rfib(n)); } int rfib(int n) if (n==1 || n==2) return 1; return rfib(n1)+rfib(n2); C. –C. Yao

Performance Analysis Space Complexity: The space complexity of a program is the amount of memory it needs to run to completion. Time Complexity: The time complexity of a program is the amount of computer time it needs to run to completion. C. –C. Yao

Space Complexity A fixed part that is independent of the characteristics of the inputs and outputs. This part typically includes the instruction space, space for simple varialbes and fixed-size component variables, space for constants, etc. A variable part that consists of the space needed by component variables whose size is dependent on the particular problem instance being solved, the space needed by referenced variables, and the recursion stack space. The space requirement S(P) of any program P is written as S(P) = c +Sp where c is a constant C. –C. Yao

Time Complexity The time, T(P), taken by a program P is the sum of the compile time and the run (or execution) time. The compile time does not depend on the instance characteristics. We focus on the run time of a program, denoted by tp (instance characteristics). C. –C. Yao

Time Complexity in C++ General statements in a C++ program Step count
Comments Declarative statements 0 Expressions and assignment statements 1 Iteration statements N Switch statement N If-else statement N Function invocation 1 or N Memory management statements 1 or N Function statements 0 Jump statements 1 or N C. –C. Yao

Time Complexity (Cont.)
Note that a step count does not necessarily reflect the complexity of the statement. Step per execution (s/e): The s/e of a statement is the amount by which count changes as a result of the execution of that statement. C. –C. Yao

Time Complexity Iterative Example
float sum (float *a, const int n) { float s = 0; for (int i = 0; i < n; i++) s += a[i]; return; } C. –C. Yao

Step Count of Iterative Example
float sum (float *a, const int n) { float s = 0; count++; // count is global for (int i = 0; i < n; i++) count++; // for for s += a[i]; count++; // for assignment } count++; // for last time of for count++; // for return return; C. –C. Yao

Step Count of Iterative Example (Simplified)
void sum (float *a, const int n) { for (int i = 0; i < n; i++) count += 2; count +=3; } If initially count = 0, then the total of steps is 2n + 3. C. –C. Yao

Time Complexity of Recursive Example
float rsum (float *a, const int n) { if (n <= 0) return 0; else return (rsum(a, n–1) + a[n-1]); } C. –C. Yao

Step Count of Recursive Example
float rsum (float *a, const int n) { count++; // for if conditional if (n <= 0) { count++; // for return return 0; } else { return (rsum(a, n–1) + a[n-1]); Assume trsum(0) = 2 trsum(n) = 2 + trsum(n-1) = trsum(n-2) = 2*2 + trsum(n-2) = 2n + trsum(0) = 2n + 2 C. –C. Yao

Matrix Addition Example
line void add (matrix a, matrix b, matrix c, int m, int n) 1 { 2 for (int i = 0; i < m; i++) for (int j = 0; j < n; j++) 4 c[i][j] = a[i][j] + b[i][j]; 5 } C. –C. Yao

Step Count of Matrix Addition Example
void add (matrix a, matrix b, matrix c, int m, int n) { for (int i = 0; i < m; i++) count++; // for for i for (int j = 0; j < n; j++) count++; // for for j c[i][j] = a[i][j] + b[i][j]; count++; // for assigment } count++; // for last time of for j count++; // for last time of for i C. –C. Yao

Step Count of Matrix Addition Example (Simplified)
line void add (matrix a, matrix b, matrix c, int m, int n) { for (int i = 0; i < m; i++) for (int j = 0; j < n; j++) count += 2; count+2; } count++; C. –C. Yao

Step Table of Matrix Addition Example
Line s/e Frequency Total steps 1 2 m+1 3 m(n+1) mn+m 4 mn 5 Total number of steps 2mn+2m+1 C. –C. Yao

Fibonacci Numbers The Fibonacci sequence of numbers starts as 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, … Each new term is obtained by taking the sum of the two previous terms. If we call the first term of the sequence F0 then F0 = 0, F1 = 1, and in general Fn = Fn-1 + Fn-2 , n ≥ 2. C. –C. Yao

C++ Program of Fibonacci Numbers
1 void fibonacci (int n) 2 // compute the Fibonacci number Fn 3 { 4 if (n <= 1) cout << n << endl; // F0 = 0, and F1 = 1 5 else { // compute Fn 6 int fn; int fnm2 = 0; int fnm1 = 1; 7 for (int i = 2; i <= n; i++) 8 { fn = fnm1 + fnm2; fnm2 = fnm1; fnm1 = fn; } // end of for cout << fn << endl; 14 } // end of else 15 } // end of fibonacci C. –C. Yao

Step Count of Fibonacci Program
Two cases: n = 0 or n = 1 Line 4 regarded as two lines: 4(a) and 4(b), total step count in this case is 2 n > 1 Line 4(a), 6, and 13 are each executed once Line 7 gets executed n times. Lines 8 – 12 get executed n-1 times each. Line 6 has s/e of 2 and the remaining lines have an s/e of 1. Total steps for the case n > 1 is 4n + 1 C. –C. Yao

Asymptotic Notation Determining step counts help us to compare the time complexities of two programs and to predict the growth in run time as the instance characteristics change. But determining exact step counts could be very difficult. Since the notion of a step count is itself inexact, it may be worth the effort to compute the exact step counts. Definition [Big “oh”]: f(n) = O(g(n)) iff there exist positive constants c and n0 such that f(n) ≤ cg(n) for all n, n ≥ n0 C. –C. Yao

Examples of Asymptotic Notation
3n + 2 = O(n) 3n + 2 ≤ 4n for all n ≥ 3 100n + 6 = O(n) 100n + 6 ≤ 101n for all n ≥ 10 10n2 + 4n + 2 = O(n2) 10n2 + 4n + 2 ≤ 11n2 for all n ≥ 5 C. –C. Yao

Asymptotic Notation (Cont.)
Theorem 1.2: If f(n) = amnm + … + a1n + a0, then f(n) = O(nm). Proof: for n ≥ 1 So, f(n) = O(nm) C. –C. Yao

Definition: [Omega] f(n) = Ω(g(n)) iff there exist positive constants c and n0 such that f(n) ≥ cg(n) for all n, n ≥ n0. Example: 3n + 2 = Ω(n) 100n + 6 = Ω(n) 10n2 + 4n + 2 =Ω(n2) C. –C. Yao

Definition: f(n) = Θ(g(n)) iff there exist positive constants c1, c2, and n0 such that c1g(n) ≤ f(n) ≤ c2g(n) for all n, n ≥ n0. C. –C. Yao

Theorem 1.3: If f(n) = amnm + … + a1n + a0 and am > 0, then f(n) = Ω(nm). Theorem 1.4: If f(n) = amnm + … + a1n + a0 and am > 0, then f(n) = Θ(nm). C. –C. Yao

Practical Complexities
If a program P has complexities Θ(n) and program Q has complexities Θ(n2), then, in general, we can assume program P is faster than Q for a sufficient large n. However, caution needs to be used on the assertion of “sufficiently large”. C. –C. Yao

Function Values log n n n log n n2 n3 2n 1 2 4 8 16 64 3 24 512 256
1 2 4 8 16 64 3 24 512 256 4096 65536 5 32 160 1024 32768 C. –C. Yao

Chap 2 Array (陣列)

ADT In C++, class consists four components: class name
data members : the data that makes up the class member functions : the set of operations that may be applied to the objects of class levels of program access : public, protected and private.

Definition of Rectangle
#ifndef RECTANGLE_H #define RECTANGLE_H Class Rectangle{ Public: Rectangle(); ~Rectangle(); int GetHeight(); int GetWidth(); Private: int xLow, yLow, height, width; }; #endif

Array char A[3][4]; // row-major logical structure physical structure
1 2 3 A[0][0] A[0][1] 1 A[0][2] 2 A[0][3] A[1][0] A[2][1] A[1][1] A[1][2] A[1][3] Mapping: A[i][j]=A[0][0]+i*4+j

Array The Operations on 1-dim Array Store 將新值寫入陣列中某個位置 A[3] = 45 O(1)
1 2 3 A 45 ...

Polynomial Representations
private: int degree; // degree ≤ MaxDegree float coef [MaxDegree + 1]; Representation 2 int degree; float *coef; Polynomial::Polynomial(int d) { degree = d; coef = new float [degree+1]; } Representation 3 class Polynomial; // forward delcaration class term { friend Polynomial; float coef; // coefficient int exp; // exponent }; static term termArray[MaxTerms]; static int free; int Start, Finish; term Polynomial:: termArray[MaxTerms]; Int Polynomial::free = 0; // location of next free location in temArray

Figure 2.1 Array Representation of two Polynomials
Represent the following two polynomials: A(x) = 2x B(x) = x4 + 10x3 + 3x2 + 1 A.Start A.Finish B.Start B.Finish free coef 2 1 10 3 exp 1000 4 5 6

Polynomial Addition 只存非零值(non-zero)：一元多項式
Add the following two polynomials: A(x) = 2x B(x) = x4 + 10x3 + 3x2 + 1 1 2 1 2 3 4 A_coef 2 2 1 B_coef 4 1 10 3 1 A_exp 1000 B_exp 4 3 2 1 2 3 4 5 C_coef 5 2 1000 1 4 10 3 3 2 2 C_exp

Polynomial Addition O(m+n) Polynomial Polynomial:: Add(Polynomial B)
// return the sum of A(x) ( in *this) and B(x) { Polynomial C; int a = Start; int b = B.Start; C.Start = free; float c; while ((a <= Finish) && (b <= B.Finish)) switch (compare(termArray[a].exp, termArray[b].exp)) { case ‘=‘: c = termArray[a].coef +termArray[b].coef; if ( c ) NewTerm(c, termArray[a].exp); a++; b++; break; case ‘<‘: NewTerm(termArray[b].coef, termArray[b].exp); b++; case ‘>’: NewTerm(termArray[a].coef, termArray[a].exp); a++; } // end of switch and while // add in remaining terms of A(x) for (; a<= Finish; a++) // add in remaining terms of B(x) for (; b<= B.Finish; b++) C.Finish = free – 1; return C; } // end of Add O(m+n)

Program 2.9 Adding a new Term
void Polynomial::NewTerm(float c, int e) // Add a new term to C(x) { if (free >= MaxTerms) { cerr << “Too many terms in polynomials”<< endl; exit(); } termArray[free].coef = c; termArray[free].exp = e; free++; } // end of NewTerm

Disadvantages of Representing Polynomials by Arrays
What should we do when free is going to exceed MaxTerms? Either quit or reused the space of unused polynomials. But costly. If use a single array of terms for each polynomial, it may alleviate the above issue but it penalizes the performance of the program due to the need of knowing the size of a polynomial beforehand.

Sparse Matrices

Sparse Matrix Representation
Use triple <row, column, value> Store triples row by row For all triples within a row, their column indices are in ascending order. Must know the number of rows and columns and the number of nonzero elements

Sparse Matrix Representation (Cont.)
class SparseMatrix; // forward declaration class MatrixTerm { friend class SparseMatrix private: int row, col, value; }; In class SparseMatrix: int Rows, Cols, Terms; MatrixTerm smArray[MaxTerms];

Transposing A Matrix Intuitive way: More efficient way:
for (each row i) take element (i, j, value) and store it in (j, i, value) of the transpose More efficient way: for (all elements in column j) place element (i, j, value) in position (j, i, value)

Transposing A Matrix The Operations on 2-dim Array Transpose
for (i = 0; i < rows; i++ ) for (j = 0; j < columns; j++ ) B[j][i] = A[i][j];

Program 2.10 Transposing a Matrix
SparseMatrix SparseMatrix::Transpose() // return the transpose of a (*this) { SparseMatrix b; b.Rows = Cols; // rows in b = columns in a b.Cols = Rows; // columns in b = rows in a b.Terms = Terms; // terms in b = terms in a if (Terms > 0) // nonzero matrix int CurrentB = 0; for (int c = 0; c < Cols; c++) // transpose by columns for (int i = 0; i < Terms; i++) // find elements in column c if (smArray[i].col == c) { b.smArray[CurrentB].row = c; b.smArray[CurrentB].col = smArray[i].row; b.smArray[CurrentB].value = smArray[i].value; CurrentB++; } } // end of if (Terms > 0) } // end of transpose O(terms*columns)

Fast Matrix Transpose The O(terms*columns) time => O(rows*columns2) when terms is the order of rows*columns A better transpose function in Program It first computes how many terms in each columns of matrix a before transposing to matrix b. Then it determines where is the starting point of each row for matrix b. Finally it moves each term from a to b.

Program 2.11 Fast Matrix Transposing
SparseMatrix SparseMatrix::Transpose() // The transpose of a(*this) is placed in b and is found in Q(terms + columns) time. { int *Rows = new int[Cols]; int *RowStart = new int[Rows]; SparseMatrix b; b.Rows = Cols; b.Cols = Rows; b.Terms = Terms; if (Terms > 0) // nonzero matrix // compute RowSize[i] = number of terms in row i of b for (int i = 0; I < Cols; i++) RowSize[i] = 0; // Initialize for ( I = 0; I < Terms; I++) RowSize[smArray[i].col]++; // RowStart[i] = starting position of row i in b RowStart[0] = 0; for (i = 0; i < Cols; i++) RowStart[i] = RowStart[i-1] + RowSize[i-1]; O(columns) O(terms) O(columns-1)

Program 2.11 Fast Matrix Transposing (Cont.)
for (i = 1; I < Terms; i++) // move from a to b { int j = RowStart[smArray[i].col]; b.smArray[j].row = smArray[i].col; b.smArray[j].col = smArray[i].row; b.smArray[j].value = smArray[i].value; RowStart[smArray[i].col]++; } // end of for } // end of if delete [] RowSize; delete [] RowStart; return b; } // end of FastTranspose O(terms) O(row * column)

Matrix Multiplication
Definition: Given A and B, where A is mxn and B is nxp, the product matrix Result has dimension mxp. Its [i][j] element is for 0 ≤ i < m and 0 ≤ j < p.

Representation of Arrays
Multidimensional arrays are usually implemented by one dimensional array via either row major order or column major order. Example: One dimensional array α α+1 α+2 α+3 α+4 α+5 α+6 α+7 α+8 α+9 α+10 α+12 A[0] A[1] A[2] A[3] A[4] A[5] A[6] A[7] A[8] A[9] A[10] A[11]

Two Dimensional Array Row Major Order
Col 0 Col 1 Col 2 Col u2 - 1 Row 0 X X X X Row 1 X X X X Row u1 - 1 X X X X u2 elements u2 elements Row 0 Row 1 Row i Row u1 - 1 i * u2 element

Memory mapping char A[3][4]; // row-major
logical structure physical structure 1 2 3 A[0][0] A[0][1] 1 A[0][2] 2 A[0][3] A[1][0] A[2][1] A[1][1] A[1][2] A[1][3] Mapping: A[i][j]=A[0][0]+i*4+j

The address of elements in 2-dim array
二維陣列宣告為A[r][c]，每個陣列元素佔len bytes，且其元第1個元素A[0][0]的記憶體位址為，並以列為主(row major)來儲存此陣列，則 A[0][3]位址為+3*len A[3][0]位址為+(3*r+0)*len ... A[m][n]位址為+(m* r + n)*len A[0][0]位址為，目標元素A[m][n]位址的算法 Loc(A[m][n]) = Loc(A[0][0]) + [(m0)*r+(n0)]*元素大小

String Usually string is represented as a character array.
General string operations include comparison, string concatenation, copy, insertion, string matching, printing, etc. H e l l o W o r l d \0

String Matching The Knuth-Morris-Pratt Algorithm
Definition: If p = p0p1…pn-1 is a pattern, then its failure function, f, is defined as If a partial match is found such that si-j … si-1 = p0p1…pj-1 and si ≠ pj then matching may be resumed by comparing si and pf(j–1)+1 if j ≠ 0. If j = 0, then we may continue by comparing si+1 and p0.

Fast Matching Example Suppose exists a string s and a pattern pat = ‘abcabcacab’, let’s try to match pattern pat in string s. j pat a b c a b c a c a b f s = ‘- a b c a ? ? ?’ pat = ‘a b c a b c a c a b’ ‘a b c a b c a c a b’ j = 4, pf(j-1)+1 = p1 New start matching point

Chap 3 Stack and Queue

Templates in C++ Template function in C++ makes it easier to reuse classes and functions. A template can be viewed as a variable that can be instantiated to any data type, irrespective of whether this data type is a fundamental C++ type or a user-defined type.

Selection Sort Template
Template <class KeyType> void sort(KeyType *a, int n) // sort the n KeyTypes a[0] to a[n-1] into nondecreasing order { for (int i = 0; i < n; i++) int j = i; // find smallest KeyType in a[i] to a[n-1] for (int k = i+1; k < n; k++) if (a[k] < a[j]) { j = k;} // interchange KeyType temp = a[i]; a[i] = a[j]; a[j] = temp; } float farray[100]; int intarray[200]; ……….. sort(farray, 100); sort(intarray, 200);

Template (Cont.) Can we use the sort template for the Rectangle class?
Well, not directly. We’ll need to use operator overloading to implement “>” for Rectangle class.

Stack What is a stack? A stack is an ordered list in which insertions and deletions are made at one end called the top. It is also called a Last-In-First-Out (LIFO) list.

Stack (Cont.) Given a stack S = (a0, …, an-1), a0 is the bottom element, an-1 is the top element, and ai is on top of element ai-1, 0 < i < n. a3 a3 a2 a2 a1 a1 a0 a0 Push (Add) Pop (Delete)

System Stack Whenever a function is invoked, the program creates a structure, referred to as an activation record or a stack frame, and places it on top of the system stack. previous frame pointer fp return address a1 local variables previous frame pointer fp previous frame pointer return address main return address main

ADT 3.1 Abstract Data Type Stack
Template <class KeyType> class Stack { // objects: A finite ordered list with zero or more elements public: Stack (int MaxStackSize = DefaultSize); // Create an empty stack whose maximum size is MaxStackSize Boolean IsFull(); // if number of elements in the stack is equal to the maximum size // of the stack, return TRUE(1) else return FALSE(0) void Add(const KeyType& item); // if IsFull(), then StackFull(); else insert item into the top of the stack. Boolean IsEmpty(); // if number of elements in the stack is 0, return TRUE(1) else return FALSE(0) KeyType* Delete(KeyType& ); // if IsEmpty(), then StackEmpty() and return 0; // else remove and return a pointer to the top element of the stack. };

Implementation of Stack by Array
an-1 a2 a1 a0 a0 a1 a2 an-1 Array index 1 2 3 n-1

Queue A queue is an ordered list in which all insertions take place at one end and all deletions take place at the opposite end. It is also known as First-In-First-Out (FIFO) lists. a0 a1 a2 an-1 front rear

ADT 3.2 Abstract Data Type Queue
Template <class KeyType> class Queue { // objects: A finite ordered list with zero or more elements public: Queue(int MaxQueueSize = DefaultSize); // Create an empty queue whose maximum size is MaxQueueSize Boolean IsFull(); // if number of elements in the queue is equal to the maximum size of // the queue, return TRUE(1); otherwise, return FALSE(0) void Add(const KeyType& item); // if IsFull(), then QueueFull(); else insert item at rear of the queue Boolean IsEmpty(); // if number of elements in the queue is equal to 0, return TRUE(1) // else return FALSE(0) KeyType* Delete(KeyType&); // if IsEmpty(), then QueueEmpty() and return 0; // else remove the item at the front of the queue and return a pointer to it };

Queue Manipulation Issue
It’s intuitive to use array for implementing a queue. However, queue manipulations (add and/or delete) will require elements in the array to move. In the worse case, the complexity is of O(MaxSize).

Shifting Elements in Queue
front rear front rear front rear

Circular Queue To resolve the issue of moving elements in the queue, circular queue assigns next element to q[0] when rear == MaxSize – 1. Pointer front will always point one position counterclockwise from the first element in the queue. Queue is empty when front == rear. But it is also true when queue is full. This will be a problem.

Circular Queue (Cont.) 4 4 n-4 n-4 3 3 n-3 n-3 2 2 n-2 n-2 1 1 n-1 n-1
J4 J3 n-4 n-4 3 3 J1 J2 n-3 n-3 J2 2 J1 2 J3 J4 n-2 n-2 1 1 n-1 n-1 front = 0; rear = 4 front = n-4; rear = 0

Circular Queue (Cont.) To resolve the issue when front == rear on whether the queue is full or empty, one way is to use only MaxSize – 1 elements in the queue at any time. Each time when adding an item to the queue, newrear is calculated before adding the item. If newrear == front, then the queue is full. Another way to resolve the issue is using a flag to keep track of last operation. The drawback of the method is it tends to slow down Add and Delete function.

Subtyping and Inheritance in C++
Inheritance is used to express subtype relationships between two ADTs. If B inherits from A, then B IS-A A. Also, A is more general than B. VW Beetle IS-A Car; Eagle IS-A Bird

Inheritance A derived class inherits all the non-private members (data and functions) of the base class. Inherited members from public inheritance have the same level of access in the derived class as they did in the base class. The derived class can reuse the implementation of a function in the base class or implement its own function, with the exception of constructor and destructor.

Class Inheritance Example
class Bag { public: Bag (int MaxSize = DefaultSize); // constructor virtual ~Bag(); // destructor virtual void Add(int); // insert element into bag virtual int* Delete (int&); //delete element from bag virtual Boolean IsFull(); // return TRUE if the bag is full; FALSE otherwise virtual Boolean IsEmpty(); // return TRUE if the bag is empty; FALSE otherwise protected: virtual void Full(); // action when bag is full virtual void Empty(); // action when bag is empty int *array; int MaxSize; // size of array int top; // highest position in array that contains an element }

Class Inheritance Example(Cont.)
class Stack : public Bag { public: Stack(int MaxSize = DefaultSize); // constructor ~Stack(); // destructor int* Delete(int&); // delete element from stack }; Stack:: Stack (int MaxStackSize) : Bag(MaxStackSize) { } // Constructor for Stack calls constructor for Bag Stack::~Stack() { } // Destructor for Bag is automatically called when Stack is destroyed. This ensures that array is deleted. int* Stack::Delete(int& x) { if (IsEmpty()) {Empty(); return 0; } x = array[top--]; return &x; }

Class Inheritance Example
Bag b(3); // uses Bag constructor to create array of size 3 Stack s(3); // uses Stack constructor to create array of size 3 b.Add(1); b.Add(2); b.Add(3); // use Bag::Add. Bag::Add calls functions Bag::IsFull and Bag::Full s.Add(1); s.Add(2); s.Add(3); // Stack::Add not defined, so use Bag::Add. Bag::Add calls Bag::IsFull // and Bag::Full because these have not been redefined in Stack int x; b.Delete(x); // uses Bag::Delete, which calls Bag::IsEmpty and Bag::Emtpy s.Delete(x); // uses Stack::Delete, which calls Bag::IsEmtpy and Bag::Emtpy because these // have not been redefined in Stack.

The Maze Problem Entrance Exit

Allowable Moves N [i-1][j-1] [i-1][j] [i-1][j+1] NE NW W [i][j-1] X
SW S SE

The Maze Problem (Cont.)
Stack is used in solving the maze problem for storing the coordinates and direction. Use of another mxp array to mark any position that has been visited before.

Evaluation Expression in C++
When evaluating operations of the same priorities, it follows the direction from left to right. Priority Operator 1 Unary minus, ! 2 *, /, % 3 +, - 4 <, <=, >=, > 5 ==, != 6 && 7 ||

Postfix Notation Expressions are converted into Postfix notation before compiler can accept and process them. X = A/B – C + D * E – A * C Infix A/B–C+D*E–A*C Postfix => AB/C-DE*+AC*- Operation Postfix T1 = A / B T1C-DE*+AC*- T2 = T1 - C T2 DE*+AC*- T3 = D * E T2T3+AC*- T4 = T2 + T3 T4AC*- T5 = A * C T4 T5 - T6 = T4 - T5 T6

Postfix notation generation
X = A+ B* C / D – E * F 後置式: ABC*D/+EF*- 輸出 A A AB AB ABC ABC* ABC* ABC*D + + * + * + + / + / + 堆疊輸出 ABC*D/ ABC*D/+ ABC*D/+E ABC*D/+E ABC*D/+EF + - - * - * - 堆疊

Postfix notation generation
X = A+ (B+C*D) – E 後置式: ABCD*++E– 輸出 A A A AB AB ABC ABC ABCD ABCD* + ( + ( + + ( + ( * + ( * + ( + ( 堆疊輸出 ABCD*+ ABCD*+ ABCD*++ ABCD*++E ABCD*++E- ( + + - - 堆疊

Postfix notation execution
後置式: ABC*D/+EF*- A B A C B A B*C A D B*C A B*C/D A A+B*C/D 堆疊 E A+B*C/D F E A+B*C/D E*F A+B*C/D A+B*C/D-E*F

Multiple Stacks and Queues
Two stacks case: More than two stacks case 1 2 3 4 m-4 m-3 m-2 m-1 Stack A Stack B m-1 b[0] b[1] b[2] b[n] t[0] t[1] t[2] t[n]

Chap 4 Linked Lists

Review of Sequential Representations
Previously introduced data structures, including array, queue, and stack, they all have the property that successive nodes of data object were stored a fixed distance apart. The drawback of sequential mapping for ordered lists is that operations such as insertion and deletion become expensive. Also sequential representation tends to have less space efficiency when handling multiple various sizes of ordered lists.

Linked List A better solutions to resolve the aforementioned issues of sequential representations is linked lists. Elements in a linked list are not stored in sequential in memory. Instead, they are stored all over the memory. They form a list by recording the address of next element for each element in the list. Therefore, the list is linked together. A linked list has a head pointer that points to the first element of the list. By following the links, you can traverse the linked list and visit each element in the list one by one.

Linked List Insertion To insert an element into the three letter linked list: Get a node that is currently unused; let its address be x. Set the data field of this node to GAT. Set the link field of x to point to the node after FAT, which contains HAT. Set the link field of the node cotaining FAT to x.

Linked List Insertion And Deletion
first BAT CAT EAT FAT HAT GAT first BAT CAT EAT FAT GAT HAT

Designing a List in C++ Design Attempt 1: Use a global variable first which is a pointer of ThreeLetterNode. Unable to access to private data members: data and link. Design Attempt 2: Make member functions public. Defeat the purpose of data encapsulation. Design Attempt 3: Use of two classes. Create a class that represents the linked list. The class contains the items of another objects of another class.

Program 4.1 Composite Classes
class ThreeLetterList; // forward delcarion class ThreeLetterNode { friend class ThreeLetterList; private: char data[3]; ThreeLetterNode * link; }; class ThreeLetterList { public: // List Manipulation operations . ThreeLetterNode *first;

Nested Classes The Three Letter List problem can also use nested classes to represent its structure. class ThreeLetterList { public: // List Manipulation operations . private: class ThreeLetterNode { // nested class char data[3]; ThreeLetterNode *link; }; ThreeLetterNode *first;

Pointer Manipulation in C++
Addition of integers to pointer variable is permitted in C++ but sometimes it has no logical meaning. Two pointer variables of the same type can be compared. x == y, x != y, x == 0 a x a x b x y b b b y y x = y *x = * y

Define A Linked List Template
A linked list is a container class, so its implementation is a good template candidate. Member functions of a linked list should be general that can be applied to all types of objects. When some operations are missing in the original linked list definition, users should not be forced to add these into the original class design. Users should be shielded from the detailed implementation of a linked list while be able to traverse the linked list. Solution => Use of ListIterator

List Iterator A list iterator is an object that is used to traverse all the elements of a container class. ListIterator<Type> is delcared as a friend of both List<Type> and ListNode<Type>. A ListIterator<Type> object is initialized with the name of a List<Type> object l with which it will be associated. The ListIterator<Type> object contains a private data member current of type ListNode<Type> *. At all times, current points to a node of list l. The ListIterator<Type> object defines public member functions NotNull(), NextNotNull(), First(), and Next() to perform various tests on and to retrieve elements of l.

Template of Linked Lists
Enum Boolean { FALSE, TRUE}; template <class Type> class List; template <class Type> class ListIterator; template <class Type> class ListNode { friend class List<Type>; friend class ListIterator <Type>; private: Type data; ListNode *link; }; Template <class Type> class List { friend class ListIterator <Type>; public: List() {first = 0;}; // List manipulation operations . ListNode <Type> *first;

Template of Linked Lists (Cont.)
template <class Type> class ListIterator { public: ListIterator(const List<Type> &l): list(l), current(l.first) {}; Boolean NotNull(); Boolean NextNotNull(); Type * First(); Type * Next(); Private: const List<Type>& list; // refers to an existing list ListNode<Type>* current; // points to a node in list };

Attaching A Node To The End Of A List
Template <class Type> Void List<Type>::InsertBack(Type k) { ListNode<Type>*newnode = new ListNode<Type>(k); if (first == 0) first = last =newnode; else { last->link = newnode; last = newnode; } };

Concatenating Two Chains
Template <class Type> void List<Type>:: Concatenate(List<Type> b) // this = (a1, …, am) and b = (b1, …, bn) m, n ≥ , // produces the new chain z = (a1, …, am, b1, bn) in this. { if (!first) { first = b.first; return;} if (b.first) { for (ListNode<Type> *p = first; p->link; p = p->link); // no body p->link = b.first; }

Reversing a list Template <class Type>
void List<Type>:: Reverse( ) // List is reversed so that (a1, …, am) becomes (am, …, a1) . { ListNode <Type> *current = first; *previous = 0; //previous trails current while (current) { ListNode <Type> *r = previous; previous = current; current = current -> link; previous->link = r; } first = previous;

When Not To Reuse A Class
If efficiency becomes a problem when reuse one class to implement another class. If the operations required by the application are complex and specialized, and therefore not offered by the class.

Circular Lists By having the link of the last node points to the first node, we have a circular list. Need to make sure when current is pointing to the last node by checking for current->link == first. Insertion and deletion must make sure that the circular structure is not broken, especially the link between last node and first node.

Diagram of A Circular List
first last

Linked Stacks and Queues
top front rear Linked Queue Linked Stack

Revisit Polynomials 14 3 2 8 1 a.first 14 8 -3 10 6 b.first

Polynomial Class Definition
struct Term // all members of Terms are public by default { int coef; // coefficient int exp; // exponent void Init(int c, int e) {coef = c; exp = e;}; }; class Polynomial friend Polynomial operator+(const Polynomial&, const Polynomial&); private: List<Term> poly;

Operating On Polynomials
With linked lists, it is much easier to perform operations on polynomials such as adding and deleting. E.g., adding two polynomials a and b a.first 3 14 2 8 1 p b.first 8 14 -3 10 10 6 q (i) p->exp == q->exp c.first 11 14

a.first 3 14 2 8 1 p b.first 8 14 -3 10 10 6 q c.first 11 14 -3 10 (ii) p->exp < q->exp

a.first 3 14 2 8 1 p b.first 8 14 -3 10 10 6 q c.first 11 14 -3 10 2 8 (iii) p->exp > q->exp

Memory Leak When polynomials are created for computation and then later on out of the program scope, all the memory occupied by these polynomials is supposed to return to system. But that is not the case. Since ListNode<Term> objects are not physically contained in List<Term> objects, the memory they occupy is lost to the program and is not returned to the system. This is called memory leak. Memory leak will eventually occupy all system memory and causes system to crash. To handle the memory leak problem, a destructor is needed to properly recycle the memory and return it back to the system.

List Destructor Template <class Type> List<Type>::~List()
// Free all nodes in the chain { ListNode<Type>* next; for (; first; first = next) { next = first->link; delete first; }

Free Pool When items are created and deleted constantly, it is more efficient to have a circular list to contain all available items. When an item is needed, the free pool is checked to see if there is any item available. If yes, then an item is retrieved and assigned for use. If the list is empty, then either we stop allocating new items or use new to create more items for use.

Using Circular Lists For Polynomials
By using circular lists for polynomials and free pool mechanism, the deleting of a polynomial can be done in a fixed amount of time independent of the number of terms in the polynomial.

Deleting A Polynomial with a Circular List Structure
av 3 a.first 3 14 2 8 1 1 2 second av

Equivalence Class For any polygon x, x ≡ x. Thus, ≡ is reflexive.
For any two polygons x and y, if x ≡ y, then y ≡ x. Thus, the relation ≡ is symetric. For any three polygons x, y, and z, if x ≡ y and y ≡ z, then x ≡ z. The relation ≡ is transitive.

Equivalence Definition: A relation ≡ over a set S, is said to be an equivalence relation over S iff it is symmetric, reflexive, and transitive over S. Example: Supposed 12 polygons 0 ≡ 4, 3 ≡ 1, 6 ≡ 10, 8 ≡ 9, 7 ≡ 4, 6 ≡ 8, 3 ≡ 5, 2 ≡ 11, and 11 ≡ 0. Then they are partitioned into three equivalence classes: {0, 2, 4, 7, 11}; {1 , 3, 5}; {6, 8, 9 , 10}

Equivalence (Cont.) Two phases to determine equivalence
In the first phase the equivalence pairs (i, j) are read in and stored. In phase two, we begin at 0 and find all pairs of the form (0, j). Continue until the entire equivalence class containing 0 has been found, marked, and printed. Next find another object not yet output, and repeat the above process.

Equivalence Classes (Cont.)
If a Boolean array pairs[n][n] is used to hold the input pairs, then it might waste a lot of space and its initialization requires complexity Θ(n2) . The use of linked list is more efficient on the memory usage and has less complexity, Θ(m+n) .

Linked List Representation
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] data 11 3 11 5 7 3 8 4 6 8 6 link data 4 1 10 9 2 link

Linked List for Sparse Matrix
Sequential representation of sparse matrix suffered from the same inadequacies as the similar representation of Polynomial. Circular linked list representation of a sparse matrix has two types of nodes: head node: tag, down, right, and next entry node: tag, down, row, col, right, value Head node i is the head node for both row i and column i. Each head node is belonged to three lists: a row list, a column list, and a head node list. For an n x m sparse matrix with r nonzero terms, the number of nodes needed is max{n, m} + r + 1.

Node Structure for Sparse Matrices
down tag right down tag row col right f i j next value Head node Typical node Setup for aij A 4x4 sparse matrix

Linked Representation of A Sparse Matrix
H0 H1 H2 H3 Matrix head 4 H0 2 11 H1 1 12 H2 2 1 -4 H3 3 -15

Reading In A Sparse Matrix
Assume the first line consists of the number of rows, the number of columns, and the number of nonzero terms. Then followed by num-terms lines of input, each of which is of the form: row, column, and value. Initially, the next field of head node i is to keep track of the last node in column i. Then the column field of head nodes are linked together after all nodes has been read in.

Complexity Analysis Input complexity: O(max{n, m} + r) = O(n + m + r)
Complexity of ~Maxtrix(): Since each node is in only one row list, it is sufficient to return all the row lists of a matrix. Each row is circularly linked, so they can be erased in a constant amount of time. The complexity is O(m+n).

Doubly Linked Lists The problem of a singly linked list is that supposed we want to find the node precedes a node ptr, we have to start from the beginning of the list and search until find the node whose link field contains ptr. To efficiently delete a node, we need to know its preceding node. Therefore, doubly linked list is useful. A node in a doubly linked list has at least three fields: left link field (llink), a data field (item), and a right link field (rlink).

Doubly Linked List A head node is also used in a doubly linked list to allow us to implement our operations more easily. llink item rlink Head Node llink item rlink Empty List

Deletion From A Doubly Linked Circular List
llink item rlink Head Node x -> left -> right = x -> right x -> right -> left = x -> left

Insertion Into An Empty Doubly Linked Circular List
node node x x p newnode p p -> left = x; p -> right = x -> right; x -> right -> left = p; x -> right = p;

Generalized Lists Definition: A generalized list, A, is a finite sequence of n ≥ 0 elements, α0, α1, α2, …, αn-1, where αi, is either an atom or a list. The elements αi,0≤ i ≤ n – 1, that are not atoms are said to be the sublists of A. A list A is written as A = (α0, …, αn-1 ), and the length of the list is n. Conventionally, a capital letter is used to represent a list and a lower case letter is to represent an atom. The α0 is the head of list A and the rest (α1, …, αn-1) is the tail of list A.

Generalized List Examples
D = ( ): the null, or empty, list; its length is zero. A = (a, (b, c)): a list of length of two; its first element is the atom a, and its second element is the linear list (b, c). B = (A, A, ( )): A list of length of three whose first two elements are the list A, and the third element is the null list D. C = (a,C): is a recursive list of length two; C corresponds to the infinite list C = (a, (a, (a, …))).

Generalized Lists head(A) = ‘a’ and tail(A) = (b, c), head(tail(A) ) = (b, c) and tail(tail(A)) = (). Lists may be shared by other lists Lists may be recursive.

Generalized List Application Example
Consider the polynomial P(x, y, z) with various variables. It is obvious the sequential representation is not suitable to this. What if a linear list is used? The size of the node will vary in size, causing problems in storage management. Let’s try the generalized list.

Generalized List Application Example
P(x, y, z) can be rewritten as follows: The above can be written as Cz2 + Dz. Both C and D are polynomials themselves but with variables x and y only. If we look at polynomial C only, it is actually of the form Ey3 + Fy2, where E and F are polynomial of x only. Continuing this way, every polynomial consists of a variable plus coefficient-exponent pairs. Each coefficient is itself a polynomial.

PolyNode Class in C++ enum Triple{ var, ptr, no }; class PolyNode {
PolyNode *link; int exp; Triple trio; union { char vble; PolyNode *dlink; int coef; };

PolyNode in C++ (Cont.) trio == var: the node is a head node.
vble indicates the name of the variable. Or it is an integer point to the variable in a variable table. exp is set to 0. trio == ptr: coefficient itself is a list and is pointed by the field dlink. exp is the exponent of the variable on which the list is based on. trio == no, coefficient is an integer and is stored in coef. exp is the exponent of the variable on which the list is based on.

Representing 3x2y trio vble exp link trio vble exp link var y ptr 1
ptr 1 var x no 3 2 P

Representation of P(x, y, z)
v z p 2 p 1 v y p 3 p 2 v y p 4 p 1 v x n 2 v x n 3 8 v x n 1 10 n 2 8 v x n 1 4 n 6 3

Recursive Algorithms For Lists
A recursive algorithm consists of two components: The recursive function (the workhorse); declared as a private function A second function that invokes the recursive function at the top level (the driver); declared as a public function.

Program 4.6 Copying A List // Driver
void GenList::Copy(const GenList& l) { first = Copy(l.first); } // Workhorse GenListNode* GenList::Copy(GenListNode *p) // Copy the nonrecursive list with no shared sublists pointed at by p GenListNode *q = 0; if (p) { q = new GenListNode; q->tag = p->tag; if (!p->tag) q->data = p->data; else q->dlink = Copy(p->dlink); q->link = Copy(p->link); return q;

Linked Representation for A
A=((a, b), ((c, d), e)) r b t t t u v s f a f b t f e w x f c f d

Generalized List Representation Example
D = 0 Empty list A=(a, (b, c)) A f a t f b t c B t t B=(A, A, ()) C f a t C=(a, C)

Recursiveness GenList::Copy
Level of recursion Value of p Continuing level p 1 b 2 r 3 u s 4 v t w 5 x 6

Important List Functions
List Equality (Program 4.37) List Depth (Program 4.38) An empty list has depth 0.

Reference Counts, Shared and Recursive Lists
Lists may be shared by other lists for the purpose of space saving. Lists that are shared by other lists create problems when performing add or delete functions. For example, let’s look at the previous A, B, C, D example. When deleting the front node of list A would requires List B to update its pointers. The use of the data field of a head node to record the reference count can resolve the aforementioned problem. The list can not be deleted unless the reference count is 0.

Example of Reference Counts, Shared and Recursive Lists
1 A=(a, (b, c)) Y f 3 f a t f 1 f b f c Z f 1 t t t B=(A, A, ()) W f 2 f a t f 1 C=(a, C)

Erasing A List Recursively
// Driver GenList::~GenList() // Each head node has a reference count. We assume first ≠ 0. { Delete(first); first = 0; } // Workhorse void GenList::Delete(GenListNode* x) x->ref--; // decrement reference coutn of head node. if (!x->ref) GenListNode *y = x; // y traverses top-level of x. while (y->link) { y= y->link; if (y->tag == 1) Delete (y->dlink);} y->link = av; // Attach top-level nodes to av list av = x;

Issue In Erasing Recursive Lists
When erasing a recursive list (either direct recursive or indirect recursive), the reference count does not become 0. Then the nodes are not returned to available list. This will cause memory leak issue.

Chap 5 Trees

Trees Definition: A tree is a finite set of one or more nodes such that: There is a specially designated node called the root. The remaining nodes are partitioned into n ≥ 0 disjoint sets T1, …, Tn, where each of these sets is a tree. We call T1, …, Tn the subtrees of the root.

Pedigree Genealogical Chart
Cheryl Kevin Rosemary John Terry Richard Kelly Jack Mary Anthony Karen Joe Michelle Mike Angela Binary Tree

Lineal Genealogical Chart
Proto Indo-European Italic Hellenic Germanic Osco-Umbrian Latin Greek North Germanic West Germanic Osco Umbrian Spanish French Italian Icelandic Norwegian Swedish Low High Yiddish

Tree Terminology Normally we draw a tree with the root at the top.
The degree of a node is the number of subtrees of the node. The degree of a tree is the maximum degree of the nodes in the tree. A node with degree zero is a leaf or terminal node. A node that has subtrees is the parent of the roots of the subtrees, and the roots of the subtrees are the children of the node. Children of the same parents are called siblings.

Tree Terminology (Cont.)
The ancestors of a node are all the nodes along the path from the root to the node. The descendants of a node are all the nodes that are in its subtrees. Assume the root is at level 1, then the level of a node is the level of the node’s parent plus one. The height or the depth of a tree is the maximum level of any node in the tree.

A Sample Tree Level A 1 B C D 2 E F G H I J 3 4 K L M

List Representation of Trees
The tree in previous slide could be written as (A (B (E (K, L), F), C(G), D(H (M), I, J))) A B F C G D I J E K L H M

Possible Node Structure For A Tree of Degree
Lemma 5.1: If T is a k-ary tree (i.e., a tree of degree k) with n nodes, each having a fixed size as in Figure 5.4, then n(k-1) + 1 of the nk child fileds are 0, n ≥ 1. Data Child 1 Child 2 Child 3 Child 4 … Child k Wasting memory!

Representation of Trees
Left Child-Right Sibling Representation Each node has two links (or pointers). Each node only has one leftmost child and one closest sibling. A data left child right sibling B C D E F G H I J K L M

Degree Two Tree Representation
B E C K F G D Binary Tree! L H M I J

Tree Representations A A A B B B A A A B C B B C C
Left child-right sibling Binary tree

Binary Tree Definition: A binary tree is a finite set of nodes that is either empty or consists of a root and two disjoint binary trees called the left subtree and the right subtree. There is no tree with zero nodes. But there is an empty binary tree. Binary tree distinguishes between the order of the children while in a tree we do not.

Binary Tree Examples A A A B B B C C D E F G A B D H I E

The Properties of Binary Trees
Lemma 5.2 [Maximum number of nodes] The maximum number of nodes on level i of a binary tree is 2i-1, i ≥ 1. The maximum number of nodes in a binary tree of depth k is 2k – 1, k ≥ 1. Lemma 5.3 [Relation between number of leaf nodes and nodes of degree 2]: For any non-empty binary tree, T, if n0 is the number of leaf nodes and n2 the number of nodes of degree 2, then n0 = n2 + 1. Definition: A full binary tree of depth k is a binary tree of depth k having 2k – 1 nodes, k ≥ 0.

Binary Tree Definition
Definition: A binary tree with n nodes and depth k is complete iff its nodes correspond to the nodes numbered from 1 to n in the full binary tree of depth k. level 1 1 2 2 3 3 4 5 6 7 4 8 9 10 11 12 13 14 15

Array Representation of A Binary Tree
Lemma 5.4: If a complete binary tree with n nodes is represented sequentially, then for any node with index i, 1 ≤ i ≤ n, we have: parent(i) is at if i ≠1. If i = 1, i is at the root and has no parent. left_child(i) is at 2i if 2i ≤ n. If 2i > n, then i has no left child. right_child(i) is at 2i + 1 if 2i + 1 ≤ n. If 2i + 1 > n, then i has no right child. Position zero of the array is not used.

Proof of Lemma 5.4 (2) Assume that for all j, 1 ≤ j ≤ i, left_child(j) is at 2j. Then two nodes immediately preceding left_child(i + 1) are the right and left children of i. The left child is at 2i. Hence, the left child of i + 1 is at 2i + 2 = 2(i + 1) unless 2(i + 1) > n, in which case i + 1 has no left child.

Array Representation of Binary Trees
C D E [1] [2] [3] [4] [5] [6] [7] [8] [9] [16] [1] A [2] B [3] C [4] D [5] E [6] F [7] G [8] H [9] I

Linked Representation
class Tree; class TreeNode { friend class Tree; private: TreeNode *LeftChild; char data; TreeNode *RightChild; }; class Tree { public: // Tree operations . TreeNode *root;

Node Representation data LeftChild RightChild LeftChild data

Linked List Representation For The Binary Trees
root root A A B C B C D E F G D E H I

Tree Traversal When visiting each node of a tree exactly once, this produces a linear order for the node of a tree. There are 3 traversals if we adopt the convention that we traverse left before right: LVR (inorder), LRV (postorder), and VLR (preorder). When implementing the traversal, a recursion is perfect for the task.

Binary Tree With Arithmetic Expression
+ * E * D / C A B

Tree Traversal Inorder Traversal: A/B*C*D+E
=> Infix form Preorder Traversal: +**/ABCDE => Prefix form Postorder Traversal: AB/C*D*E+ => Postfix form

Inorder traversal template <class T> void Tree <T>::Inorder() {//Driver calls workhorse for traversal of entire tree. Inorder(root); } void Tree <T>::Inorder (TreeNode <T> *currentNode) {//Workhouse traverses the subtree rooted at currentNode. If (currentNode){ Inorder(currentNode->leftChild); Visit(currentNode); Inorder(currentNode->Child);

Iterative Inorder Traversal
void Tree::NonrecInorder() // nonrecursive inorder traversal using a stack { Stack<TreeNode *> s; // declare and initialize stack TreeNode *CurrentNode = root; while (1) { while (CurrentNode) { // move down LeftChild fields s.Add(CurrentNode); // add to stack CurrentNode = CurrentNode->LeftChild; } if (!s.IsEmpty()) { // stack is not empty CurrentNode = *s.Delete(CurrentNode); cout << CurrentNode->data << endl; CurrentNode = CurrentNode->RightChild; else break;

Level-Order Traversal
All previous mentioned schemes use stacks. Level-order traversal uses a queue. Level-order scheme visit the root first, then the root’s left child, followed by the root’s right child. All the node at a level are visited before moving down to another level.

Level-Order Traversal of A Binary Tree
void Tree::LevelOrder() // Traverse the binary tree in level order { Queue<TreeNode *> q; TreeNode *CurrentNode = root; while (CurrentNode) { cout << CurrentNode->data<<endl; if (CurrentNode->LeftChild) q.Add(CurrentNode->LeftChild); if (CurrentNode->RightChild) q.Add(CurrentNode->RightChild); CurrentNode = *q.Delete(); } +*E*D/CAB

Traversal Without A Stack
Use of parent field to each node. Use of two bits per node to represents binary trees as threaded binary trees.

Some Other Binary Tree Functions
With the inorder, postorder, or preorder mechanisms, we can implement all needed binary tree functions. E.g., Copying Binary Trees Testing Equality Two binary trees are equal if their topologies are the same and the information in corresponding nodes is identical.

The Satisfiability Problem
Expression Rules A variable is an expression If x and y are expressions then are expressions Parentheses can be used to alter the normal order of evaluation, which is not before and before or.

Propositional Formula In A Binary Tree
x3 x1 x3 x2 x1 O(g2n)

Perform Formula Evaluation
To evaluate an expression, we can traverse its tree in postorder. To perform evaluation, we assume each node has four fields LeftChild data value RightChild LeftChild data value RightChild

First Version of Satisfiability Algorithm
For all 2n possible truth value combinations for the n variables { generate the next combination; replace the variables by their values; evaluate the formula by traversing the tree it points to in postorder; if (formula.rootvalue()) {cout << combination; return;} } Cout << “no satisfiable combination”;

Evaluating A Formula void SatTree::PostOrderEval() // Driver {
PostOrderEval(root); } void SatTree::PostOrderEval(SatNode * s) // workhorse if (s) { PostOrderEval(s->LeftChild); PostOrderEval(s->RightChild); switch (s->data) { case LogicalNot: s->value =!s->RightChild->value; break; case LogicalAnd: s->value = s->LeftChild->value && s->RightChild->value; break; case LogicalOr: s->value = s->LeftChild->value || s->RightChild->value; case LogicalTrue: s->value = TRUE; break; case LogicalFalse: s->value = FALSE;

Threaded Binary Tree Threading Rules
A 0 RightChild field at node p is replaced by a pointer to the node that would be visited after p when traversing the tree in inorder. That is, it is replaced by the inorder successor of p. A 0 LeftChild link at node p is replaced by a pointer to the node that immediately precedes node p in inorder (i.e., it is replaced by the inorder predecessor of p).

Threaded Tree Corresponding to Figure 5.10(b)
Inorder sequence: H, D, I, B, E, A, F, C, G

Threads To distinguish between normal pointers and threads, two boolean fields, LeftThread and RightThread, are added to the record in memory representation. t->LeftChild = TRUE => t->LeftChild is a thread t->LeftChild = FALSE => t->LeftChild is a pointer to the left child.

Threads (Cont.) To avoid dangling threads, a head node is used in representing a binary tree. The original tree becomes the left subtree of the head node. Empty Binary Tree LeftThread LeftChild data RightChild RightThread TRUE FALSE

Memory Representation of Threaded Tree of Figure 5.20
- f f A f f B f f B f f D t E t f D f t E t t H t t I t

Finding the inorder successor without stack
By using the threads, we can perform an inorder traversal without making use of a stack. T* ThreadedInorderIterator::Next() {//Return the inorder successor of currentnode ThreadedNode <T> *temp = currentNode -> rightChild; if (!currentNode->rightThread) while (!temp->leftThread) temp = temp -> leftChild; currentNode = temp; if (currentNode == root) return 0; else return &currentNode -> data; }

Inserting A Node to AThreaded Binary Tree
Inserting a node r as the right child of a node s. If s has an empty right subtree, then the insertion is simple and diagram in Figure 5.23(a). If the right subtree of s is not empty, the this right subtree is made the right subtree of r after insertion. When thisis done, r becomes the inorder predecessor of a node that has a LdeftThread==TRUE field, and consequently there is an thread which has to be updated to point to r. The node containing this thread was previously the inorder successor of s. Figure 5.23(b) illustrates the insertion for this case.

Insertion of r As A Right Child of s in A Threaded Binary Tree
before after

Insertion of r As A Right Child of s in A Threaded Binary Tree (Cont.)
before after

Program 5.16 Inserting r As The Right Child of s
void ThreadedTree::InsertRight(ThreadNode *s, ThreadedNode *r) // Insert r as the right child of s { r->RightChild = s->RightChild; r->RightThread = s->RightThread; r->LeftChild = s; r->LeftThread = TRUE; // LeftChild is a thread r->RightChild = r; // attach r to s r->RightThread = FALSE; if (!r->RightThread) { ThreadedNode *temp = InorderSucc(r); // returns the inorder successor of r temp->LeftChild = r; }

Priority Queues In a priority queue, the element to be deleted is the one with highest (or lowest) priority. An element with arbitrary priority can be inserted into the queue according to its priority. A data structure supports the above two operations is called max (min) priority queue.

Examples of Priority Queues
Suppose a server that serve multiple users. Each user may request different amount of server time. A priority queue is used to always select the request with the smallest time. Hence, any new user’s request is put into the priority queue. This is the min priority queue. If each user needs the same amount of time but willing to pay more money to obtain the service quicker, then this is max priority queue.

Priority Queue Representation
Unorder Linear List Addition complexity: O(1) Deletion complexity: O(n) Chain Ordered List Addition complexity: O(n) Deletion complexity: O(1)

Max (Min) Heap Heaps are frequently used to implement priority queues. The complexity is O(log n). Definition: A max (min) tree is a tree in which the key value in each node is no smaller (larger) than the key values in its children (if any). A max heap is a complete binary tree that is also a max tree. A min heap is a complete binary tree that is also a min tree.

Max Heap Examples 9 14 12 7 6 3 10 8 6 5

Insertion Into A Max Heap (1)
20 20 15 2 15 2 14 10 14 10 1

20 20 15 2 15 2 5 5 14 10 14 10 2

20 21 20 15 2 15 20 2 14 10 14 10 2

Deletion From A Max Heap
template <class Type> void MaxHeap <T>::Pop() {// Delete from the max heap if (IsEmpty()) throw “Heap is empty.”; heap[1].~T(); //delete max element //remove last element from heap T lastE = heap [heapSize--]; int currentNode = 1; int child = 2; while (child >= leapSize) { if (child < heapSize && heap[child] < heap[child+1]) child++; if (lastE >= heap[child]) break; heap[currentNode] = heap[child]; currentNode = child; child *= 2; } heap[currentNode] = lastE;

Deletion From A Max Heap (Cont.)
21 2 15 20 15 20 14 10 2 14 10 2

Deletion From A Max Heap (Cont.)
2 20 15 20 15 2 14 10 14 10

Binary Search Tree Definition: A binary serach tree is a binary tree. It may be empty. If it is not empty then it satisfies the following properties: Every element has a key and no two elements have the same key (i.e., the keys are distinct) The keys (if any) in the left subtree are smaller than the key in the root. The keys (if any) in the right subtree are larger than the key in the root. The left and right subtrees are also binary search trees.

Binary Trees 30 60 20 70 5 40 15 25 65 80 14 10 22 2 Binary search trees Not binary search tree

Searching A Binary Search Tree
If the root is 0, then this is an empty tree. No search is needed. If the root is not 0, compare the x with the key of root. If x equals to the key of the root, then it’s done. If x is less than the key of the root, then no elements in the right subtree can have key value x. We only need to search the left tree. If x larger than the key of the root, only the right subtree is to be searched.

Search Binary Search Tree by Rank
To search a binary search tree by the ranks of the elements in the tree, we need additional field “LeftSize”. LeftSize is the number of the elements in the left subtree of a node plus one. It is obvious that a binary search tree of height h can be searched by key as well as by rank in O(h) time.

Searching A Binary Search Tree by Rank
template <class Type> BstNode <Type>* BST<Type>::Search(int k) // Search the binary search tree for the kth smallest element { BstNode<Type> *t = root; while(t) if (k == t->LeftSize) return n; if (k < t->LeftSize) t = t->LeftChild; else { k -= LeftSize; t = t->RightChild; } return 0;

Insertion To A Binary Search Tree
Before insertion is performed, a search must be done to make sure that the value to be inserted is not already in the tree. If the search fails, then we know the value is not in the tree. So it can be inserted into the tree. It takes O(h) to insert a node to a binary search tree.

Inserting Into A Binary Search Tree
30 30 5 40 5 40 2 80 2 35 80

Insertion Into A Binary Search Tree
Template <class Type> Boolean BST<Type>::Insert(const Element<Type>& x) // insert x into the binary search tree { // Search for x.key, q is the parent of p BstNode<Type> *p = root; BstNode<Type> *q = 0; while(p) { q = p; if (x.key == p->data.key) return FALSE; // x.key is already in tree if (x.key < p->data.key) p = p->LeftChild; else p = p->RightChild; } // Perform insertion p = new BstNode<Type>; p->LeftChild = p->RightChild = 0; p->data = x; if (!root) root = p; else if (x.key < q->data.key) q->LeftChild = p; else q->RightChild = p; return TRUE; O(h)

Deletion From A Binary Search Tree
Delete a leaf node A leaf node which is a right child of its parent A leaf node which is a left child of its parent Delete a non-leaf node A node that has one child A node that has two children Replaced by the largest element in its left subtree, or Replaced by the smallest element in its right subtree Again, the delete function has complexity of O(h)

Deleting From A Binary Search Tree
30 30 30 5 40 2 5 40 2 35 80 2 80

Deleting From A Binary Search Tree
30 30 30 5 30 5 2 5 40 2 5 40 2 2 40 2 80 2 80 80

Joining and Splitting Binary Trees
C.ThreeWayJoin(A, x, B): Creates a binary search tree C that consists of binary search tree A, B, and element x. C.TwoWayJoin(A, B): Joins two binary search trees A and B to obtain a single binary search tree C. A.Split(i, B, x, C): Binary search tree A splits into three parts: B (a binary search tree that contains all elements of A that have key less than i); if A contains a key i than this element is copied into x and a pointer to x returned; C is a binary search tree that contains all records of A that have key larger than i.

ThreeWayJoin(A, x, B) 81 30 90 81 5 40 85 94 2 35 80 84 92 A B x

TwoWayJoin(A, B) 84 80 30 90 5 40 85 94 2 35 80 84 92 A B

A.Split(i, B, x, C) 30 x i = 30 5 40 2 35 80 B 75 81 A C

A.Split(i, B, x, C) C x A B i = 80 30 Y L Z t R 30 5 40 L 81 t R 5 40
2 35 80 t 2 35 75 80 75 81 x A B

Selection Trees When trying to merge k ordered sequences (assume in non-decreasing order) into a single sequence, the most intuitive way is probably to perform k – 1 comparison each time to select the smallest one among the first number of each of the k ordered sequences. This goes on until all numbers in every sequences are visited. There should be a better way to do this.

Winner Tree A winner tree is a complete binary tree in which each node represents the smaller of its two children. Thus the root represents the smallest node in the tree. Each leaf node represents the first record in the corresponding run. Each non-leaf node in the tree represents the winner of its right and left subtrees.

Winner Tree For k = 8 1 6 2 3 6 8 4 5 7 6 9 6 8 17 8 9 10 11 12 13 14 15 10 9 20 6 8 9 90 17 15 20 20 15 15 11 95 18 16 38 30 25 50 16 99 20 28 run1 run2 run3 run4 run5 run6 run7 run8

Winner Tree For k = 8 1 8 2 3 9 8 4 5 7 6 9 15 8 17 8 9 10 11 12 13 14 15 10 9 20 15 8 9 90 17 15 20 20 25 15 11 95 18 16 38 30 28 50 16 99 20 run1 run2 run3 run4 run5 run6 run7 run8

Analysis of Winner Tree
The number of levels in the tree is The time to restructure the winner tree is O(log2k). Since the tree has to be restructured each time a number is output, the time to merge all n records is O(n log2k). The time required to setup the selection tree for the first time is O(k). Total time needed to merge the k runs is O(n log2k).

Loser Tree A selection tree in which each nonleaf node retains a pointer to the loser is called a loser tree. Again, each leaf node represents the first record of each run. An additional node, node 0, has been added to represent the overall winner of the tournament.

Loser Tree 6 Overall winner 8 9 17 10 20 9 90 10 9 20 6 8 9 90 17 run
6 Overall winner 1 8 2 3 9 17 4 5 7 6 10 20 9 90 9 10 11 12 13 14 15 10 9 20 6 8 9 90 17 run 1 2 3 4 5 6 7 8

Loser Tree 8 Overall winner 9 15 17 10 20 9 90 10 9 20 15 8 9 90 17
8 Overall winner 1 9 2 3 15 17 4 5 7 6 10 20 9 90 9 10 11 12 13 14 15 10 9 20 15 8 9 90 17 run 1 2 3 4 5 6 7 8

Forests Definition: A forest is a set of n ≥ 0 disjoint trees.
When we remove a root from a tree, we’ll get a forest. E.g., Removing the root of a binary tree will get a forest of two trees.

Transforming A Forest Into A Binary Tree
Definition: If T1, …, Tn is a forest of trees, then the binary tree corresponding to this forest, denoted by B(T1, …, Tn), is empty if n = 0 has root equal to root (T1); has left subtree equal to B(T11, T12,…, T1m), where T11, T12,…, T1m are the subtrees of root (T1); and has right subtree B(T2, …, Tn).

Set Representation Trees can be used to represent sets.
Disjoint set union: If Si and Sj are two disjoint sets, then their union Si ∪Sj = {all elements x such that x is in Si or Sj}. Find(i). Find the set containing element i.

Possible Tree Representation of Sets
4 2 6 7 8 1 9 3 5 S1 S2 S3

Possible Representations of Si ∪Sj
4 6 7 8 4 1 9 1 9 6 7 8

Unions of Sets To obtain the union of two sets, just set the parent field of one of the roots to the other root. To figure out which set an element is belonged to, just follow its parent link to the root and then follow the pointer in the root to the set name.

Data Representation for S1, S2, S3
Set Name Pointer S1 4 2 S2 6 7 8 1 9 3 5 S3

Array Representation of S1, S2, S3
We could use an array for the set name. Or the set name can be an element at the root. Assume set elements are numbered 0 through n-1. i [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] parent -1 4 -1 2 -1 2 4

Union-Find Operations
For a set of n elements each in a set of its own, then the result of the union function is a degenerate tree. The time complexity of the following union-find operation is O(n2). union(0, 1), union(1, 2), …, union(n-2, n-1) find(0), find (1), …, find(n-1) The complexity can be improved by using weighting rule for union.

Degenerate Tree n-1 Union operation union(0, 1), find(0) O(n) n-1
Find operation O(n2) union(n-2, n-1), find(0)

Weighting Rule Definition [Weighting rule for union(I, j)]: If the number of nodes in the tree with root i is less than the number in the tree with root j, then make j the parent of i; otherwise make i the parent of j.

Trees Obtained Using The Weighting Rule
1 n-1 2 n-1 3 n-1 1 1 2 4 n-1 1 2 3 1 2 3 n-1

Weighted Union Lemma 5.5: Assume that we start with a forest of trees, each having one node. Let T be a tree with m nodes created as a result of a sequence of unions each performed using function WeightedUnion. The height of T is no greater than For the processing of an intermixed sequence of u – 1 unions and f find operations, the time complexity is O(u + f*log u).

Trees Achieving Worst-Case Bound
[-1] [-1] [-1] [-1] [-1] [-1] [-1] [-1] 1 2 3 4 5 6 7 (a) Initial height trees [-2] [-2] [-2] [-2] 2 4 6 1 3 5 7 (b) Height-2 trees following union (0, 1), (2, 3), (4, 5), and (6, 7)

Trees Achieving Worst-Case Bound (Cont.)
[-4] [-4] [-8] 4 1 2 5 6 1 2 4 3 7 3 5 6 (c) Height-3 trees following union (0, 2), (4, 6) 7 (d) Height-4 trees following union (0, 4)

Collapsing Rule Definition [Collapsing rule]: If j is a node on the path from i to its root and parent[i]≠ root(i), then set parent[j] to root(i). The first run of find operation will collapse the tree. Therefore, all following find operation of the same element only goes up one link to find the root.

Collapsing Find [-8] [-8] 1 2 4 1 2 4 6 7 3 5 6 3 5 7 After collapsing
1 2 4 1 2 4 6 7 3 5 6 3 5 7 After collapsing Before collapsing

Analysis of WeightedUnion and CollapsingFind
The use of collapsing rule roughly double the time for an individual find. However, it reduces the worst-case time over a sequence of finds. Lemma 5.6 [Tarjan and Van Leeuwen]: Assume that we start with a forest of trees, each having one node. Let T(f, u) be the maximum time required to process any intermixed sequence of f finds and u unions. Assume that u ≥ n/2. Then k1(n + fα(f + n, n)) ≤ T(f, u) ≤ k2(n + fα(f + n, n)) for some positive constants k1 and k2.

Revisit Equivalence Class
The aforementioned techniques can be applied to the equivalence class problem. Assume initially all n polygons are in an equivalence class of their own: parent[i] = -1, 0 ≤ i < n. Firstly, we must determine the sets that contains i and j. If the two are in different set, then the two sets are to be replaced by their union. If the two are in the same set, then nothing need to be done since they are already in the equivalence class. So we need to perform two finds and at most one union. If we have n polygons and m equivalence pairs, we need O(n) to set up the initial n-tree forest. 2m finds at most min{n-1, m} unions. If weightedUnion and CollapsingFind are used, the time complexity is O(n + m (2m, min{n-1, m})). This seems to slightly worse than section 4.7 (O(m+n)). But this scheme demands less space.

Example 5.5 [-1] [-1] [-1] [-1] [-1] [-1] [-1] [-1] [-1] [-1] [-1]
1 2 3 4 5 6 7 8 9 10 11 (a) Initial trees [-2] [-2] [-2] [-2] [-1] [-1] [-1] [-1] 3 6 8 2 5 7 11 4 1 10 9 (b) Height-2 trees following 0≡4, 3≡1, 6≡10, and 8≡9

Example 5.5 (Cont.) [-3] [-4] [-3] [-2] 6 3 2 4 7 10 8 1 5 11 9 [-3]
6 3 2 4 7 10 8 1 5 11 9 [-3] [-4] [-3] 6 3 5 4 7 2 10 8 1 11 9

Uniqueness of A Binary Tree
In section 5.3 we introduced preorder, inorder, and postorder traversal of a binary tree. Now suppose we are given a sequence (e.g., inorder sequence BCAEDGHFI), does the sequence uniquely define a binary tree?

Constructing A Binary Tree From Its Inorder Sequence
B, C D, E, F, G, H, I B D, E, F, G, H, I C

Constructing A Binary Tree From Its Inorder Sequence (Cont.)
1 A 2 4 B D 3 5 6 C E F 7 9 G I 8 H Preorder: 1, 2, 3, 4, 5, 6, 7, 8, 9 Inorder: 2, 3, 1, 5, 4, 7, 8, 6, 9

Distinct Binary Trees 1 1 1 1 1 2 2 3 2 2 2 3 3 3 3 (1, 2, 3)
(1, 3, 2) (2, 1, 3) (2, 3, 1) (3, 2, 1)

Distinct Binary Trees (Cont.)
The number of distinct binary trees is equal to the number of distinct inorder permutations obtainable from binary trees having the preorder permutation, 1, 2, …, n. Computing the product of n matrices are related to the distinct binary tree problem. M1 * M2 * … * Mn n = 3 (M1 * M2) * M3 M1 * (M2 * M3 ) n = 4 ((M1 * M2) * M3) * M4 (M1 * (M2 * M3)) * M4 M1 * ((M2 * M3) * M4 ) (M1 * (M2 * (M3 * M4 ))) ((M1 * M2) * (M3 * M4 )) Let bn be the number of different ways to compute the product of n matrices. b2 = 1, b3 = 2, and b4 = 5.

The number of distinct binary trees of n nodes is bn bi bn-i-1

Assume we let which is the generating function for the number of binary trees. By the recurrence relation we get

Chap 6 Graph

Konigsberg Bridge Problem
A river Pregel flows around the island Keniphof and then divides into two. Four land areas A, B, C, D have this river on their borders. The four lands are connected by 7 bridges a – g. Determine whether it’s possible to walk across all the bridges exactly once in returning back to the starting land area.

Konigsberg Bridge Problem (Cont.)
A Kneiphof e D C g f a B c d b e A D a b f B

Euler’s Graph Define the degree of a vertex to be the number of edges incident to it Euler showed that there is a walk starting at any vertex, going through each edge exactly once and terminating at the start vertex iff the degree of each vertex is even. This walk is called Eulerian. No Eulerian walk of the Konigsberg bridge problem since all four vertices are of odd edges.

Application of Graphs Analysis of electrical circuits
Finding shortest routes Project planning Identification of chemical compounds Statistical mechanics Genertics Cybernetics Linguistics Social Sciences, and so on …

Definition of A Graph A graph, G, consists tof two sets, V and E.
V is a finite, nonempty set of vertices. E is set of pairs of vertices called edges. The vertices of a graph G can be represented as V(G). Likewise, the edges of a graph, G, can be represented as E(G). Graphs can be either undirected graphs or directed graphs. For a undirected graph, a pair of vertices (u, v) or (v, u) represent the same edge. For a directed graph, a directed pair <u, v> has u as the tail and the v as the head. Therefore, <u, v> and <v, u> represent different edges.

Three Sample Graphs 1 2 1 2 1 3 3 4 5 6 2 (a) G1 (b) G2 (c) G3
1 2 1 2 1 3 3 4 5 6 2 V(G1) = {0, 1, 2, 3} V(G2) = {0, 1, 2, 3, 4, 5, 6} V(G3) = {0, 1, 2} E(G1) = {(0, 1), (0, 2), (0, 3), (1, 2), (1, 3), (2, 3)} E(G2) = {(0, 1), (0, 2), (1, 3), (1, 4), (2, 5), (2, 6)} E(G3) = {<0, 1>, <1, 0>, <1, 2>} (a) G1 (b) G2 (c) G3

Graph Restrictions A graph may not have an edge from a vertex back to itself. (v, v) or <v, v> are called self edge or self loop. If a graph with self edges, it is called a graph with self edges. A graph man not have multiple occurrences of the same edge. If without this restriction, it is called a multigraph.

Complete Graph The number of distinct unordered pairs (u, v) with u≠v in a graph with n vertices is n(n-1)/2. A complete unordered graph is an unordered graph with exactly n(n-1)/2 edges. A complete directed graph is a directed graph with exactly n(n-1) edges.

Examples of Graphlike Structures
1 1 3 2 2 (a) Graph with a self edge (b) Multigraph

Graph Edges If (u, v) is an edge in E(G), vertices u and v are adjacent and the edge (u, v) is the incident on vertices u and v. For a directed graph, <u, v> indicates u is adjacent to v and v is adjacent from u.

Subgraph and Path Subgraph: A subgraph of G is a graph G’ such that V(G’) V(G) and E(G’) E(G). Path: A path from vertex u to vertex v in graph G is a sequence of vertices u, i1, i2, …, ik, v, such that (u, i1), (i1, i2), …, (ik, v) are edges in E(G). The length of a path is the number of edges on it. A simple path is a path in which all vertices except possibly the first and last are distinct. A path (0, 1), (1, 3), (3, 2) can be written as 0, 1, 3, 2. Cycle: A cycle is a simple path in which the first and last vertices are the same. Similar definitions of path and cycle can be applied to directed graphs.

G1 and G3 Subgraphs 1 2 1 2 1 2 3 3 (a) Some subgraphs of G1 1 1 1 2 2
1 2 1 2 1 2 3 (i) (ii) (iii) 3 (iv) (a) Some subgraphs of G1 1 1 1 (i) (ii) 2 2 2 (a) Some subgraphs of G3 (iii) (iv)

Connected Graph Two vertices u and v are connected in an undirected graph iff there is a path from u to v (and v to u). An undirected graph is connected iff for every pair of distinct vertices u and v in V(G) there is a path from u to v in G. A connected component of an undirected is a maximal connected subgraph. A tree is a connected acyclic graph.

Strongly Connected Graph
A directed graph G is strongly connected iff for every pair of distinct vertices u and v in V(G), there is directed path from u to v and also from v to u. A strongly connected component is a maximal subgraph that is strongly connected.

Graphs with Two Connected Components
1 2 1 2 3 3 G4

Strongly Connected Components of G3
1 2

Degree of A Vertex Degree of a vertex: The degree of a vertex is the number of edges incident to that vertex. If G is a directed graph, then we define in-degree of a vertex: is the number of edges for which vertex is the head. out-degree of a vertex: is the number of edges for which the vertex is the tail. For a graph G with n vertices and e edges, if di is the degree of a vertex i in G, then the number of edges of G is

Abstract of Data Type Graphs
class Graph { // objects: A nonempty set of vertices and a set of undirected edges // where each edge is a pair of vertices public: Graph(); // Create an empty graph void InsertVertex(Vertex v); void InsertEdge(Vertex u, Vertex v); void DeleteVertex(Vertex v); void DeleteEdge(Vertex u, Vertex v); Boolean IsEmpty(); // if graph has no vertices return TRUE List<List> Adjacent(Vertex v); // return a list of all vertices that are adjacent to v };

Adjacent Matrix Let G(V, E) be a graph with n vertices, n ≥ 1. The adjacency matrix of G is a two-dimensional nxn array, A. A[i][j] = 1 iff the edge (i, j) is in E(G). The adjacency matrix for a undirected graph is symmetric, it may not be the case for a directed graph. For an undirected graph the degree of any vertex i is its row sum. For a directed graph, the row sum is the out-degree and the column sum is the in-degree.

Adjacency Matrices (a) G1 (b) G3 (c) G4

Adjacency Lists Instead of using a matrix to represent the adjacency of a graph, we can use n linked lists to represent the n rows of the adjacency matrix. Each node in the linked list contains two fields: data and link. data: contain the indices of vertices adjacent to a vertex i. Each list has a head node. For an undirected graph with n vertices and e edges, we need n head nodes and 2e list nodes. The degree of any vertex may be determined by counting the number nodes in its adjacency list. The number of edges in G can be determined in O(n + e). For a directed graph (also called digraph), the out-degree of any vertex can be determined by counting the number of nodes in its adjacency list. the in-degree of any vertex can be obtained by keeping another set of lists called inverse adjacency lists.

Adjacent Lists [0] 3 1 2 [1] 2 3 [2] 1 3 [3] 1 2 (a) G1 [0] 1 [1] 2
HeadNodes [0] 3 1 2 [1] 2 3 [2] 1 3 [3] 1 2 (a) G1 HeadNodes [0] 1 [1] 2 [2] (b) G3

Adjacent Lists (Cont.) [0] 2 1 [1] 3 [2] 3 [3] 1 1 [4] 5 [5] 6 4 [6] 5
HeadNodes [0] 2 1 [1] 3 [2] 3 [3] 1 1 [4] 5 [5] 6 4 [6] 5 7 [7] 6 (c) G4

Sequential Representation of Graph G4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 9 11 13 15 17 18 20 22 23 2 1 3 5 6 4 7

Inverse Adjacency Lists for G3
[0] 1 [1] 1 [2]

Multilists In the adjacency-list representation of an undirected graph, each edge (u, v) is represented by two entries. Multilists: To be able to determine the second entry for a particular edge and mark that edge as having been examined, we use a structure called multilists. Each edge is represented by one node. Each node will be in two lists.

Orthogonal List Representation for G3
head nodes (shown twice) 1 2 1 1 1 1 2 2

Adjacency Multilists for G1
HeadNodes [0] N0 1 N1 N3 edge (0, 1) [1] N1 2 N2 N3 edge (0, 2 [2] [3] N2 3 N4 edge (0, 3) N3 1 2 N4 N5 edge (1, 2) The lists are N4 1 3 N5 edge (1, 3) Vertex 0: N0 -> N1 -> N2 Vertex 1: N0 -> N3 -> N4 2 3 edge (2, 3) N5 Vertex 2: N1 -> N3 -> N5 Vertex 3: N2 -> N4 -> N5

Weighted Edges Very often the edges of a graph have weights associated with them. distance from one vertex to another cost of going from one vertex to an adjacent vertex. To represent weight, we need additional field, weight, in each entry. A graph with weighted edges is called a network.

Graph Operations A general operation on a graph G is to visit all vertices in G that are reachable from a vertex v. Depth-first search Breath-first search

Depth-First Search Starting from vertex, an unvisited vertex w adjacent to v is selected and a depth-first search from w is initiated. When the search operation has reached a vertex u such that all its adjacent vertices have been visited, we back up to the last vertex visited that has an unvisited vertex w adjacent to it and initiate a depth-first search from w again. The above process repeats until no unvisited vertex can be reached from any of the visited vertices.

Graph G and Its Adjacency Lists
1 2 3 4 5 6 HeadNodes 7 [0] 1 2 [1] 3 4 [2] 5 6 [3] 1 7 [4] 1 7 [5] 2 7 [6] 2 7 [7 3 4 5 6

Analysis of DFS If G is represented by its adjacency lists, the DFS time complexity is O(e). If G is represented by its adjacency matrix, then the time complexity to complete DFS is O(n2).

Breath-First Search Starting from a vertex v, visit all unvisited vertices adjacent to vertex v. Unvisited vertices adjacent to these newly visited vertices are then visited, and so on. If an adjacency matrix is used, the BFS complexity is O(n2). If adjacency lists are used, the time complexity of BFS is O(e).

A Complete Graph and Three of Its Spanning Trees

Depth-First and Breath-First Spanning Trees
3 4 1 2 5 6 1 2 3 4 5 6 7 7 (a) DFS (0) spanning tree (b) BFS (0) spanning tree

Spanning Tree Any tree consisting solely of edges in G and including all vertices in G is called a spanning tree. Spanning tree can be obtained by using either a depth-first or a breath-first search. When a nontree edge (v, w) is introduced into any spanning tree T, a cycle is formed. A spanning tree is a minimal subgraph, G’, of G such that V(G’) = V(G), and G’ is connected. (Minimal subgraph is defined as one with the fewest number of edges). Any connected graph with n vertices must have at least n-1 edges, and all connected graphs with n – 1 edges are trees. Therefore, a spanning tree has n – 1 edges.

A Connected Graph and Its Biconnected Components
8 8 9 9 1 7 1 7 7 2 3 5 1 7 4 6 2 3 3 5 5 4 6 (a) A connected graph (b) Its biconnected components

Biconnected Components
Definition: A vertex v of G is an articulation point iff the deletion of v, together with the deletion of all edges incident to v, leaves behind a graph that has at least two connected components. Definition: A biconnected graph is a connected graph that has no articulation points. Definition: A biconnected component of a connected graph G is a maximal biconnected subgraph H of G. By maximal, we mean that G contains no other subgraph that is both biconnected and properly contains H.

Biconnected Components (Cont.)
Two biconnected components of the same graph can have at most one vertex in common. No edge can be in two or more biconnected components. The biconnected components of G partition the edges of G. The biconnected components of a connected, undirected graph G can be found by using any depth-first spanning tree of G. A nontree edge (u, v) is a back edge with respect to a spanning tree T iff either u is an ancestor of v or v is an ancestor of u. A nontree edge that is not back edge is called a cross edge. No graph can have cross edges with respect to any of its depth-first spanning trees.

Biconnected Components (Cont.)
The root of the depth-first spanning tree is an articulation point iff it has at least two children. Any other vertex u is an articulation point iff it has at least one child, w, such that it is not possible to reach an ancestor of u using apath composed solely of w, descendants of w, and a single back edge. Define low(w) as the lowest depth-first number that can be reached fro w using a path of descendants followed by, at most, one back edge.

u is an articulation point iff u is either the root of the spanning tree and has two or more children or u is not the root and u has a child w such that low(w) ≥ dfn(u).

Depth-First Spanning Tree
1 3 5 10 8 9 9 2 6 4 5 4 1 7 8 1 6 3 2 6 7 3 2 3 5 4 1 8 7 4 6 2 7 5 9 8 9 10

dfn and low values for the Spanning Tree
vertex 1 2 3 4 5 6 7 8 9 dfn 10 low

Minimal Cost Spanning Tree
The cost of a spanning tree of a weighted, undirected graph is the sum of the costs (weights) of the edges in the spanning tree. A minimum-cost spanning tree is a spanning tree of least cost. Three greedy-method algorithms available to obtain a minimum-cost spanning tree of a connected, undirected graph. Kruskal’s algorithm Prim’s algorithm Sollin’s algorithm

Kruskal’s Algorithm Kruskal’s algorithm builds a minimum-cost spanning tree T by adding edges to T one at a time. The algorithm selects the edges for inclusion in T in nondecreasing order of their cost. An edge is added to T if it does not form a cycle with the edges that are already in T. Theorem 6.1: Let G be any undirected, connected graph. Kruskal’s algorithm generates a minimum-cost spanning tree.

Stages in Kruskal’s Algorithm
28 1 1 10 1 10 14 16 5 6 2 5 6 2 5 6 2 24 25 18 12 4 4 4 3 22 3 3 (a) (b) (c)

Stages in Kruskal’s Algorithm (Cont.)
1 1 10 1 10 10 14 14 16 5 6 2 5 6 2 5 6 2 12 12 4 12 4 4 3 3 3 (d) (e) (f)

Stages in Kruskal’s Algorithm (Cont.)
1 1 10 10 14 16 14 16 5 6 2 5 6 2 25 12 12 4 4 3 3 22 22 (g) (g)

Prim’s Algorithm Similar to Kruskal’s algorithm, Prim’s algorithm constructs the minimum-cost spanning tree edge by edge. The difference between Prim’s algorithm and Kruskal’s algorithm is that the set of selected edges forms a tree at all times when using Prim’s algorithm while a forest is formed when using Kruskal’s algorithm. In Prim’s algorithm, a least-cost edge (u, v) is added to T such that T∪ {(u, v)} is also a tree. This repeats until T contains n-1 edges. Prim’s algorithm in program 6.7 has a time complexity O(n2).

Stages in Prim’s Alogrithm
1 1 1 10 10 10 5 6 2 5 6 2 5 6 2 25 25 4 4 4 3 3 3 22 (a) (b) (c)

Stages in Prim’s Alogrithm (Cont.)
1 1 1 10 10 10 14 16 16 5 6 2 5 6 2 5 6 2 25 25 25 12 12 12 4 4 4 3 3 3 22 22 22 (d) (e) (f)

Sollin’s Algorithm Contrast to Kruskal’s and Prim’s algorithms, Sollin’s algorithm selects multiple edges at each stage. At the beginning, the selected edges and all the n vertices form a spanning forest. During each stage, an minimum-cost edge is selected for each tree in the forest. It’s possible that two trees in the forest to select the same edge. Only one should be used. Also, it’s possible that the graph has multiple edges with the same cost. So, two trees may select two different edges that connect them together. Again, only one should be retained.

Stages in Sollin’s Algorithm
1 1 10 10 14 14 16 5 6 2 5 6 2 25 12 12 4 4 3 3 22 22 (b) (a)

Shortest Paths Usually, the highway structure can be represented by graphs with vertices representing cities and edges representing sections of highways. Edges may be assigned weights to represent the distance or the average driving time between two cities connected by a highway. Often, for most drivers, it is desirable to find the shortest path from the originating city to the destination city.

Single Source/All Destinations: Nonnegative Edge Costs
Let S denotes the set of vertices to which the shortest paths have already been found. If the next shortest path is to vertex u, then the path begins at v, ends at u, and goes through only vertices that are in S. The destination of the next path generated must be the vertex u that has the minimum distance among all vertices not in S. The vertex u selected in 2) becomes a member of S. The algorithm is first given by Edsger Dijkstra. Therefore, it’s sometimes called Dijstra Algorithm.

Single Source/All Destinations: General Weights
When negative edge lengths are permitted, the graph must not have cycles of negative length. When there are no cycles of negative length, there is a shortest path between any two vertices of an n-vertex graph that has at most n-1 edges on it. If the shortest paht from v to u with at most k, k > 1, edges has no more than k – 1 edges, then distk[u] = distk-1[u]. If the shortest path from v to u with at most k, k > 1, edges has exactly k edges, then it is comprised of a shortest path from v to some vertex j followed by the edge <j, u>. The path from v to j has k – 1 edges, and its length is distk-1[j]. The distance can be computed in recurrence by the following: The algorithm is also referred to as the Bellman and Ford Algorithm.

Graph and Shortest Paths From Vertex 0 to all destinations
50 10 1 2 Path Length 35 15 1) 0, 3 10 10 20 30 20 2) 0, 3, 4 25 3 4 5 3) 0, 3, 4, 1 45 15 3 4) 0, 2 45 (b) Shortest paths from 0 (a) Graph

Diagram for Example 6.5 Boston Chicago 1500 250 1200 1000 San Francisco 800 New York Denver 1400 300 1000 900 1700 1000 Los Angeles New Orleans Miami

Action of Shortest Path
iteration S Vertex selected Distance LA SF DEN CHI BOST NY MIA NO [0] [1] [2] [3] [4] [5] [6] [7] Initial -- --- +∞ 1500 250 1 {4} 5 1250 1150 1650 2 {4,5} 6 3 {4,5,6} 2450 4 {4,5,6,3} 7 3350 {4,5,6,3,7} 3250 {4,5,6,3,7,2} {4,5,6,3,7,2,1}

Directed Graphs 1 2 (a) Directed graph with a negative-length edge 1 2
5 7 -5 1 2 (a) Directed graph with a negative-length edge -2 1 2 1 1 (b) Directed graph with a cycle of negative length

Shortest Paths with Negative Edge Lengths
k distk[7] 1 2 3 4 5 6 ∞ 7 1 4 2 6 3 5 (a) A directed graph (b) distk

All-Pairs Shortest Paths
In all-pairs shortest-path problem, we are to find the shortest paths between all pairs of vertices u and v, u ≠ v. Use n independent single-source/all-destination problems using each of the n vertices of G as a source vertex. Its complexity is O(n3) (or O(n2 logn + ne) if Fibonacci heaps are used). On graphs with negative edges the run time will be O(n4). if adjacency matrices are used and O(n2e) if adjacency lists are used.

All-Pairs Shortest Paths (Cont.)
A simpler algorithm with complexity O(n3) is available. It works faster when G has edges with negative length, as long as the graphs have at least c*n edges for some suitable constant c. An-1[i][j]: the length of the shortest i-to-j path in G Ak[i][j]: the length of the shortest path from I to j going through no intermediate vertex of index greater than k. A-1[i][j]: is just the length[i][j] The shortest path from i to j going through no vertex with index greater than k does not go through the vertex with index k. so its length is Ak-1[i][j]. The shortest path goes through vertex k. The path consists of subpath from i to k and another one from k to j. Ak[i][j] = min{Ak-1[i][j], Ak-1[i][k]+ Ak-1[k][j] }, k ≥ 0

Example for All-Pairs Shortest-Paths Problem
1 2 4 11 6 3 ∞ A0 1 2 4 11 6 3 7 6 1 4 (b) A-1 (c) A0 11 2 3 2 A1 1 2 4 6 3 7 A2 1 2 4 6 5 3 7 (d) A1 (e) A2

Transitive Closure Definition: The transitive closure matrix, denoted A+, of a graph G, is a matrix such that A+[i][j] = 1 if there is a path of length > 0 fromi to j; otherwise, A*[i][j] = 0. Definition: The reflexive transitive closure matrix, denoted A*, of a graph G, is a matrix such that A*[i][j] = 1 if there is a path of length 0 from i to j; otherwise, A*[i][j] = 0.

Graph G and Its Adjacency Matrix A, A+, A*
1 2 3 4 (a) Digraph G (b) Adjacency matrix A (c) A+ (d) A*

Activity-on-Vertex (AOV) Networks
Definition: A directed graph G in which the vertices represent tasks or activities and the edges represent precedence relations between tasks is an activity-on-vertex network or AOV network. Definition: Vertex i in an AOV network G is a predecessor of vertex j iff there is a directed path from vertex i to vertex j. i is an immediate predecessor of j iff <i, j> is an edge in G. If i is a predecessor of j, then j is an successor of i. If i is an immediate predecessor of j, then j is an immediate successor of i.

Activity-on-Vertex (AOV) Networks (Cont.)
Definition: A relation · is transitive iff it is the case that for all triples, i, j, k, i.j and j·k => i·k. A relation · is irreflexive on a set S if for no element x in S it is the case that x·x. A precedence relation that is both transitive and irreflexive is a partial order. Definition: A topological order is a linear ordering of the vertices of a graph such that, for any two vertices I and j, if I is a predecessor of j in the network, then i precedes j in the linear ordering.

An Activity-on-Vertex (AOV) Network
Course number Course name Prerequisites C1 Programming I None C2 Discrete Mathematics C3 Data Structures C1, C2 C4 Calculus I C5 Calculus II C6 Linear Algebra C7 Analysis of Algorithms C3, C6 C8 Assembly Language C9 Operating Systems C7, C8 C10 Programming Languages C11 Compiler Design C12 Artificial Intelligence C13 Computational Theory C14 Parallel Algorithms C15 Numerical Analysis

An Activity-on-Vertex (AOV) Network (Cont.)

Figure 6.36 Action of Program 6.11 on an AOV network
2 4 2 4 2 4 3 5 3 5 5 (a) Initial (b) Vertex 0 deleted (c) Vertex 3 deleted 1 1 4 4 4 5 (d) Vertex 2 deleted (e) Vertex 5 deleted (f) Vertex 1 deleted

Figure 6.37 Internal representation used by topological sorting algorithm
count first data link [0] 1 2 3 [1] 1 4 [2] 1 4 5 [3] 1 5 4 [4] 3 [5] 2

An AOE Network 1 6 start 4 finish 8 2 7 3 5 event interpretation
4 finish 8 a2 = 4 a5 = 1 a11 = 4 2 7 a3 = 5 a9 = 4 3 5 a6 = 2 event interpretation Start of project 1 Completion of activity a1 4 Completion of activities a4 and a5 7 Completion of activities a8 and a9 8 Completion of project

Adjacency lists for Figure 6.38 (a)
count first vertex dur link [0] 1 6 2 4 3 5 [1] 1 4 1 [2] 1 4 1 [3] 1 5 2 [4] 3 6 9 7 7 [5] 2 7 4 [6] 2 8 2 [7] 2 8 4 [8] 2

Computation of ee ee [0] [1] [2] [3] [4] [5] [6] [7] [8] Stack Initial
output 0 6 4 5 [3,2,1] output 3 7 [5,2,1] output 5 11 [2,1] output 2 output 1 output 4 14 [7,6] output 7 16 18 output 6 output 8

Chap 7 Sorting

Motivation of Sorting The term list here is a collection of records.
Each record has one or more fields. Each record has a key to distinguish one record with another. For example, the phone directory is a list. Name, phone number, and even address can be the key, depending on the application or need.

Sorting Two ways to store a collection of records
Sequential Non-sequential Assume a sequential list f. To retrieve a record with key f[i].key from such a list, we can do search in the following order: f[n].key, f[n-1].key, …, f[1].key => sequential search

Example of An Element of A Search List
class Element { public: int getKey() const {return key;}; void setKey(int k) {key = k;}; private: int key; // other records … }

Sequential Search int SeqSearch (Element *f, const int n, const int k)
// Search a list f with key values // f[1].key, …, f[n].key. Return I such // that f[i].key == k. If there is no such record, return 0 { int i = n; f[0].setKey(k); while (f[i].getKey() != k) i--; return i; }

Sequential Search The number of comparisons for a record key i is n – i +1. The average number of comparisons for a successful search is For the phone directory lookup, there should be a better way than this.

Search A binary search only takes O(log n) time to search a sequential list with n records. In fact, if we look up the name start with W in the phone directory, we start search toward the end of the directory rather than the middle. This search method is based on interpolation scheme. An interpolation scheme relies on a ordered list.

Verifying Two Lists With Sequential Search
void Verify1(Element* F1, Element* F2, const int n, const int m) // Compare two unordered lists F1 and F2 of size n and m, respectively { Boolean *marked = new Boolean[m]; for (int i = 1; i <= m; i++) marked[i] = FALSE; for (i = 1; i<= n; i++) int j = SeqSearch(F2, m, F1[i].getKey()); if (j == 0) cout << F1[i].getKey() <<“not in F2 “ << endl; else if (F1[i].other != F2[j].other) cout << “Discrepancy in “<<F[i].getKey()<<“:”<<F1[i].other << “and “ << F2[j].other << endl; marked[j] = TRUE; // marked the record in F2[j] as being seen } for (i = 1; i <= m; i++) if (!marked[i]) cout << F2[i].getKey() <<“not in F1. “ << endl; delete [ ] marked; O(mn)

Fast Verification of Two Lists
void Verify2(Element* F1, Element* F2, const int n, const int m) // Same task as Verfy1. But sort F1 and F2 so that the keys are in // increasing order. Assume the keys in each list are identical { sort(F1, n); sort(F2, m); int i = 1; int j = 1; while ((i <= n) && (j <= m)) switch(compare(F1[i].getKey(), F2[j].getKey())) case ‘<‘: cout<<F1[i].getKey() <<“not in F2”<< endl; i++; break; case ‘=‘: if (F1[i].other != F2[j].other) cout << “Discrepancy in “ << F1[i].getKey()<<“:” <<F1[i].other<<“ and “<<F2[j].other << endl; i++; j++; break; case ‘>’: cout <<F2[j].getKey()<<“ not in F1”<<endl; j++; } if (i <= n)PrintRest(F1, i, n, 1); //print records I through n of F1 else if (j <= m) PrintRest(F2, j, m, 2); // print records j through m of F2 O(max{n log n, m log m})

Formal Description of Sorting
Given a list of records (R1, R2, …, Rn). Each record has a key Ki. The sorting problem is then that of finding permutation, σ, such that Kσ(i) ≤ K σ(i+1) , 1 ≤ i ≤ n – 1. The desired ordering is (Rσ(1), Rσ(2), Rσ(n)).

Formal Description of Sorting (Cont.)
If a list has several key values that are identical, the permutation, σs, is not unique. Let σs be the permutation of the following properties: (1) Kσ(i) ≤ K σ(i+1) , 1 ≤ i ≤ n – 1 (2) If i < j and Ki == Kj in the input list, then Ri precedes Rj in the sorted list. The above sorting method that generates σs is stable.

Categories of Sorting Method
Internal Method: Methods to be used when the list to be sorted is small enough so that the entire sort list can be carried out in the main memory. Insertion sort, quick sort, merge sort, heap sort and radix sort. External Method: Methods to be used on larger lists.

Insert Into A Sorted List
void insert(const Element e, Element* list, int i) // Insert element e with key e.key into the ordered sequence list[0], …, list[i] such that the // resulting sequence is also ordered. Assume that e.key ≥ list[0].key. The array list must // have space allocated for at least i + 2 elements { while (e.getKey() < list[i].getKey()) list[i+1] = list[i]; i--; } list[i+1] = e; O(i)

Insertion Sort void InsertSort(Element* list, const int n)
// Sort list in nondecreasing order of key { list[0].setKey(MININT); for (int j = 2; j <= n; j++) insert(list[j], list, j-1); }

Insertion Sort Example 1
Record Ri is left out of order (LOO) iff Ri < Example 7.1: Assume n = 5 and the input key sequence is 5, 4, 3, 2, 1 j [1] [2] [3] [4] [5] - 5 4 3 2 1

Insertion Sort Example 2
Example 7.2: Assume n = 5 and the input key sequence is 2, 3, 4, 5, 1 j [1] [2] [3] [4] [5] - 2 3 4 5 1 O(1) O(1) O(1) O(n)

Insertion Sort Ananlysis
If there are k LOO records in a list, the computing time for sorting the list via insertion sort is O((k+1)n) = O(kn). Therefore, if k << n, then insertion sort might be a good sorting choice.

Insertion Sort Variations
Binary insertion sort: the number of comparisons in an insertion sort can be reduced if we replace the sequential search by binary search. The number of records moves remains the same. List insertion sort: The elements of the list are represented as a linked list rather than an array. The number of record moves becomes zero because only the link fields require adjustment. However, we must retain the sequential search.

Quick Sort Quick sort is developed by C. A. R. Hoare. The quick sort scheme has the best average behavior among the sorting methods. Quick sort differs from insertion sort in that the pivot key Ki is placed at the correct spot with respect to the whole list. Kj ≤ Ks(i) for j < s(i) and Kj ≥ s(i) for j > s(i). Therefore, the sublist to the left of S(i) and to the right of s(i) can be sorted independently.

Quick Sort void QuickSort(Element* list, const int left, const int right) // Sort records list[left], …, list[right] into nondecreasing order on field key. Key pivot = list[left].key is // arbitrarily chosen as the pivot key. Pointers I and j are used to partition the sublist so that at any time // list[m].key pivot, m < I, and list[m].key pivot, m > j. It is assumed that list[left].key ≤ list[right+1].key. { if (left < right) { int i = left, j = right + 1, pivot = list[left].getKey(); do { do i++; while (list[i].getKey() < pivot); do j--; while (list[j].getKey() > pivot); if (i<j) InterChange(list, i, j); } while (i < j); InterChange(list, left, j); QuickSort(list, left, j–1); QuickSort(list, j+1, right); }

Quick Sort Example Example 7.3: The input list has 10 records with keys (26, 5, 37, 1, 61, 11, 59, 15, 48, 19). R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 Left Right [26 5 37 1 61 11 59 15 48 19 10 [11 15] 26 [59 37] [1 5] [19 2 4 7 [48 [61] 8

Quick Sort (Cont.) In QuickSort(), list[n+1] has been set to have a key at least as large as the remaining keys. Analysis of QuickSort The worst case O(n2) If each time a record is correctly positioned, the sublist of its left is of the same size of the sublist of its right. Assume T(n) is the time taken to sort a list of size n: T(n) ≤ cn + 2T(n/2), for some constant c ≤ ≤ cn + 2(cn/2 +2T(n/4)) ≤ 2cn + 4T(n/4) : ≤ cn log2n + T(1) = O(n logn)

Lemma 7.1 Lemma 7.1: Let Tavg(n) be the expected time for function QuickSort to sort a list with n records. Then there exists a constant k such that Tavg(n) ≤ kn logen for n ≥ 2.

Analysis of Quick Sort Unlike insertion sort (which only needs additional space for a record), quick sort needs stack space to implement the recursion. If the lists split evenly, the maximum recursion depth would be log n and the stack space is of O(log n). The worst case is when the lists split into a left sublist of size n – 1 and a right sublist of size 0 at each level of recursion. In this case, the recursion depth is n, the stack space of O(n). The worst case stack space can be reduced by a factor of 4 by realizing that right sublists of size less than 2 need not be stacked. Asymptotic reduction in stack space can be achieved by sorting smaller sublists first. In this case the additional stack space is at most O(log n).

Quick Sort Variations Quick sort using a median of three: Pick the median of the first, middle, and last keys in the current sublist as the pivot. Thus, pivot = median{Kl, K(l+r)/2, Kr}.

Decision Tree So far both insertion sorting and quick sorting have worst-case complexity of O(n2). If we restrict the question to sorting algorithms in which the only operations permitted on keys are comparisons and interchanges, then O(n logn) is the best possible time. This is done by using a tree that describes the sorting process. Each vertex of the tree represents a key comparison, and the branches indicate the result. Such a tree is called decision tree.

Decision Tree for Insertion Sort
K1 ≤ K2 Yes No K2 ≤ K3 K1 ≤ K3 No Yes No Yes stop K1 ≤ K3 stop K2 ≤ K2 No Yes No IV Yes I stop stop stop stop V II III VI

Decision Tree (Cont.) Theorem 7.1: Any decision tree that sorts n distinct elements has a height of at least log2(n!) + 1 Corollary: Any algorithm that sorts only by comparisons must have a worst-case computing time of Ω(n log n)

Simple Merge void merge(Element* initList, Element* mergeList, const int l, const int m, const int n) { for (int i1 =l,iResult = l, i2 = m+1; i1<=m && i2<=n; iResult++){ if (initList[i1].getKey() <= initList[i2].getKey()) { mergeList[iResult] = initList[i1]; i1++; } else { mergeList[iResult] = initList[i2]; i2++; if (i1 > m) for (int t = i2; t <= n; t++) mergeList[iResult + t - i2] = initList[t]; else for (int t = i1; t <= m; t++) mergeList[iResult + t - i1] = initList[t]; O(n - l + 1)

Analysis of Simple Merge
If an array is used, additional space for n – l +1 records is needed. If linked list is used instead, then additional space for n – l + 1 links is needed.

O(1) Space Merge A second merge algorithm only requires O(1) additional space. Assume total of n records to be merged into a list, where n is a perfect square. And the numbers of records in the left sublist and the right sublist are multiple of

O(1) Space Merge Steps Step 1: Identify the records with largest keys. This is done by following right to left along the two lists to be merged. Step 2: Exchange the records of the second list that were identified in Step 1 with those just to the left of those identified from the first list so that the records with largest keys are contiguous. Step 3: Swap the block of largest with the leftmost block (unless it is already the leftmost block). Sort the rightmost block. Step 4: Reorder the blocks, excluding the block of largest records, into nondecreasing order of the last key in the blocks. Step 5: Perform as many merge substeps as needed to merge the blocks, other than the block with the largest keys. Step 6: Sort the block with the largest keys.

Analysis of O(1) Space Merge
Step 1 and 2 and the swapping of Step 3 each take O( ) time and O(1) space. The sort of Step 3 can be done in O(n) time and O(1) space using an insertion sort. Step 4 can be done in O(n) time and O(1) space using a selection sort. (Selection sort sorts m records using O(m2) key comparisons and O(m) record moves. So it needs O(n) comparisons and the time to move blocks is O(n). If insertion sort is used in Step 4, then the time becomes O(n1.5) since insertion sort needs O(m2) record moves ( records per block * n record moves).

Analysis of O(1) Space Merge (Cont.)
The total number of merge substeps is at most The total time for Step 5 is O(n). The sort of Step 6 can be done in O(n) by using either a selection sort or an insertion sort. Therefore, the total time is O(n) and the additional space used is O(1).

Iterative Merge Sort Treat the input as n sorted lists, each of length 1. Lists are merged by pairs to obtain n/2 lists, each of size 2 (if n is odd, the one list is of length 1). The n/2 lists are then merged by pairs, and so on until we are left with only one list.

Merge Tree 26 5 77 1 61 11 59 15 48 19

Iterative Merge Sort void MergeSort(Element* list, const int n)
// Sort list list into non-decreasing order of the keys list[1].key, …,list[n].key. { Element* tempList = new Element[n+1]; // l is the length of the sublist currently being merged. for (int l = 1; l < n; l *= 2) MergePass(list, tempList, n, l); l *= 2; MergePass(tempList, list, n, l); //interchange role of list and tempList } delete[ ] tempList;

Merge Pass void MergePass(Element* initList, Elemen* resultList, const int n, const int l) // One pass of merge sort. Adjacent pairs of sublists of length l are merged // from list initList to list resultList. n is the number of records in initList { for (int i = 1; i <= n – 2*l + 1; // Are enough elements remaining to form two sublists of length l? i += 2*l) merge(initList, resultList, i, i + l - 1, i + 2*l – 1); // merge remaining list of length < 2 * l if ((i + l – 1) < n) merge(initList, resultList, i, i+l–1, n); else for (int t = i; t <= n; t++) resultList[t] = initList[t]; }

Analysis of MergeSort Total of passes are made over the data. Each pass of merge sort takes O(n) time. The total of computing time is O(n log n)

Recursive Merge Sort Recursive merge sort divides the list to be sorted into two roughly equal parts: the left sublist [left : (left+right)/2] the right sublist [(left+right)/2 +1 : right] These sublists are sorted recursively, and the sorted sublists are merged. To avoid copying, the use of a linked list (integer instead of real link) for sublist is desirable.

Sublist Partitioning For Recursive Merge Sort
26 5 77 1 61 11 59 15 48 19

Program 7.11 (Recursive Merge Sort )
class Element { private: int key; Field other; int link; public: Element() {link = 0;}; }; int rMergeSort(Element* list, const int left, const int right) // List list = (list[left], …, list[right]) is to be sorted on the field key. // link is a link field in each record that is initially 0 // list[0] is a record for intermediate results used only in ListMerge if (left >= right) return left; int mid = (left + right)/2; return ListMerge(list, rMergeSort(list, left, mid), rMergeSort(list, mid+1, right)); } integer array is used. link is integer not pointer here. O(n log n)

Program 7.12 (Merging Linked Lists)
int ListMerge(Element* list, const int start1, const int start2) { int iResult = 0; for (int i1 = start1, i2 = start2; i1 && i2;){ if (list[i1].key <= list[i2].key) { list[iResult].link = i1; iResult = i1; i1 = list[i1].link; } else { list[iResult].link = i2; iResult = i2; i2 = list[i2].link; // move remainder if (i1 == 0) list[iResult].link = i2; else list[iResult] = i1; return list[0].link;

Natural Merge Sort Natural merge sort takes advantage of the prevailing order within the list before performing merge sort. It runs an initial pass over the data to determine the sublists of records that are in order. Then it uses the sublists for the merge sort.

Natural Merge Sort Example
26 19

Heap Sort Merge sort needs additional storage space proportional to the number of records in the file being sorted, even though its computing time is O(n log n) O(1) merge only needs O(1) space but the sorting algorithm is much slower. We will see that heap sort only requires a fixed amount of additional storage and achieves worst-case and average computing time O(n log n). Heap sort uses the max-heap structure.

Heap Sort (Cont.) For heap sort, first of all, the n records are inserted into an empty heap. Next, the records are extracted from the heap one at a time. With the use of a special function adjust(), we can create a heap of n records faster.

Program 7.13 (Adjusting A Max Heap)
void adjust (Element* tree, const int root, const int n) // Adjust the binary tree with root root to satisfy the heap property. The left and right subtrees of root // already satisfy the heap property. No node has index greater than n. { Element e = tree[root]; int k = e.getKey(); for (int j = 2*root; j <= n; j *= 2) if (j < n) if (tree[j].getKey() < tree[j+1].getKey()) j++; // compare max child with k. If k is max, then done. if (k >= tree[j].getKey()) break; tree[j/2] = tree[j]; // move jth record up the tree } tree[j/2] = e;

Program 7.14 (Heap Sort) O(n log n)
void HeapSort (Element* list, const int n) // The list list = (list[1], …, list[n]) is sorted into nondecreasing order of the field key. { for (int i = n/2; i >= 1; i--) // convert list into a heap adjust(list, i, n); for (i = n-1; i >= 1; i--) // sort list Element t = list[i+1]; // interchange list1 and list i+1 list[i+1] = list[1]; list[1] = t; adjust(list, 1, i); } O(n log n)

Converting An Array Into A Max Heap
[1] 26 [1] 77 [2] 5 77 [3] [2] 61 [3] 59 [4] 1 [5] 61 11 59 48 [4] [5] 19 11 26 [6] [7] [6] [7] 15 48 19 15 1 5 [8] [9] [10] [8] [9] [10] (b) Initial heap (a) Input array

Heap Sort Example [1] 61 [1] 59 [2] 48 [3] 59 [2] 48 [3] 26 [4] 15 [5]
19 11 26 15 [5] 19 11 1 [6] [7] [6] [7] 5 1 5 [8] [9] [8] Heap size = 9, Sorted = [77] Heap size = 8, Sorted = [61, 77]

Sorting Several Keys A list of records are said to be sorted with respect to the keys K1, K2, …, Kr iff for every pair of records i and j, i < j and (K1i, K2i, …, Kri) ≤ (K1j, K2j, …, Krj). The r-tuple (x1, x2, …, xr) is less than or equal to the r-tuple (y1, y2, …, yr) iff either xi = yi, 1 ≤ i ≤ j, and xj+1 < yj+1 for some j < r or xi = yi , 1 ≤ i ≤ r. Two popular ways to sort on multiple keys sort on the most significant key into multiple piles. For each pile, sort on the second significant key, and so on. Then piles are combined. This method is called sort on most-significant-digit-first (MSD). The other way is to sort on the least significant digit first (LSD). Example, sorting a deck of cards: suite and face value. Spade > Heart > Diamond > Club

Sorting Several Keys (Cont.)
LSD and MSD only defines the order in which the keys are to be sorted. But they do not specify how each key is sorted. Bin sort can be used for sorting on each key. The complexity of bin sort is O(n). LSD and MSD can be used even when there is only one key. E.g., if the keys are numeric, then each decimal digit may be regarded as a subkey. => Radix sort.

Radix Sort In a radix sort, we decompose the sort key using some radix r. The number of bins needed is r. Assume the records R1, R2, …, Rn to be sorted based on a radix of r. Each key has d digits in the range of 0 to r-1. Assume each record has a link field. Then the records in the same bin are linked together into a chain: f[i], 0 ≤ i ≤ r (the pointer to the first record in bin i) e[i], (the pointer to the end record in bin i) The chain will operate as a queue. Each record is assumed to have an array key[d], 0 ≤ key[i] ≤ r, 0 ≤ i ≤ d.

Program 7.15 (LSD Radix Sort)
void RadixSort (Element* list, const int d, const int n) { int e[radix], f[radix]; // queue pointers for (int i = 1; i <= n; i++) list[i].link = i + 1; // link into a chain starting at current list[n].link = 0; int current = 1; for (i = d – 1; i >= 0; i--) // sort on key key[i] for (int j = 0; j < radix; j++) f[j] = 0; // initialize bins to empty queues for (; current; current = list[current].link) { // put records into queues int k = list[current].key[i]; if (f[k] == 0) f[k] = current; else list[e[k]].link = current; e[k] = current; } for (j = 0; f[j] == 0; j++); // find the first non-empty queue current = f[j]; int last = e[j]; for (int k = j+1; k < radix; k++){ // concatenate remaining queues if (f[k]){ list[last].link = f[k]; last = e[k]; list[last].link = 0; d passes O(n) O(r) O(d(n+r))

Radix Sort Example list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10] 179 208 306 93 859 984 55 9 271 33 e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9] 9 33 859 271 93 984 55 306 208 179 f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9] list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10] 271 93 33 984 55 306 208 179 859 9

Radix Sort Example (Cont.)
list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10] 271 93 33 984 55 306 208 179 859 9 e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9] 9 208 859 179 306 33 55 271 984 93 f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9] list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10] 306 208 9 33 55 859 271 179 984 93

Radix Sort Example (Cont.)
list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10] 306 208 9 33 55 859 271 179 984 93 e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9] 93 55 33 271 9 179 208 306 859 984 f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9] list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10] 9 33 55 93 179 208 271 306 859 948

List And Table Sorts Apart from radix sort and recursive merge sort, all the sorting methods we have looked at so far require excessive data movement. When the amount of data to be sorted is large, data movement tends to slow down the process. It is desirable to modify sorting algorithms to minimize the data movement. Methods such as insertion sort or merge sort can be modified to work with a linked list rather than a sequential list. Instead of physical movement, an additional link field is used to reflect the change in the position of the record in the list.

Program 7.16 (Rearranging Records Using A Doubly Linked List
void list (Element* list, const int n, int first) { int prev = 0; for (int current = first; current; current = list[current].link) // convert chain into a doubly linked list list[current].linkb = prev; prev = current; } for (int i = 1; i < n; i++) // move listfirst to position i while maintaining the list if (first != i) { if (list[i].link) list[list[i].link].linkb = first; list[list[i].linkb].link = first; Element a = list[first]; list[first] = list[i];list[i] = a; first = list[i].link; O(n) Assume each record is m words O(nm)

Example 7.9 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R1 R2 R3 R4 R5 R6 R7 R8 R9
i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 key 26 5 77 1 61 11 59 15 48 19 link 9 6 2 3 8 10 7 i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 key 26 5 77 1 61 11 59 15 48 19 link 9 6 2 3 8 10 7 linkb 4

Example 7.9 (Cont.) R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R1 R2 R3 R4 R5 R6
i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 key 1 5 77 26 61 11 59 15 48 19 link 2 6 9 3 8 10 7 4 linkb Configuration after first iteration of the for loop of list1, first = 2 i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 key 1 5 77 26 61 11 59 15 48 19 link 2 6 9 3 8 10 7 linkb 4 Configuration after second iteration of the for loop of list1, first = 6

Example 7.9 (Cont.) R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R1 R2 R3 R4 R5 R6
i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 key 1 5 11 26 61 77 59 15 48 19 link 2 6 8 9 10 7 4 linkb Configuration after third iteration of the for loop of list1, first = 8 i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 key 1 5 11 15 61 77 59 26 48 19 link 2 6 8 10 9 7 linkb 4 Configuration after fourth iteration of the for loop of list1, first = 10

Example 7.10 (Rearranging Records Using Only One Link)
void list2(Element* list, const int n, int first) // Same function as list1 except that a second link field linkb is not required { for (int i = 1; i < n; i++) while (first < i ) first = list[first].link; int q = list[first].link; // listq is next record in nondecreasing order if (first != i) // interchange listi and listfirst moving listfirst to its correct spot as listfirst has ith smallest key. // Also set link from old position of listi to new one Element t = list[i]; list[i] = list[first]; list[first] = t; list[i].link = first; } first = q; O(n) O(nm)

key 1 5 77 26 61 11 59 15 48 19 link 4 6 9 3 8 10 7 Configuration after first iteration of the for loop of list1, first = 2 i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 key 1 5 77 26 61 11 59 15 48 19 link 4 6 9 3 8 10 7 Configuration after second iteration of the for loop of list1, first = 6

key 1 5 11 26 61 77 59 15 48 19 link 4 6 9 3 10 7 Configuration after third iteration of the for loop of list1, first = 8 i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 key 1 5 11 15 61 77 59 26 48 19 link 4 6 8 3 9 7 Configuration after fourth iteration of the for loop of list1, first = 10

key 1 5 11 15 19 77 59 26 48 61 link 4 6 8 10 9 7 3 Configuration after fifth iteration of the for loop of list1, first = 1 i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 key 1 5 11 15 19 26 59 77 48 61 link 4 6 8 7 3 Configuration after sixth iteration of the for loop of list1, first = 9

Table Sort The list-sort technique is not well suited for quick sort and heap sort. One can maintain an auxiliary table, t, with one entry per record. The entries serve as an indirect reference to the records. Initially, t[i] = i. When interchanges are required, only the table entries are exchanged. It may be necessary to physically rearrange the records according to the permutation specified by t sometimes.

Table Sort (Cont.) The function to rearrange records corresponding to the permutation t[1], t[2], …, t[n] can be considered as an application of a theorem from mathematics: Every permutation is made up of disjoint cycles. The cycle for any element i is made up of i, t[i], t2[i], …, tk[i], where tj[i]=t[tj-1[i]], t0[i]=i, tk[i]=i.

Program 7.18 (Table Sort) void table(Element* list, const int n, int *t) { for (int i = 1; i < n; i++) { if (t[i] != i) { Element p = list[i]; int j = i; do { int k = t[j]; list[j] = list[k]; t[j] = j; j = k; } while (t[j] != i) list[j] = p; t[j] = j; }

Table Sort Example R1 R2 R3 R4 R5 R6 R7 R8 key t key t key t
35 14 12 42 26 50 31 18 t 3 2 8 5 7 1 4 6 Initial configuration 1 2 3 key 12 14 18 42 26 35 31 50 t 1 2 3 5 7 6 4 8 Configuration after rearrangement of first cycle key 12 14 18 26 31 35 42 50 t 1 2 3 4 5 6 7 8 Configuration after rearrangement of second cycle

Analysis of Table Sort To rearrange a nontrivial cycle including k distinct records one needs k+1 moves. Total of record moves Since the records on all nontrivial cycles must be different, then The total number of record moves is maximum when and there are cycles. Assume that one record move costs O(m) time, the total computing time is O(mn).

Summary Of Internal Sorting
No one method is best for all conditions. Insertion sort is good when the list is already partially ordered. And it is best for small number of n. Merge sort has the best worst-case behavior but needs more storage than heap sort. Quick sort has the best average behavior, but its worst-case behavior is O(n2). The behavior of radix sort depends on the size of the keys and the choice of r.

External Sorting There are some lists that are too large to fit in the memory of a computer. So internal sorting is not possible. Some records are stored in the disk (tape, etc.). System retrieves a block of data from a disk at a time. A block contains multiple records. The most popular method for sorting on external storage devices is merge sort. Segments of the input list are sorted. Sorted segments (called runs) are written onto external storage. Runs are merged until only run is left.

Example 7.12 Consider a computer which is capable of sorting 750 records is used to sort 4500 records. Six runs are generated with each run sorting 750 records. Allocate three 250-record blocks of internal memory for performing merging runs. Two for input run 2 runs and the last one is for output. Three factors contributing to the read/write time of disk: seek time latency time transmission time

Example 7.12 (Cont.) run3 (1501 – 2250) run2 (751 – 1500)

Example 7.12 (Cont.) tIO = ts + tl +trw
tIS = time to internal sort 750 records ntm = time to merge n records from input buffers to the output buffer ts = maximum seek time tl = maximum latency time trw = time to read or write on block of 250 records

Example 7.12 (Cont.) Operation Time (1) (2) (3) (4) Total time
Read 18 blocks of input, 18tIO, internally sort, 6tIS , write 18 blocks, 18tIO 36tIO + 6tIS (2) Merge runs 1 to 6 in pairs 36tIO tm (3) Merge two runs fo 1500 records each, 12 blocks 24tIO tm (4) Merge one run of 3000 records with one run of 1500 records Total time 132tIO tm+ 6tIS

K-Way Merging To merge m runs via 2-way merging will need passes.
If we use higher order merge, the number of passes over would be reduced. With k-way merge on m runs, we need passes over. But is it always true that the higher order of merging, the less computing time we will have? Not necessary! k-1 comparisons are needed to determine the next output. If loser tree is used to reduce the number of comparisons, we can achieve complexity of O(n log2 m) The data block size reduced as k increases. Reduced block size implies the increase of data passes over

Buffer Handling for Parallel Operation
To achieve better performance, multiple input buffers and two output buffers are used to avoid idle time. Evenly distributing input buffers among all runs may still have idle time problem. Buffers should be dynamically assigned to whoever needs to retrieve more data to avoid halting the computing process. We should take advantage of task overlapping and keep computing process busy and avoid idle time.

Buffering Algorithm Step 1: Input the first block of each of the k runs, setting up k linked queues, each having one block of data. Step 2: Let LastKey[i] be the last key input from run i. Let NextRun be the run for which LastKey is minimum. Step 3: Use a function kwaymerge to merge records from the k input queues into the output buffer. Step 4: Wait for any ongoing disk I/O to complete. Step 5: If an input buffer has been read, add it to the queue for the appropriate run. Step 6: If LastKey[NextRun] != +infinity, then initiate reading the next block from run NextRun into a free input buffer. Step 7: Initiate the writing of output buffer. Step 8: If a record with key +infinity has not been merged into the output buffer, go back to Step 3. Otherwise, wait for the ongoing write to complete and then terminate.

Optimal Merging of Runs
26 26 11 15 6 20 6 5 2 4 5 15 2 4 weighted external path length = 2*3 + 4*3 + 5*2 + 15*1 = 43 weighted external path length = 2*2 + 4*2 + 5*2 + 15*2 = 52

Huffman Code Assume we want to obtain an optimal set of codes for messages M1, M2, …, Mn+1. Each code is a binary string that will be used for transmission of the corresponding message. At receiving end, a decode tree is used to decode the binary string and get back the message. A zero is interpreted as a left branch and a one as a right branch. These codes are called Huffman codes. The cost of decoding a code word is proportional to the number bits in the code. This number is equal to the distance of the corresponding external node from the root node. If qi is the relative frequency with which message Mi will be transmitted, then the expected decoding time is where di is the distance of the external node for message Mi from the root node.

Huffman Codes (Cont.) The expected decoding time is minimized by choosing code words resulting in a decode tree with minimal weighted external path length. 1 M4 1 M3 1 M1 M2

Huffman Function O(nlog n) class BinaryTree { public:
BinaryTree(BinaryTree bt1, BinaryTree bt2) { root->LeftChild = bt1.root; root->RightChild = bt2.root; root->weight = bt1.root->weight + bt2.root->weight; } private: BinaryTreeNode *root; void huffman (List<BinaryTree> l) // l is a list of single node binary trees as decribed above { int n = l.Size(); // number of binary trees in l for (int i = 0; i < n-1; i++) { // loop n-1 times BinaryTree first = l.DeleteMinWeight(); BinaryTree second = l.DeleteMinWeight(); BinaryTree *bt = new BinaryTree(first, second); l.Insert(bt); O(nlog n)

Huffman Tree Example 5 10 16 5 9 7 2 3 5 (c) 2 3 (a) 39 (b) 23 16 23
13 10 9 7 13 5 5 5 5 2 3 (d) (e) 2 3

Chap 8 Hashing

Symbol Table Symbol table is used widely in many applications.
dictionary is a kind of symbol table data dictionary is database management In general, the following operations are performed on a symbol table determine if a particular name is in the table retrieve the attribute of that name modify the attributes of that name insert a new name and its attributes delete a name and its attributes

Symbol Table (Cont.) Popular operations on a symbol table include search, insertion, and deletion A binary search tree could be used to represent a symbol table. The complexities for the operations are O(n). A technique called hashing can provide a good performance for search, insert, and delete. Instead of using comparisons to perform search, hashing relies on a formula called the hash function. Hashing can be divided into static hashing and dynamic hashing

Static Hashing Identifiers are stored in a fixed-size table called hash table. The address of location of an identifier, x, is obtained by computing some arithmetic function h(x). The memory available to maintain the symbol table (hash table) is assumed to be sequential. The hash table consists of b buckets and each bucket contains s records. h(x) maps the set of possible identifiers onto the integers 0 through b-1. If the identifier is 6 alpha-numerical long, with the first one being a letter. Then, the total of distinct identifiers are

Hash Tables Definition: The identifier density of a hash table is the ratio n/T, where n is the number of identifiers in the table and T is the total number of possible identifiers. The loading density or loading factor of a hash table is α= n/(sb).

Hash Tables (Cont.) Two identifiers, I1, and I2, are said to be synonyms with respect to h if h(I1) = h(I2). An overflow occurs when a new identifier i is mapped or hashed by h into a full bucket. A collision occurs when two non-identical identifiers are hashed into the same bucket. If the bucket size is 1, collisions and overflows occur at the same time.

Example 8.1 Slot 1 Slot 2 A A2 If no overflow occur, the time required for hashing depends only on the time required to compute the hash function h. 1 2 3 D 4 5 Large number of collisions and overflows! 6 GA G 25

Hash Function A hash function, h, transforms an identifier, x, into a bucket address in the hash table. Ideally, the hashing function should be both easy to compute and results in very few collisions. Also because the size of the identifier space, T, is usually several orders of magnitude larger than the number of buckets, b, and s is small, overflows necessarily occur. Hence, a mechanism to handle overflow is needed.

Hash Function (Cont.) Generally, a hash function should not have bias on the use of the hash table. A uniform hash function supports that a random x has an equal chance of hashing into any of the b buckets

Mid-Square Mid-Square function, hm, is computed by squaring the identifier and then using an appropriate number of bits from the middle of the square to obtain the bucket address. Since the middle bits of the square usually depend on all the characters in the identifier, different identifiers are expected to result in different hash addresses with high probability.

Division Another simple hash function is using the modulo (%) operator. An identifier x is divided by some number M and the remainder is used as the hash address for x. The bucket addresses are in the range of 0 through M-1. If M is a power of 2, then hD(x) depends only on the least significant bits of x.

Division (Cont.) If identifiers are stored right-justified with leading zeros and M = 2i, i ≤ 6, the identifiers A1, B1, C1, etc., all have the same bucket. Because programmers tend to use many variables with the same suffix, the choice of M as a power of two would result in many collisions. If left-justified is used, all one-character identifiers would map to the same bucket, 0, for M = 2i, i ≤ 54; all two character identifiers would map to the bucket 0 for M = 2i, i ≤ 48. If a division function hD is used as the hash function, the table size should not be a power of two. If M is divisible by two, the odd keys are mapped to odd buckets and even keys are mapped to even buckets. Thus, the hash table is biased.

Division (Cont.) Let x=x1x2 and y = x2x1 be two identifiers. If internal binary representation of x1 is C(x1) and that for x2 has value C(x2), then if each character is represented by six bits, the numeric value of x is 26C(x1) +C(x2), and that for y is 26C(x2)+C(x1). If p is a prime number that divides M, then If p =3, then

Division (Cont.) Program in which many variables are permutations of each other would again result in a biased use of the table and hence result in many collisions. In the previous example, 64%3 =1 and 64%7=1. To avoid the above problem, M needs to be a prime number. Then, the only factors of M are M and 1.

Folding The identifier x is partitioned into several parts, all but the last being of the same length. All partitions are added together to obtain the hash address for x. Shift folding: different partitions are added together to get h(x). Folding at the boundaries: identifier is folded at the partition boundaries, and digits falling into the same position are added together to obtain h(x). This is similar to reversing every other partition and then adding.

Example 8.2 x= are partitioned into three decimal digits long. P1 = 123, P2 = 203, P3 = 241, P4 = 112, P5 = 20. Shift folding: Folding at the boundaries: 123 203 241 112 20 Folding 1 time 123 203 211 142 Folding 2 times 123 302 h(x) = = 897 211 241

Digit Analysis This method is useful when a static file where all the identifiers in the table are known in advance. Each identifier x is interpreted as a number using some radix r. The digits of each identifier are examined. Digits having most skewed distributions are deleted. Enough digits are deleted so that the number of remaining digits is small enough to give an address in the range of the hash table.

Overflow Handling There are two ways to handle overflow:
Open addressing Chaining

Open Addressing Assumes the hash table is an array
The hash table is initialized so that each slot contains the null identifier. When a new identifier is hashed into a full bucket, find the closest unfilled bucket. This is called linear probing or linear open addressing

Example 8.3 Assume 26-bucket table with one slot per bucket and the following identifiers: GA, D, A, G, L, A2, A1, A3, A4, Z, ZA, E. Let the hash function h(x) = first character of x. When entering G, G collides with GA and is entered at ht[7] instead of ht[6]. A 1 A2 2 A1 3 D 4 A3 5 A4 6 GA 7 G 8 ZA 9 E 25

Open Addressing (Cont.)
When linear open address is used to handle overflows, a hash table search for identifier x proceeds as follows: compute h(x) examine identifiers at positions ht[h(x)], ht[h(x)+1], …, ht[h(x)+j], in this order until one of the following condition happens: ht[h(x)+j]=x; in this case x is found ht[h(x)+j] is null; x is not in the table We return to the starting position h(x); the table is full and x is not in the table

Linear Probing In example 8.3, we found that when linear probing is used to resolve overflows, identifiers tend to cluster together. Also adjacent clusters tend to coalesce. This increases search time. e.g., to find ZA, you need to examine ht[25], ht[0], …, ht[8] (total of 10 comparisons). To retrieve each identifier once, a total of 39 buckets are examined (average 3.25 bucket per identifier). The expected average number of identifier comparisons, P, to look up an identifier is approximately (2 -α)/(2-2α), whereαis the loading density. Example 8.3 α=12/26=.47 and P = 1.5. Even though the average number of probes is small, the worst case can be quite large. A 1 A2 2 A1 3 D 4 A3 5 A4 6 GA 7 G 8 ZA 9 E 25 Z

Quadratic Probing One of the problems of linear open addressing is that it tends to create clusters of identifiers. These clusters tend to merge as more identifiers are entered, leading to bigger clusters. A quadratic probing scheme improves the growth of clusters. A quadratic function of i is used as the increment when searching through buckets. Perform search by examining bucket h(x), (h(x)+i2)%b, (h(x)-i2)%b for 1 ≤ i ≤ (b-1)/2. When b is a prime number of the form 4j+3, for j an integer, the quadratic search examine every bucket in the table.

Rehashing Another way to control the growth of clusters is to use a series of hash functions h1, h2, …, hm. This is called rehashing. Buckets hi(x), 1 ≤ i ≤ m are examined in that order.

Chaining We have seen that linear probing perform poorly because the search for an identifier involves comparisons with identifiers that have different hash values. e.g., search of ZA involves comparisons with the buckets ht[0] – ht[7] which are not possible of colliding with ZA. Unnecessary comparisons can be avoided if all the synonyms are put in the same list, where one list per bucket. As the size of the list is unknown before hand, it is best to use linked chain. Each chain has a head node. Head nodes are stored sequentially.

Hash Chain Example ht A4 A3 A1 A2 A 1 2 3 D 4 E 5 6 G GA 7 8
A4 A3 A1 A2 A 1 2 3 D 4 E 5 6 G GA 7 8 Average search length is (6*1+3*2+1*3+1*4+1*5)/12 = 2 9 10 11 L 25 ZA Z

Chaining (Cont.) The expected number of identifier comparisons can be shown to be ~ 1 +α/2, where αis the loading density n/b (b=number of head nodes). For α=0.5, it’s And if α=1, then it’s about 1.5. Another advantage of this scheme is that only the b head nodes must be sequential and reserved at the beginning. The scheme only allocates other nodes when they are needed. This could reduce overall space requirement for some load densities, despite of links.

Hash Functions Theoretically, the performance of a hash table depends only on the method used to handle overflows and is independent of the hash function as long as an uniform hash function is used. In reality, there is a tendency to make a biased use of identifiers. Many identifiers in use have a common suffix or prefix or are simple permutations of other identifiers. Therefore, different hash functions would give different performance.

Average Number of Bucket Accesses Per Identifier Retrieved
a = n/b 0.50 0.75 0.90 0.95 Hash Function Chain Open Chain Open Chain Open Chain Open mid square division shift fold bound fold digit analysis theoretical

Theoretical Evaluation of Overflow Techniques
In general, hashing provides pretty good performance over conventional techniques such as balanced tree. However, the worst-case performance of hashing can be O(n). Let ht[0..b-1] be a hash table with b buckets. If n identifiers x1, x2, …, xn are entered into the hash table, then there are bn distinct hash sequences h(x1), h(x2), …, h(xn).

Theoretical Evaluation of Overflow Techniques (Cont.)
Sn is the average number of comparisons neede to find the jth key xj, averaged over 1 ≤ j ≤ n, with each j equally likely, and averaged over all bn hash sequences, assuming each of these also to be equally likely. Un is the expected number of identifier comparisons when a search is made for an identifier not in the hash table.

Theorem 8.1 Theorem 8.1: Let α=n/b be the loading density of a hash table using a uniform hashing function h. Then for linear open addressing for rehashing, random probing, and quadratic probing for chaining

Dynamic Hashing The purpose of dynamic hashing is to retain the fast retrieval time of conventional hashing while extending the technique so that it can accommodate dynamically increasing and decreasing file size without penalty. Assume a file F is a collection of records R. Each record has a key field K by which it is identified. Records are stored in pages or buckets whose capacity is p. The goal of dynamic hashing is to minimize access to pages. The measure of space utilization is the ratio of the number of records, n, divided by the total space, mp, where m is the number of pages.

Dynamic Hashing Using Directories
Given a list of identifiers in the following: Now put these identifiers into a table of four pages. Each page can hold at most two identifiers, and each page is indexed by two-bit sequence 00, 01, 10, 11. Now place A0, B0, C2, A1, B1, and C3 in a binary tree, called Trie, which is branching based on the last significant bit at root. If the bit is 0, the upper branch is taken; otherwise, the lower branch is taken. Repeat this for next least significant bit for the next level. Identifiers Binary representation A0 A1 B0 B1 C0 C1 C2 C3 C5

A Trie To Hold Identifiers
A0, B0 A0, B0 1 C2 1 C2 A1, B1 A1, B1 1 1 C5 1 1 C3 1 C3 A0, B0 (a) two-level trie on four pages (b) inserting C5 with overflow 1 C2 A1, C1 1 B1 1 (c) inserting C1 with overflow C5 1 1 C3

Issues of Trie Representation
From the example, we find that two major factors that affects the retrieval time. Access time for a page depends on the number of bits needed to distinguish the identifiers. If identifiers have a skewed distribution, the tree is also skewed.

Extensible Hashing Fagin et al. present a method, called extensible hashing, for solving the above issues. A hash function is used to avoid skewed distribution. The function takes the key and produces a random set of binary digits. To avoid long search down the trie, the trie is mapped to a directory, where a directory is a table of pointers. If k bits are needed to distinguish the identifiers, the directory has 2k entries indexed 0, 1, …, 2k-1 Each entry contains a pointer to a page.

Trie Collapsed Into Directories
0000 A0, B0 00 A0, B0 000 A0, B0 c c c 0001 A1, C1 b 01 A1, B1 001 A1, B1 0010 C2 b b f 0011 C3 10 C2 010 C2 a d e 0100 011 C3 e 11 C3 0101 C5 a b 100 0110 d f 0111 101 C5 a b 1000 d 110 1001 B1 c b 111 1010 f 1011 a 1100 e 1101 b 1110 f 1111 (a) 2 bits (b) 3 bits © 4 bits

Hashing With Directory
Using a directory to represent a trie allows table of identifiers to grow and shrink dynamically. Accessing any page only requires two steps: First step: use the hash function to find the address of the directory entry. Second step: retrieve the page associated with the address Since the keys are not uniformly divided among the pages, the directory can grow quite large. To avoid the above issue, translate the bits into a random sequence using a uniform hash function. So identifiers can be distributed uniformly across the entries of directory. In this case, multiple hash functions are needed.

Overflow Handling A simple hash function family is simply adding a leading zero or one as the new leading bit of the result. When a page identified by i bits overflows, a new page needs to be allocated and the identifiers are rehashed into those two pages. Identifiers in both pages should have their low-order I bits in common. These are referred as buddies. If the number of identifiers in buddy pages is no more than the capacity of one page, then they can be coalesce into one page. After adding a bit, if the number of bits used is greater than the depth of the directory, the whole directory doubles in size and its depth increases by 1.

Program 8.5 Extensible Hashing
const int WordSize = 5; // maximum no. of directory bits const int PageSize = 10; // maximum size of a page struct TwoChars { char str[2];}; struct page { int LocalDepth; // no. of bits to distinguish ids TwoChars names[PageSize]; //the actual identifiers int NumIdents; // no. of identifiers in this page }; typedef page* paddr; struct record { // a sample record TwoChars KeyField; int IntData; char CharData; paddr rdirectory[MaxDir]; // will contain pointers to pages int gdepth; // not to exceed WordSize

Program 8.5 Extensible Hashing (Cont.)
paddr hash(const TwoChars& key, const int precision); // key is hashed using a uniform hash function, and the low order precision bits are returned //as the page address paddr buddy(const paddr index); // Take an address of a page and return the page’s buddy; i.e., the leading bit it complemented int size(const paddr ptr); // Return the number of identifiers in the page paddr coalesce(const paddr ptr, const paddr buddy); // Combine page ptr and buddy, buddy into a single page Boolean PageSearch(const TwoChars& key, const paddr index); // Search page index for key key. If found, return TRUE; otherwise, return FALSE. int convert(const paddr p); // Convert a pointer to a page to an equivalent integer

void enter(const record r, const paddr p); // Insert the new record r into the page pointed at by p void PageDelete(const TwoChars& key, const paddr p); // Remove the record with key key from the page pointed at by p paddr find(const TwoChars& key); // Search for a record with key key in the file. If found, return the address of the page in which it was // found. If not return 0. { paddr index = hash(key, gdepth); int IntIndex = convert(index); padd ptr = rdirectory[IntIndex]; if (PageSearch(key, ptr) retrun ptr; else return 0; }

void insert(const record& r, const TwoChars& key) // Insert a new record into the file pointed at by the directory { paddr p = find(key); // check if key is present if(p) return; // key already in if(p->NumIdents != PageSize) { // page not full enter(r, p); p->NumIdents ++; } else { Split the page into two, insert the new key, and update gdepth if necessary; if this causes gdepth to exceed WordSize, print an error and terminate. void Delete(const TwoChars& key) //Find and delete the record with key key paddr p = find(key); if(p) { PageDelete(key, p); if ((size(p) + size(buddy(p)) <= PageSize) coalesce(p, buddy(p));

Analysis of Directory-Based Dynamic Hashing
The most important of the directory version of extensible hashing is that it guarantees only two disk accesses for any page retrieval. We get the performance at the expense of space. This is because a lot of pointers could point to the same page. One of the criteria to evaluate a hashing function is the space utilization. Space utilization is defined as the ratio of the number of records stored in the table divided by the total number of space allocated. Research has shown that dynamic hashing has 69% of space utilization if no special overflow mechanisms are used.

Directoryless Dynamic Hashing
Directoryless hashing (or linear hashing) assume a continuous address space in the memory to hold all the records. Therefore, the directory is not needed. Thus, the hash function must produce the actual address of a page containing the key. Contrasting to the directory scheme in which a single page might be pointed at by several directory entries, in the directoryless scheme one unique page is assigned to every possible address.

A Trie Mapped To Directoryless, Continuous Storage
A0, B0 00 A0 B0 01 1 C2 C2 - 10 A1 A1, B1 1 B1 11 C3 1 C3 -

Directoryless Scheme Overflow Handling
00 A0 000 A0 000 A0 B0 B0 B0 overflow page 01 C2 001 C2 001 C2 - - - 10 A1 010 A1 010 A1 B1 B1 C5 B1 C1 C5 11 C3 011 C3 011 C3 - - - 100 - 100 - - new page - new page 101 - -

Figure 8.14 The rth Phase of Expansion of Directoryless Method
pages already split pages not yet split pages added so far addresses by r+1 bits addresses by r bits addresses by r+1 bits q r 2r pages at start

Analysis of Directoryless Hashing
The advantage of this scheme is that for many retrievals the time is one access for those identifiers that are in the page directly addressed by the hash function. The problem is that for others, substantially more than two accesses might be required as one moves along the overflow chain. Also when a new page is added and the identifiers split across the two pages, all identifiers including the overflows are rehashed. Hence, the space utilization is not good, about 60%. (shown by Litwin).

Chap 9 Priority Queues

Operations supported by priority queue
The functions that have been supported: SP1: Return an element with minmum priority SP2: Insert an element with an arbitrary priority SP2: Delete an element with minimum priority Extended functions: Meld two priority queues delete an arbitrary element decrease the key/priority

Double-ended priority queue(DEPQ)
Support the following operations DP1: Return an element with minimum priority DP2: Return an element with maximum priority DP3: Insert an element with an arbitrary DP4: Delete an element with minimum priority DP5: Delete an element with maximum priority

Leftist Trees To combine two priority queues into a single priority queue, the combine operation takes O(n) time if heaps are used. The complexity of combine operation can be reduced to O(log n) if a leftist tree is used. Leftist trees are defined using the concept of an extended binary tree. An extended binary tree is a binary tree in which all empty binary subtrees have been replaced by a square node. The square nodes in an extended binary tree are called external nodes. The original nodes of the binary tree are called internal nodes.

Extended Binary Trees A G B C H I D E F J

Leftist Trees (Cont.) Let x be a node in an extended binary tree. Let LeftChild(x) and RightChild(x), respectively, denote the left and right children of the internal node x. Define shortest(x) to be the length of a shortest path from x to an external node. It is easy to see that shortest(x) satisfies the following recurrence: shortest(x) = 0 if x is an external node 1 + min{shortest(LeftChild(x)), RightChild(x))} otherwise

Shortest(x) of Extended Binary Trees
2 2 A G 1 1 1 2 B C H I 1 1 1 1 D E F J

Leftist Tree Definition
Definition: A leftist tree is a binary tree such that if it is not empty, then shortest(LeftChild(x)) ≥ shortest(RightChild(x)) for every internal node x. Lemma 9.1: Let x be the root of a leftist tree that has n (internal) nodes n ≥ 2shortest(x) – 1 The rightmost root to external node path is the shortest root to external node path. Its length is shortest(x).

Class Definition of A Leftist Tree
template<class KeyType>class MinLeftistTree; // forward declaration template<class KeyType> class LeftistNode { friend class MinLeftistTree<KeyType>; private: Element<KeyType>data; LeftistNode *LeftChild, *RightChild; int shortest; } class MinLeftistTree:public MinPQ<KeyType> { public: // constructor MinLeftistTree(LeftistNode<KeyType> *int = 0) root(int) {}; // the three min-leftist tree operations void Insert(const Element<KeyType>&); Element<KeyType>* DeleteMin(Element<KeyType>&); void MinCombine(MinLeftistTree<KeyType>*); LeftistNode<KeyType>* MinUnion(LeftistNode<KeyType>*, LeftistNode<KeyType>*); LeftistNode<KeyType>* root; };

Definition of A Min (Max) Leftist Tree
Definition: A min leftist tree (max leftist tree) is a leftist tree in which the key value in each node is no larger (smaller) than the key values in its children (if any). In other words, a min (max) leftist tree is a leftist tree that is also a min (max) tree.

Examples of Min Leftist Trees
2 2 2 2 1 1 1 1 7 50 9 8 1 1 2 1 11 80 12 10 1 1 1 1 13 20 18 15

Min (Max) Leftist Tree (Cont.)
Like any other tree structures, the popular operations on the min (max) leftist trees are insert, delete, and combine. The insert and delete-min operations can both be done by using the combine operation. e.g., to insert an element x into a min leftist tree, we first create a min leftist tree that contains the single element x. Then we combine the two min leftist trees. To delete the min element from a nonempty min leftist tree, we combine the min leftist trees root->LeftChild and root->RightChild and delete the node root.

Combine Leftist Trees To combine two leftist trees:
First, a new binary tree containing all elements in both trees is obtained by following the rightmost paths in one or both trees. Next, the left and right subtrees of nodes are interchanged as necessary to convert this binary tree into a leftist tree. The complexity of combining two leftist trees is O(log n)

Combining The Min Leftist Tree
2 2 2 8 1 1 1 2 10 50 7 5 1 1 2 1 1 2 15 80 11 9 8 2 1 2 1 1 2 2 13 12 10 50 5 7 2 1 1 1 5 20 18 15 80 2 1 1 2 8 1 9 11 9 8 1 1 2 1 2 1 1 10 50 12 13 12 10 50 1 1 1 1 1 1 15 80 20 18 20 18 15 80

Binomial Heaps A binomial heap is a data structure that supports the same functions as those supported by leftist trees. Unlike leftist trees, where an individual operation can be performed in O(log n) time, it is possible that certain individual operations performed on a binomial heap may take O(n) time. By amortizing part of the cost of expensive operations over the inexpensive ones, then the amortized complexity of an individual operation is either O(1) or O(log n) depending on the type of operations.

Cost Amortization Given a sequence of operations I1, I2, D1, I3, I4, I5, I6, D2, I7. Assume each insert operation costs one time unit and D1 and D2 operations take 8 and 10 time units, respectively. The total cost to perform the sequence of operations is 25. If we charge some actual cost of an operation to other operations, this is called cost amortization. In this example, the amortized cost of I1 – I6 each has 2 time units, I7 has one, and D1 and D2 each has 6. Now suppose we can prove that no matter what sequence of insert and delete-min operations is performed, we can charge costs in such a way that the amortized cost of each insertion is no more than 2 and that of each deletion is no more than 6. We can claim that the sequence of insert/delete-min operations has cost no more than 2*i + 6*d. With the actual cost, we conclude that the sequence cost is no more than i+10*d. Combining the above two bounds, we obtain min{2*i+6*d, i+10*d}.

Binomial Heaps Binomial heaps have min binomial heap and max binomial heap. We refer to the min binomial heap as B-heap. B-heap can perform an insert and a combine operation in O(1) actual and amortized time and a delete-min operation with O(log n) amortized time. A node in a B-heap has the following data members: degree: is the number of children it has child: is a pointer points to any one of its children. All children forms a circular list. link: is a singly link used to maintain a circular list with its siblings. data The roots of the min trees that comprise a B-heap are linked to form a singly linked circular list. The B-heap is then pointed at by a single pointer min to the min tree root with smallest key.

B-Heap Example 1 3 8 12 7 16 10 5 4 15 30 9 6 min 20 8 3 1 10 12 7 16 5 4 6 15 30 9 20

Insertion Into A B-Heap
An element x can be inserted into a B-heap by first putting x into a new node and then inserting this node into the circular list pointed at by min. The operation is done in O(1) Time. To combine two nonempty B-heaps, combine the top circular lists of each into a single circular list. The new combined B-heap pointer is the min pointer of one of the two trees, depending on which has the smaller key. Again the combine operation can be done in O(1) time.

The Tree Structure After Deletion of Min From B-Heap
8 16 3 12 7 10 9 5 4 15 30 6 20

Deletion of Min Element
If min is 0, then the B-heap is empty. No delete operations can be performed. If min is not 0, the node is pointed by min. Delete-min operation deletes this node from the circular list. The new B-heap consists of the remaining min trees and the submin trees of the delete root. To form the new B-heap, min trees with the same degrees are joined in pairs. The min tree whose root has the larger key becomes the subtree of the other min tree.

Joining Two Degree-One Min Trees
16 7 3 12 8 9 30 5 4 15 10 6 20

Joining Two Degree-Two Min Trees
3 16 12 7 5 4 15 30 8 9 6 20 10 Since no two min trees have the same degree, the min join process stops.

The New B-Heap min 12 3 16 30 7 15 5 4 20 8 9 6 10

Program 9.12 Steps In A Delete-Min Operation
template<class KeyType> Element<KeyType>*Binomial<KeyType>::DeleteMin(Element<KeyType>&x) Step 1: [Handle empty B-heap] if (!min){ DeletionError(); return();} Step 2: [Deletion from nonempty B-heap] x=min->data; y=min->child; delete min from its circular list; following this deletion, min points to any remaining node in the resulting list; if there is no such node, then min = 0; Step 3: [Min-tree joining] Consider the min trees in the lists min and y; join together pairs of min trees of the same degree until all remaining min trees have different degrees; Step 4: [Form min tree root list] Link the roots of the remaining min trees (if any) together to form a circular list; set min to point to the root (if any) with minimum key; return &x; Step 1 takes O(1) time. Step 2 takes O(1) time by copying over the data from the next node in the circular list and physically deleting that node instead. Step 3 may be implemented by using an array tree[] that is indexed from 0 to the maximum possible degree, MaxDegree, of a min tree.

Binomial Tree Definition
Definition: The binomial tree Bk, of degree k is a tree such that if k = 0, then the tree has exactly one node, and if k > 0, then the tree consists of a root whose degree is k and whose subtrees are B0, B1, …, Bk-1. Lemma 9.2: Let a be a B-heap with n elements that results from a sequence of insert, combine, and delete-min operations performed on a collection of initially empty B-heaps. Each min tree in a has degree ≤ log2n. Consequently, MaxDegree ≤ , and the actual cost of a delete-min operation is O(log n + s).

B-Heap Costs Theorem 9.1: If a sequence of n insert, combine, and delete-min operations is performed on initially empty B-heaps, then we can amortize costs such that the amortized time complexity of each insert and combine is O(1), and that of each delete-min operation is O(log n).

Fibonacci Heaps A Fibonacci heap is a data structure that supports the three binomial heap operations: insert, delete-min (or delete-max), and combine, as well as the following additional operations: decrease key: Decrease the key of a specified node by a given positive amount delete: Delete the element in a specified node. The first of these can be done in O(1) amortized time and the second in O(log n) amortized time. The binomial heap operations can be performed in the same asymptotic times using a Fibonacci heap as they can be using a binomial heap.

Fibonacci Heaps (Cont.)
Two varieties of Fibonacci heaps: Min Fibonacci heap: is a collection of min trees Max Fibonacci heap: is a collection of max trees. Refers to min Fibonacci heap as F-heaps. B-heaps are a special case of F-heaps. A node in F-heap data structure contains additional data members other than those in B-heaps: parent: is used to point to the node’s parent (if any). ChildCut: to support cascading cut described later. LeftLink and RightLink: replace the link data member in B-heap node. These links form a doubly linked circular list. In F-heaps, singly linked circular list is replaced by doubly linked circular list.

Deletion From An F-Heap
The basic operations insert, delete-min, and combine are performed exactly as for the case of B-heaps. Follow the below steps to delete a node b from an F-heap: If min = b, then do a delete-min; otherwise do Steps 2, 3, and 4 below Delete b from its doubly linked list Combine the doubly linked list of b’s children with the doubly linked list pointed at by min into a single doubly linked list. Trees of equal degree are not joined as in a delete-min operation. Dispose of node b.

F-Heap After Deletion of 12
8 3 1 11 30 10 4 20 5 7 16 6 9 min 8 3 1 11 30 10 20 5 4 7 16 6 9

Decrease Key To decrease the key in node b, do the following:
Reduce the key in b If b is not a min tree root and its key is smaller than that in its parent, then delete b from its doubly linked list and insert it into the doubly linked list of min tree roots. Change min to point to b if the key in b is smaller than that in min.

F-Heap After The Reduction of 15 by 4
min 8 10 3 5 6 4 12 15 20 30 1 7 9 16 min 8 3 1 11 10 12 7 16 20 5 4 6 30 9

Cascading Cut Because the new delete and decrease-key operations, the F-heap is not necessary a Binomial tree. Therefore, the analysis of theorem 9.1 is no longer true for F-heaps if no restructuring is done. To ensure that each min tree of degree k has at least ck nodes, for some c, c> 1, each delete and decrease-key operations must be followed by a particular step called cascading cut. The data member ChildCut is used to assist the cascading cut step. ChildCut data member is only used for non-root node.

Cascading Cut (Cont.) ChildCut of node x is TRUE iff one of the children of node x was cut off after the most recent time x was made the child of its current parent. Whenever a delete or decrease-key operation deletes a node q that is not a min tree root from its doubly linked list, then the cascading cut step is invoked. During the steps, we examine the nodes on the path from the parent p of the deleted node q up the nearest ancestor of the deleted node with ChildCut = FALSE. If there is no such ancestor, then the path goes from p to the root of the min tree containing p. All nonroot nodes on this path with ChildCut data member TRUE are deleted from their respective doubly linked list and added to the doubly linked list of min tree root nodes of the F-heap. If the path has a node with ChildCut set to FALSE, then it is changed to TRUE.

A Cascading Cut Example
10 12 10 2 4 5 16 15 18 30 11 6 60 8 6 20 8 7 20 7 10 2 30 12 11 4 5 ChildCut=TRUE * 14 18 60 16 15

F-Heap Analysis Lemma 9.3: Let a be an F-heap with n elements that results from a sequence of insert, combine, delete-min, delete, and decrease-key operations performed on initially empty F-heaps. Let b be any node in any of the min trees of a. The degree of b is at most logΦ m, where and, m is the number elements in the subtree with root b. MaxDegree ≤ , and the actual cost of a delete-min operation is O(log n + s).

Theorem 9.2 Theorem 9.2: If a sequence of n insert, combine, delete, delete-min, and decrease-key operations is performed on an initially empty F-heap, then we can amortize costs such that the amortized time complexity of each insert, combine, and decrease-key operation is O(1) and that of each delete-min and delete operation is O(log n). The total time complexity of the entire sequence is the sum of the amortized complexities of the individual operations in the sequence.

Min-Max Heaps A double-ended priority queue is a data structure that supports the following operations: inserting an element with an arbitrary key deleting an element with the largest key deleting an element with the smallest key A Min-Max Heap supports all of the above operations.

Min-Max Heap (Cont.) Definition: A min-max heap is a complete binary tree such that if it is not empty, each element has a data member called key. Alternating levels of this tree are min levels and max levels, respectively. The root is on a min level. Let x be any node in a min-max heap. If x is on a min (max) level then the element in x has the minimum (maximum) key from among all elements in the subtree with root x. A node on a min (max) level is called a min (max) node.

Figure 9.1: A 12-element Min-Max Heap
7 min 70 40 max 30 9 10 15 min 45 50 30 20 12 max

Min-Max Heap (Cont.) The min-max heap stores in a one-dimension array h. Insert a key 5 into this min-max heap. Initially key 5 is inserted at j. Now since 5 < 10 (which is j’s parent), 5 is guaranteed to be smaller than all keys in nodes that are both on max levels and on the path from j to root. Only need to check nodes on min levels. When inserting 80 into this min-max heap, since 80 > 10, and 10 is on the min level, we are assured that 80 is larger than all keys in the nodes that are both on min levels and on the path from j to the root. Only need to check nodes on max levels.

Insert to Min-Max Heap 7 min 70 40 max 30 9 10 15 min 45 50 30 20 12 j

Min-Max Heap After Inserting Key 5
70 40 max 30 9 7 15 min 45 50 30 20 12 10 max

Min-Max Heap After Inserting Key 80
7 min 70 80 max 30 9 10 15 min 45 50 30 20 12 40 max

Program 9.3 Insertion Into A Min-Max Heap
template <class KeyType> void MinMaxHeap<KeyType>::Insert(const Element<KeyType>& x) // inset x into the min-max heap { if (n==MaxSize) {MinMaxFull(); return;} n++; int p =n/2; // p is the parent of the new node if(!p) {h[1] = x; return;} // insert into an empty heap switch(level(p)) { case MIN: if (x.key < h[p].key) { // follow min levels h[n] = h[p]; VerifyMin(p, x); } else { VerifyMax(n, x); } // follow max levels break; case MAX: if (x.key > h[p].key) { // follow max levels VerifyMax(p, x); else { VerifyMin(n, x); } // follow min levels

Program 9.4 Searching For The Correct Max Node For Insertion
template <class KeyType> void MinMaxHeap<KeyType>::VerifyMax(int i, const Elemetn<KeyType>& x) // Follow max nodes from the max node I to the root and insert x at proper place { for (int gp = i/4; // grandparent of i gp && (x.key > h[gp].key); gp /= 4) { // move h[gp] to h[i] h[i] = h[gp]; i = gp; } h[i] = x; // x is to be inserted into node i O(log n)

Deletion of the Min Element
12 min 70 40 max 30 9 10 15 min 45 50 30 20 max

Deletion of the Min Element (Cont.)
When delete the smallest key from the min-max heap, the root has the smallest key (key 7). So the root is deleted. The last element with key 12 is also deleted from the min-max heap and then reinsert into the min-max heap. Two steps to follow: The root has no children. In this case x is to be inserted into the root. The root has at least one child. Now the smallest key in the min-max heap is in one of the children or grandchildren of the root. Assume node k has the smallest key, then following conditions must be considered: x.key ≤ h[k].key. x may be inserted into the root. x.key >h[k].key and k is a child of the root. Since k is a max node, it has not descendents with key larger than h[k].key. So, node k has no descendents with key larger than x.key. So the element h[k] may be moved to the root, and x can be inserted into node k. x.key> h[k] and k is a grandchild of the root. h[k] is moved to the root. Let p the parent of k. If x.key > h[p].key, then h[p] and x are to be interchanged.

Min-Max Heap After Deleting Min Element
9 min 70 40 max 30 12 10 15 min 45 50 30 20 max

Deaps A deap is a double-ended heap that supports the double-ended priority operations of insert, delet-min, and delete-max. Similar to min-max heap but deap is faster on these operations by a constant factor, and the algorithms are simpler.

Deaps (Cont.) Definition: A deap is a complete binary tree that is either empty or satisfies the following properties: (1) The root contains no element (2) The left subtree is a min heap. (3) The right subtree is a max heap. (4) If the right subtree is not empty, then let i be any node in the left subtree. Let j be the corresponding node in the right subtree. If such a j does not exist, then let j be the node in the right subtree that corresponds to the parent of i. The key in node i is less than or equal to that of j.

A Deap Example 5 45 10 8 25 40 15 19 9 30 20

Deap (Cont.) From Deap’s definition, it’s obvious that for an n-element deap, the min element is the root of min heap and the max element is the root of the max heap. If n = 1, then the min and max elements are the same and are in the root of the min heap. Since deap is a complete binary tree, it may be stored as an implicit data structure in a one-dimension array similar to min, max, min-max heaps. In the case of deap, the position 1 of the array is not used. For an n-element deap, it occupied n+1 element of an array. If i is a node in the min heap of the deap, its corresponding node in the max heap is Then j defined in property (4) of definition is given by if (j > n+1) j /= 2;

Figure 9.7 Insertion Into A Deap
5 45 10 8 25 40 15 19 9 30 20 j i

Figure 9.8: Deap Structure After Insertion of 4
45 5 8 25 40 15 10 9 30 20 19

Figure 9.8: Deap Structure After Insertion of 30
5 45 10 8 30 40 15 19 9 30 20 25

Program 9.7: Inserting Into A Deap
template <class KeyType> void Deap<KeyType>::Insert(const Element<KeyType>& x) { //Insert x into the deap int I; if (n==MaxSize) {DeapFull(); return;} n++; if (n==1) {d[2]=x; return;} // insert into an empty deap int p = n+1; // p is the new last position of the deap switch(MaxHead(p)) { case TRUE: // p is a position in the max heap i = MinPartner(p); if (x.key < d[i].key) { d[p] = d[i]; MinInsert(i, x); } else MaxInsert(p, x); break; case FALSE: // p is a position in the min heap i = MaxPartner(p); if (x.key > d[i].key) { MaxInsert(i, x); else MinInsert(p, x); O(log n)

Deletion of Min Element
Suppose we want to remove the minimum element from the deap. We first place the last element into a temporary element t. Vacancy created by the deletion of the min element is filled by moving from position 2 to a leaf. Each move is preceded by a step that moves the smaller of the elements in the children of the current node up. Then move to the position previously occupied by the moved element. Repeat the process until we move the empty node to a leaf position. Compare the key put in the temporary element with the max partner. If <, no exchange is needed. The temporary element is inserted into the empty leaf node. If >, exchange them.

Deap Structure After Deletion of Min Element
8 45 10 9 25 40 15 19 20 30

Data Structure Assistant Prof. Chih-Chia Yao（ C. –C. Yao.

Similar presentations

Presentation on theme: "Data Structure Assistant Prof. Chih-Chia Yao（ C. –C. Yao."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Data Structure Assistant Prof. Chih-Chia Yao（ C. –C. Yao.

Similar presentations

Presentation on theme: "Data Structure Assistant Prof. Chih-Chia Yao（ C. –C. Yao."— Presentation transcript:

Similar presentations

About project

Feedback