Data Structures and Algorithms Data Structures and algorithms (IS ZC361) An overview of topics for the mid-semester exam S.P.Vimal BITS-Pilani Source :This presentation is composed from the presentation materials provided by the authors (GOODRICH and TAMASSIA) of text book -1 specified in the handout

Data Structures and Algorithms Overview topics 1.Algorithm analysis 2.Linear Data structures 3.Trees 4.Sorting Algorithms

Data Structures and Algorithms Introduction Data Structures: A systematic way of organizing and accessing data. -- No single data structure works well for ALL purposes. An algorithm is a step-by-step procedure for solving a problem in a finite amount of time. Algorithm Input Output

Data Structures and Algorithms Algorithm Descriptions Nature languages: Chinese, English, etc. Pseudo-code: codes very close to computer languages, e.g., C programming language. Programs: C programs, C++ programs, Java programs. Goal: Allow a well-trained programmer to be able to implement. Allow an expert to be able to analyze the running time.

Data Structures and Algorithms Algorithm: An example Algorithm sorting(X, n) Input array X of n integers Output array X sorted in a non-decreasing order for i  0 to n  1 do for j  i+1 to n do if (X[i]>X[j]) then { s=X[i]; X[i]=X[j]; X[j]=s; } return X

Data Structures and Algorithms What do we analyze in an algorithm??? –Estimate the running time –Estimate the memory space required. >> Depends on the input size.

Data Structures and Algorithms Running time of an algorithm Most algorithms transform input objects into output objects. The running time of an algorithm typically grows with the input size.

Data Structures and Algorithms Counting Primitive Operations By inspecting the pseudo code, we can determine the maximum number of primitive operations executed by an algorithm, as a function of the input size Algorithm arrayMax(A, n) # operations currentMax  A[0] 2 for i  1 to n  1 do 2  n if A[i]  currentMax then2(n  1) currentMax  A[i]2(n  1) { increment counter i }2(n  1) return currentMax 1 Total 7n  1

Data Structures and Algorithms Growth Rate of Running Time Changing the hardware/ software environment –Affects T(n) by a constant factor, but –Does not alter the growth rate of T(n) The linear growth rate of the running time T(n) is an intrinsic property of algorithm arrayMax Growth rates of functions: –Linear  n –Quadratic  n 2 –Cubic  n 3 In a log-log chart, the slope of the line corresponds to the growth rate of the function

Data Structures and Algorithms Growth rates

Data Structures and Algorithms Big-Oh notation To simplify the running time estimation, for a function f(n), we ignore the constants and lower order terms. Example: 10n 3 +4n 2 -4n+5 is O(n 3 ) Formally, Given functions f(n) and g(n), we say that f(n) is O(g(n)) if there are positive constants c and n 0 such that f(n)  cg(n) for n  n 0

Data Structures and Algorithms Example: 2n + 10 is O(n) 2n + 10  cn (c  2) n  10 n  10/(c  2) Pick c = 3 and n0 = 10

Data Structures and Algorithms Big-Oh notation - examples Example: the function n 2 is not O(n) –n 2  cn –n  c –The above inequality cannot be satisfied since c must be a constant –n 2 is O(n 2 ).

Data Structures and Algorithms Big-Oh notation - examples 7n-2 is O(n) –need c > 0 and n0  1 such that 7n-2  cn for n  n0 –this is true for c = 7 and n0 = 1 3n3 + 20n2 + 5 is O(n3) –need c > 0 and n0  1 such that 3n3 + 20n2 + 5  cn3 for n  n0 –this is true for c = 4 and n0 = 21 3 log n + 5 is O(log n) –need c > 0 and n0  1 such that 3 log n + 5  clog n for n  n0 –this is true for c = 8 and n0 = 2

Data Structures and Algorithms The big-Oh notation gives an upper bound on the growth rate of a function The statement “ f(n) is O(g(n)) ” means that the growth rate of f(n) is no more than the growth rate of g(n) We can use the big-Oh notation to rank functions according to their growth rate

Data Structures and Algorithms 1.Algorithm analysis (  ) 2.Linear Data structures 3.Trees 4.Sorting Algorithms 

Data Structures and Algorithms ADT (Abstract Data Type) An abstract data type (ADT) is an abstraction of a data structure An ADT specifies: –Data stored –Operations on the data –Error conditions associated with operations

Data Structures and Algorithms ADT (Abstract Data Type) Example: ADT modeling a students record –The data stored are Student name, id No., as1, as2,as3, exam –The operations supported are int averageAs(as1,as2,as3) Int finalMark(as1, as2,as3, exam) ) –Error conditions: Calculate the final mark for absent student

Data Structures and Algorithms The Stack ADT The Stack ADT stores arbitrary objects Insertions and deletions follow the last-in first-out scheme Main stack operations: –push (object): inserts an element –object pop(): removes and returns the last inserted element

Data Structures and Algorithms Auxiliary stack operations: –object top(): returns the last inserted element without removing it –integer size(): returns the number of elements stored –boolean isEmpty(): indicates whether no elements are stored

Data Structures and Algorithms Array-based Stack A simple way of implementing the Stack ADT uses an array We add elements from left to right A variable t keeps track of the index of the top element (size is t+1) Algorithm pop(): if isEmpty() then throw EmptyStackException else t  t  1 return S[t + 1] Algorithm push(o) if t = S.length  1 then throw FullStackException else t  t + 1 S[t]  o

Data Structures and Algorithms The Queue ADT The Queue ADT stores arbitrary objects Insertions and deletions follow the first-in first-out scheme Insertions are at the rear of the queue and removals are at the front of the queue Main queue operations: –enqueue(object): inserts an element at the end of the queue –object dequeue(): removes and returns the element at the front of the queue

Data Structures and Algorithms Auxiliary queue operations: –object front(): returns the element at the front without removing it –integer size(): returns the number of elements stored –boolean isEmpty(): indicates whether no elements are stored Exceptions –Attempting the execution of dequeue or front on an empty queue throws an EmptyQueueException

Data Structures and Algorithms Singly Linked List A singly linked list is a concrete data structure consisting of a sequence of nodes Each node stores –element –link to the next node next elem node ABCD 

Data Structures and Algorithms Queue with a Singly Linked List We can implement a queue with a singly linked list –The front element is stored at the first node –The rear element is stored at the last node The space used is O(n) and each operation of the Queue ADT takes O(1) time f nodes elements r 

Data Structures and Algorithms List ADT The List ADT models a sequence of positions storing arbitrary objects It allows for insertion and removal in the “middle” Query methods: –isFirst(p), isLast(p) Accessor methods: –first(), last() –before(p), after(p) Update methods: –replaceElement(p, o), swapElements(p, q) –insertBefore(p, o), insertAfter(p, o), –insertFirst(o), insertLast(o) –remove(p)

Data Structures and Algorithms Doubly Linked List A doubly linked list provides a natural implementation of the List ADT Nodes implement Position and store: –element –link to the previous node –link to the next node Special trailer and header nodes prevnext elem trailer header nodes/positions elements node

Data Structures and Algorithms 1.Algorithm analysis (  ) 2.Linear Data structures (  ) 3.Trees 4.Sorting Algorithms 

Data Structures and Algorithms Trees (§2.3) In computer science, a tree is an abstract model of a hierarchical structure A tree consists of nodes with a parent-child relation Applications: –Organization charts –File systems –Programming environments Computers”R”Us SalesR&DManufacturing LaptopsDesktops US International EuropeAsiaCanada

Data Structures and Algorithms Tree ADT (§2.3.1) We use positions to abstract nodes Generic methods: –integer size() –boolean isEmpty() –objectIterator elements() –positionIterator positions() Accessor methods: –position root() –position parent(p) –positionIterator children(p) Query methods: –boolean isInternal(p) –boolean isExternal(p) –boolean isRoot(p) Update methods: –swapElements(p, q) –object replaceElement(p, o) Additional update methods may be defined by data structures implementing the Tree ADT

Data Structures and Algorithms Preorder Traversal (§2.3.2) A traversal visits the nodes of a tree in a systematic manner In a preorder traversal, a node is visited before its descendants Application: print a structured document Make Money Fast! 1. MotivationsReferences2. Methods 2.1 Stock Fraud 2.2 Ponzi Scheme 1.1 Greed1.2 Avidity 2.3 Bank Robbery Algorithm preOrder(v) visit(v) for each child w of v preorder (w)

Data Structures and Algorithms Post order Traversal In a postorder traversal, a node is visited after its descendants Application: compute space used by files in a directory and its subdirectories Algorithm postOrder(v) for each child w of v postOrder (w) visit(v) cs16/ homeworks/ todo.txt 1K programs/ DDR.java 10K Stocks.java 25K h1c.doc 3K h1nc.doc 2K Robot.java 20K

Data Structures and Algorithms Binary Trees A binary tree is a tree with the following properties: –Each internal node has two children –The children of a node are an ordered pair We call the children of an internal node left child and right child Alternative recursive definition: a binary tree is either –a tree consisting of a single node, or –a tree whose root has an ordered pair of children, each of which is a binary tree Applications: –arithmetic expressions –decision processes –searching A B C FG D E H I

Data Structures and Algorithms Arithmetic Expression Tree Binary tree associated with an arithmetic expression –internal nodes: operators –external nodes: operands Example: arithmetic expression tree for the expression (2  (a  1)  (3  b))    2 a1 3b

Data Structures and Algorithms Properties of Binary Trees Notation nnumber of nodes enumber of external nodes inumber of internal nodes hheight Properties: –e  i  1 –n  2e  1 –h  i –h  (n  1)  2 –e  2 h –h  log 2 e –h  log 2 (n  1)  1

Data Structures and Algorithms Inorder Traversal In an inorder traversal a node is visited after its left subtree and before its right subtree Application: draw a binary tree –x(v) = inorder rank of v –y(v) = depth of v Algorithm inOrder(v) if isInternal (v) inOrder (leftChild (v)) visit(v) if isInternal (v) inOrder (rightChild (v))

Data Structures and Algorithms Printing Arithmetic Expressions Specialization of an inorder traversal –print operand or operator when visiting node –print “(“ before traversing left subtree –print “)“ after traversing right subtree Algorithm printExpression(v) if isInternal (v) print( “(’’ ) inOrder (leftChild (v)) print(v.element ()) if isInternal (v) inOrder (rightChild (v)) print ( “)’’ )    2 a1 3b ((2  ( a  1))  (3  b))

Data Structures and Algorithms  Linked Data Structure for Representing Trees A node is represented by an object storing –Element –Parent node –Sequence of children nodes Node objects implement the Position ADT B D A CE F B  ADF  C  E

Data Structures and Algorithms Linked Data Structure for Binary Trees A node is represented by an object storing –Element –Parent node –Left child node –Right child node Node objects implement the Position ADT B D A CE   BADCE 

Data Structures and Algorithms Array-Based Representation of Binary Trees nodes are stored in an array … let rank(node) be defined as follows: rank(root) = 1 if node is the left child of parent(node), rank(node) = 2*rank(parent(node)) if node is the right child of parent(node), rank(node) = 2*rank(parent(node)) A HG FE D C B J

Data Structures and Algorithms Binary Search Tree A binary search tree is a binary tree storing keys (or key- element pairs) at its internal nodes and satisfying the following property: –Let u, v, and w be three nodes such that u is in the left subtree of v and w is in the right subtree of v. We have key(u)  key(v)  key(w) External nodes do not store items An inorder traversal of a binary search trees visits the keys in increasing order

Data Structures and Algorithms Search To search for a key k, we trace a downward path starting at the root The next node visited depends on the outcome of the comparison of k with the key of the current node If we reach a leaf, the key is not found and we return NO_SUCH_KEY Example: findElement(4) Algorithm findElement(k, v) if T.isExternal (v) return NO_SUCH_KEY if k  key(v) return findElement(k, T.leftChild(v)) else if k  key(v) return element(v) else { k  key(v) } return findElement(k, T.rightChild(v))   

Data Structures and Algorithms Insertion To perform operation insertItem(k, o), we search for key k Assume k is not already in the tree, and let let w be the leaf reached by the search We insert k at node w and expand w into an internal node Example: insert    w w

Data Structures and Algorithms Deletion To perform operation removeElement(k), we search for key k Assume key k is in the tree, and let let v be the node storing k If node v has a leaf child w, we remove v and w from the tree with operation removeAboveExternal(w) Example: remove v w  

Data Structures and Algorithms Deletion (cont.) We consider the case where the key k to be removed is stored at a node v whose children are both internal –we find the internal node w that follows v in an inorder traversal –we copy key(w) into node v –we remove node w and its left child z (which must be a leaf) by means of operation removeAboveExternal(z) Example: remove v w z v 2

Data Structures and Algorithms Performance Consider a dictionary with n items implemented by means of a binary search tree of height h –the space used is O(n) –methods findElement, insertItem and removeElement take O(h) time The height h is O(n) in the worst case and O(log n) in the best case

Data Structures and Algorithms 1.Algorithm analysis (  ) 2.Linear Data structures (  ) 3.Trees (  ) 4.Sorting Algorithms 

Data Structures and Algorithms Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: –Divide: divide the input data S in two disjoint subsets S 1 and S 2 –Recur: solve the subproblems associated with S 1 and S 2 –Conquer: combine the solutions for S 1 and S 2 into a solution for S The base case for the recursion are subproblems of size 0 or 1 Merge-sort is a sorting algorithm based on the divide-and-conquer paradigm Like heap-sort –It uses a comparator –It has O(n log n) running time Unlike heap-sort –It does not use an auxiliary priority queue –It accesses data in a sequential manner (suitable to sort data on a disk)

Data Structures and Algorithms Merge-Sort Merge-sort on an input sequence S with n elements consists of three steps: –Divide: partition S into two sequences S 1 and S 2 of about n  2 elements each –Recur: recursively sort S 1 and S 2 –Conquer: merge S 1 and S 2 into a unique sorted sequence Algorithm mergeSort(S, C) Input sequence S with n elements, comparator C Output sequence S sorted according to C if S.size() > 1 (S 1, S 2 )  partition(S, n/2) mergeSort(S 1, C) mergeSort(S 2, C) S  merge(S 1, S 2 )

Data Structures and Algorithms Merging Two Sorted Sequences The conquer step of merge-sort consists of merging two sorted sequences A and B into a sorted sequence S containing the union of the elements of A and B Merging two sorted sequences, each with n  2 elements and implemented by means of a doubly linked list, takes O(n) time Algorithm merge(A, B) Input sequences A and B with n  2 elements each Output sorted sequence of A  B S  empty sequence while  A.isEmpty()   B.isEmpty() if A.first().element() < B.first().element() S.insertLast(A.remove(A.first())) else S.insertLast(B.remove(B.first())) while  A.isEmpty() S.insertLast(A.remove(A.first())) while  B.isEmpty() S.insertLast(B.remove(B.first())) return S

Data Structures and Algorithms Merge-Sort Tree An execution of merge-sort is depicted by a binary tree –each node represents a recursive call of merge-sort and stores unsorted sequence before the execution and its partition sorted sequence at the end of the execution –the root is the initial call –the leaves are calls on subsequences of size 0 or  9 4   2  2 79  4   72  29  94  4

Data Structures and Algorithms Execution Example Partition       1 67  72  29  94  43  38  86  61   

Data Structures and Algorithms Execution Example (cont.) Recursive call, partition 7 2  9 4       1 67  72  29  94  43  38  86  61   

Data Structures and Algorithms Execution Example (cont.) Recursive call, partition 7 2  9 4    2      72  29  94  43  38  86  61   

Data Structures and Algorithms Execution Example (cont.) Recursive call, base case 7 2  9 4    2      77  7 2  29  94  43  38  86  61   

Data Structures and Algorithms Execution Example (cont.) Recursive call, base case 7 2  9 4    2      77  72  22  29  94  43  38  86  61   

Data Structures and Algorithms Execution Example (cont.) Merge 7 2  9 4    2      77  72  22  29  94  43  38  86  61   

Data Structures and Algorithms Execution Example (cont.) Recursive call, …, base case, merge 7 2  9 4    2      77  72  22  23  38  86  61     94  4

Data Structures and Algorithms Execution Example (cont.) Merge 7 2  9 4    2      77  72  22  29  94  43  38  86  61   

Data Structures and Algorithms Execution Example (cont.) Recursive call, …, merge, merge 7 2  9 4    2      77  72  22  29  94  43  33  38  88  86  66  61  11   

Data Structures and Algorithms Execution Example (cont.) Merge 7 2  9 4    2      77  72  22  29  94  43  33  38  88  86  66  61  11   

Data Structures and Algorithms Analysis of Merge-Sort The height h of the merge-sort tree is O(log n) –at each recursive call we divide in half the sequence, The overall amount or work done at the nodes of depth i is O(n) –we partition and merge 2 i sequences of size n  2 i –we make 2 i  1 recursive calls Thus, the total running time of merge-sort is O(n log n) depth#seqssize 01n 12 n2n2 i2i2i n2in2i ………

Data Structures and Algorithms Summary of Sorting Algorithms AlgorithmTimeNotes selection-sortO(n2)O(n2) slow in-place for small data sets (< 1K) insertion-sortO(n2)O(n2) slow in-place for small data sets (< 1K) heap-sortO(n log n) fast in-place for large data sets (1K — 1M) merge-sortO(n log n) fast sequential data access for huge data sets (> 1M)

Data Structures and Algorithms 1.Algorithm analysis (  ) 2.Linear Data structures (  ) 3.Trees (  ) 4.Sorting Algorithms  (  ) Questions ?

Data Structures and Algorithms Reference : Chapter 1,2,3 and 4 of Text book-1