Data Structures and Algorithms revision

Merge sort

Analysis of Merge-Sort The “height” h of the merge-sort tree is O(log n) This is a consequence of the dividing the problem in 2 at each step, The overall amount of work done at each merge i is O(n) Thus, the total running time of merge-sort is O(n log n) size n … Divide and Conquer Sorting

Worst-case Running Time The worst case for quick-sort occurs when the pivot is the unique minimum or maximum element One of L and G has size n - 1 and the other has size 0 The running time is proportional to the sum n + (n - 1) + … + 2 + 1 Thus, the worst-case running time of quick-sort is O(n2) depth time n 1 n - 1 … … Divide and Conquer Sorting

the fourth iteration of this loop is shown here Insertion Sort GREEN– sorted, RED unsorted. while some elements unsorted: Using linear search, find the location in the sorted portion where the 1st element of the unsorted portion should be inserted Move all the elements after the insertion location up one position to make space for the new element 45 38 60 60 66 66 45 79 47 13 74 36 21 94 22 57 16 29 81 the fourth iteration of this loop is shown here 38 45 60 66 60 45 66 79 47 13 74 36 21 94 22 57 16 29 81

Insert sort in action Start of algorithm. Sorted portion of list will be marked in colour. 15 3 91 68 2 25 31 32 16 4 21 62 1 5 6 7 8 9 10 11 12 Assign i the value 1. Set pivot to be equal to numbers[i] (next…)

Insert sort in action 3 15 91 68 2 25 31 32 16 4 21 62 i pivot 1 3 5 6 7 8 9 10 11 12 i Is the value before the gap greater than pivot? Yes! Move this value into the gap (next…)

Insert sort in action 3 15 91 68 2 25 31 32 16 4 21 62 pivot i 1 3 5 6 7 8 9 10 11 12 i Is the value before the gap greater than pivot? No. Move pivot into the gap. Add 1 to i, then select the ith value as pivot (next…)

Insert sort in action 91 3 15 68 2 25 31 32 16 4 21 62 pivot i 1 5 6 7 8 9 10 11 12 i Is the value before the gap greater than pivot? No. Move pivot into the gap. Add 1 to i, then select the ith value as pivot (next…)

Insert sort in action 68 3 15 91 2 25 31 32 16 4 21 62 pivot i 1 5 6 7 8 9 10 11 12 i Is the value before the gap greater than pivot? Yes! Move this value into the gap (next…)

Insert sort in action 68 3 15 91 2 25 31 32 16 4 21 62 pivot i 1 5 6 7 8 9 10 11 12 i Is the value before the gap greater than pivot? No. Move pivot into the gap. Add 1 to i, then select the ith value as pivot (next…)

Insert sort in action 2 3 15 68 91 25 31 32 16 4 21 62 pivot i 1 2 5 6 7 8 9 10 11 12 i Is the value before the gap greater than pivot? Yes! Move this value into the gap (next…)

Insert sort in action 2 3 15 68 91 25 31 32 16 4 21 62 pivot i 1 2 5 6 7 8 9 10 11 12 i Is the value before the gap greater than pivot? Yes! Move this value into the gap (next…)

Insert sort in action 2 3 15 68 91 25 31 32 16 4 21 62 pivot i 1 2 5 6 7 8 9 10 11 12 i Is the value before the gap greater than pivot? Yes! Move this value into the gap (next…)

Insert sort in action 2 3 15 68 91 25 31 32 16 4 21 62 pivot i 1 2 5 6 7 8 9 10 11 12 i Is the value before the gap greater than pivot? Yes! Move this value into the gap (next…)

Insert sort in action 2 3 15 68 91 25 31 32 16 4 21 62 pivot i 1 2 5 6 7 8 9 10 11 12 i Is the value before the gap greater than pivot? No. Move pivot into the gap. Add 1 to i, then select the ith value as pivot (next…)

Insert sort in action 25 2 3 15 68 91 31 32 16 4 21 62 pivot i 1 5 6 7 1 5 6 7 8 9 10 11 12 i

And so on…until the list is finally in order: pivot 2 3 4 15 16 21 25 31 32 62 68 91 1 5 6 7 8 9 10 11 12 i

An insertion sort partitions the array into two regions; variable i referred to as pivot.

Selection Sort 5 1 3 4 6 2 Comparison Data Movement Sorted

Selection Sort 5 1 3 4 6 2 Comparison Data Movement Sorted

Selection Sort 5 1 3 4 6 2 Comparison Data Movement Sorted

Selection Sort 5 1 3 4 6 2 Comparison Data Movement Sorted

Selection Sort 5 1 3 4 6 2 Comparison Data Movement Sorted

Selection Sort 5 1 3 4 6 2 Comparison Data Movement Sorted

Selection Sort 5 1 3 4 6 2 Comparison Data Movement Sorted

Selection Sort 5 1 3 4 6 2  Smallest Comparison Data Movement Sorted

Selection Sort 1 5 3 4 6 2 Comparison Data Movement Sorted

Selection Sort 1 5 3 4 6 2 Comparison Data Movement Sorted

Selection Sort 1 5 3 4 6 2 Comparison Data Movement Sorted

Selection Sort 1 5 3 4 6 2 Comparison Data Movement Sorted

Selection Sort 1 5 3 4 6 2 Comparison Data Movement Sorted

Selection Sort 1 5 3 4 6 2  Smallest Comparison Data Movement Sorted

Selection Sort 1 5 3 4 6 2  Smallest Comparison Data Movement Sorted

Selection Sort 1 2 3 4 6 5 Comparison Data Movement Sorted

Selection Sort 1 2 3 4 6 5 Comparison Data Movement Sorted

Selection Sort Algorithm Input: An array ‘A’ of n comparable items Output: The array ‘A’ with elements in non-decreasing order for i0 to n-2 do // note n-2 not n-1 //Insert smallest item in 0th slot, 2nd smallest in 1st, etc. min  i for j  i+1 to j <= n-1 do if (A[j] < A[min]) min  j swap A[i] and A[min]. // What is really happening here? What is the complexity of Selection Sort, and does it vary with input?

Complexity Time complexity is the amount of time (or computational steps) to achieve a task. Space complexity is the amount of memory needed to complete that task. Time complexity>Space complexity. WHY??? Time complexity is considered more important. We do not use wall clock time – as this depends on the hardware. We use an abstract measure O(n), O(n log n)

How long does this take to open 1) know 2) don’t know. What is the smallest operation we can count Analysis of Algorithms

Analysis of Algorithms Loop runs n times 1 loop comparison 1 loop increment 1 operation 3n operations Counting Operations n = number of elements public static double avg(int[] a, int n){ double sum=0; for(int j=0; j<n; j++) sum+=a[j]; return (sum/n); } There is one operation in the for-loop Total Operations: 3n + 3 Analysis of Algorithms

Analysis of Algorithms O(n2) Method 1 Loop assignments. 3 operations for each of the n-1 iterations. Total = n(n-1)/2 iterations public static void bubble(int[] a, int n){ for (i=0; i<n-1; i++) {//how many are sorted for (j=0; j<n-1-i; j++)//unsorted part if (a[j+1] < a[j]) //compare neighbors swap(a, j, j+1); } } Inner loop has n-1 iterations n-2 iterations n-3 iterations ….. 1 iteration 2 loop ops 1 comparison + 3 ops for swap 6 operations On Each iteration 6[n(n-1)/2] =3n(n-1) =3n2-3n Operations from inner loop Operation Count = 1 Operation Count = 3n2 – 2 Operation Count = 3n – 2 + 3n2 – 3n Operation Count = 1 + 3(n-1) Operation Count = 3n – 2 Analysis of Algorithms

Analysis of Algorithms Functions If the following are runtimes expressed as functions of the number of inputs (x), we would label them as follows: f(x) = 7 f(x) = log(x + 3) f(x) = 3x + 5 f(x) = 4x2 + 15x + 90 f(x) = 10x3 – 30 f(x) = 2(3x + 3) O(1) O(log n) O(n) O(n2) O(n3) O(2n) Analysis of Algorithms

Singly Linked List A singly linked list is a concrete data structure consisting of a sequence of nodes Each node stores: an element a link to the next node next node elem  A B C D Linked Lists

The Node Class for List Nodes public class StringNode { // Attributes private String element; // The data to be stored in this node private StringNode next; // A link to the next node in the chain /** Constructor Creates a node with the given element and next node. */ public StringNode(String e, StringNode n) element = e; next = n; } Creates a node with null references to its element and next node. */ public StringNode() this(null, null); Linked Lists

The Node Class for List Nodes // Accessor methods public String getElement() { return element; } public StringNode getNext() return next; // Modifier methods: public void setElement(String newElem) element = newElem; public void setNext(StringNode newNext) next = newNext; Linked Lists

For any data structure/storage: Operations Location of Operation How to add How to retreive How to remove/delete. Beginning End Middle Linked Lists

Inserting at the Head of a List The current list... Create a new node with content X Insert the node Have new node point to old head Update head to point to new node head A B C  node X  node X head A B C  node head X A B C  Linked Lists

Linked List - addFirst public class StringLinkedList { StringNode head = null; // The start of the list StringNode tail = null; // The last element in the list public void addFirst(StringNode n) { // Check we have a node to add... if (n == null) return; // Set our new node to point to the head // of the current list. n.setNext(head); // Our new node 'n' will now become the head of the list head = n; // If the list was empty, make the tail point to // the new node as well if (tail == null) tail = n; } Linked Lists

Removing from the Head Update head to point to next node in the list Allow garbage collector to reclaim the former first node head A B C D  head A B C D  head B C D  A Linked Lists

Linked List - removeFirst public void removeFirst() { // If list is empty, we can't remove anything so leave if (head == null) return; // Move head to the next item in the list head = head.getNext(); // If the list is empty, set the tail reference to null if (head == null) tail = null; // The original item that was at the head of the list // no longer has anything referencing it and will be // garbage collected in Java. In other programming languages // e.g. C++, you would have to delete it other wise you // would get a memory leak. } Linked Lists

Inserting at the Tail The current list... Create a new node pointing to null Insert new element Have old last node point to new node Update tail to point to new node head A B C  node X  node X  tail head A B C node tail head A B X  C Linked Lists

Linked List - addLast public void addLast(StringNode node) { // If we were not given a node, leave if (node == null) return; // If list is empty, our new node will // be the head and tail of list if (head == null) head = node; tail = node; return; } // Make the current last node point to our new node tail.setNext(node); // Now update the tail to be our new node Linked Lists

Removing at the Tail Removing at the tail of a singly linked list is not efficient! There is no constant-time way to update the tail to point to the previous node tail head A B C D  Linked Lists

Linked List - removeLast public void removeLast() { if (head == null) return; // If list is empty, leave // If head is also the tail, the list // will be empty if (head == tail) { head = null; tail = null; return; } // Start at the head of the list StringNode n = head; // Now look for the last item while (n.getNext() != tail) n = n.getNext(); // n should now be pointing to the last but one // node in the list. This will be the new tail // We are going to drop the last element in the list // so make the current node's next pointer null n.setNext(null); // The old tail node is now replaced with 'n'. The // old tail node has no reference and will be garbage tail = n; Linked Lists

HASH TABLES. HASH TABLES. Try to take advantage of the fast speed we can access arrays (a build in data structure.) We could use an id (e.g. passport number or student number, mobile number), but the array would be huge. We “squash” or “compress” the possible indexes into a smaller range with hashing function. However collisions can occur between entries.

Array as table 0012345 andy 81.5 0033333 betty 90 0056789 david 56.8 studid name score 0012345 andy 81.5 0033333 betty 90 0056789 david 56.8 ... 9801010 peter 20 9802020 mary 100 ... 9903030 tom 73 9908080 bill 49 Consider this problem. We want to store 1,000 student records and search them by student id.

Array as table : : : 12345 andy 81.5 : : : 33333 betty 90 : : : 56789 name score : : : One way is to store the records in a huge array (index 0..9999999). The index is used as the student id, i.e. the record of the student with studid 0012345 is stored at A[12345] -- Is this a good idea? 12345 andy 81.5 : : : 33333 betty 90 : : : 56789 david 56.8 : : : : : : 9908080 bill 49 : : : 9999999 Most of the table would be empty

Array as table It is also called Direct-address Hash Table. • Each slot, or position, corresponds to a key in U. • If there’s an element x with key k, then T [k] contains a pointer to x. • Otherwise, T [k] is empty, represented by NIL.

Array as table Store the records in a huge array where the index corresponds to the key add - very fast O(1) delete - very fast O(1) search - very fast O(1) But it wastes a lot of memory! Not feasible.

function Hash(key: KeyType): integer; Hash function function Hash(key: KeyType): integer; Imagine that we have such a magic function Hash. It maps the key (studid) of the 1000 records into the integers 0..999, one to one. No two different keys maps to the same number. H(‘0012345’) = 134 H(‘0033333’) = 67 H(‘0056789’) = 764 … H(‘9908080’) = 3

Hash Table : : : 9908080 bill 49 : : : 0033333 betty 90 : : : 0012345 name score To store a record, we compute Hash(stud_id) for the record and store it at the location Hash(stud_id) of the array. To search for a student, we only need to peek at the location Hash(target stud_id). : : : 3 9908080 bill 49 : : : 67 0033333 betty 90 : : : 134 0012345 andy 81.5 : : : 764 0056789 david 56.8 : : : 999 : : :

Chained Hash Table nil nil nil : nil One way to handle collision is to store the collided records in a linked list. The array now stores pointers to such lists. If no key maps to a certain hash value, that array entry points to nil. 1 nil 2 nil 3 4 nil 5 : Key: 9903030 name: tom score: 73 HASHMAX nil

Collections Framework Diagram Each collection class implements an interface from a hierarchy Each class is designed for a specific type of storage Copyright © 2013 by John Wiley & Sons. All rights reserved.

Depth first search Breath first search Greedy MST PRIM kRUKLALK DISTTAR.

Depth First Search (DFS) Depth First Search (DFS) algorithm traverses a graph in a depthward motion and uses a stack to remember to get the next vertex to start a search, when a dead end occurs in any iteration.

Depth First Search (DFS)

Depth First Search (DFS) Rule 1 − Visit the adjacent unvisited vertex. Mark it as visited. Display it. Push it in a stack. Rule 2 − If no adjacent vertex is found, pop up a vertex from the stack. (It will pop up all the vertices from the stack, which do not have adjacent vertices.) Rule 3 − Repeat Rule 1 and Rule 2 until the stack is empty.

dfs1 Initialize the stack.

dfs2 Mark S as visited and put it onto the stack. Explore any unvisited adjacent node from S. We have three nodes and we can pick any of them. For this example, we shall take the node in an alphabetical order.

dfs3 Mark A as visited and put it onto the stack. Explore any unvisited adjacent node from A. Both Sand D are adjacent to A but we are concerned for unvisited nodes only.

dfs4 Visit D and mark it as visited and put onto the stack. Here, we have B and C nodes, which are adjacent to D and both are unvisited. However, we shall again choose in an alphabetical order.

dfs5 We choose B, mark it as visited and put onto the stack. Here Bdoes not have any unvisited adjacent node. So, we pop Bfrom the stack.

dfs6 We choose B, mark it as visited and put onto the stack. Here Bdoes not have any unvisited adjacent node. So, we pop Bfrom the stack.

dfs7 Only unvisited adjacent node is from D is C now. So we visit C, mark it as visited and put it onto the stack.

Breadth First Search (BFS) Breadth First Search (BFS) algorithm traverses a graph in a breadthward motion and uses a queue to remember to get the next vertex to start a search, when a dead end occurs in any iteration. Visit all nodes depth 1 first, then depth 2, …

Breadth First Traversal

Rule 1 − Visit the adjacent unvisited vertex. Mark it as visited Rule 1 − Visit the adjacent unvisited vertex. Mark it as visited. Display it. Insert it in a queue. Rule 2 − If no adjacent vertex is found, remove the first vertex from the queue. Rule 3 − Repeat Rule 1 and Rule 2 until the queue is empty.

Bfs 1 Initialize the queue.

Bfs 2 We start from visiting S(starting node), and mark it as visited.

Bfs 3 We then see an unvisited adjacent node from S. In this example, we have three nodes but alphabetically we choose A, mark it as visited and enqueue it.

Bfs 4 Next, the unvisited adjacent node from S is B. We mark it as visited and enqueue it.

Bfs 5 Next, the unvisited adjacent node from S is C. We mark it as visited and enqueue it.

Bfs 6 Now, S is left with no unvisited adjacent nodes. So, we dequeue and find A.

Bfs 7 From A we have D as unvisited adjacent node. We mark it as visited and enqueue it.

binary tree is traversed in-order, We start from A, and following in-order traversal, we move to its left subtree B. B is also traversed in-order. The process goes on until all the nodes are visited.

binary tree is traversed in-order, D → B → E → A → F → C → G

Algorithm inorder Until all nodes are traversed − Step 1 − Recursively traverse left subtree. Step 2 − Visit root node. Step 3 − Recursively traverse right subtree.

Pre order We start from A, and following pre-order traversal, we first visit A itself and then move to its left subtree B. B is also traversed pre-order. The process goes on until all the nodes are visited.

Pre order A → B → D → E → C → F → G

Algorithm preorder Until all nodes are traversed − Step 1 − Visit root node. Step 2 − Recursively traverse left subtree. Step 3 − Recursively traverse right subtree.

traversed post-order We start from A, and following pre-order traversal, we first visit the left subtree B. B is also traversed post-order. The process goes on until all the nodes are visited. The output of post-order traversal of this tree will be −

traversed post-order D → E → B → F → G → C → A

Algorithm post order Until all nodes are traversed − Step 1 − Recursively traverse left subtree. Step 2 − Recursively traverse right subtree. Step 3 − Visit root node.

Binary Search Tree BST is a collection of nodes arranged in a way where they maintain BST properties. Each node has a key and an associated value. While searching, the desired key is compared to the keys in BST and if found, the associated value is retrieved.

Is this a BST

Basic Operations Search − Searches an element in a tree. Insert − Inserts an element in a tree. Pre-order Traversal − Traverses a tree in a pre-order manner. In-order Traversal − Traverses a tree in an in-order manner. Post-order Traversal − Traverses a tree in a post-order manner.