1. Nirmalya Roy School of Electrical Engineering and Computer Science Washington State University Cpt S 223 – Advanced Data Structures Course Review Final Exam

2. Final Exam When: Tuesday (05/1) 1pm – 3pm Where: In Class Closed book, Closed notes Comprehensive (Chapter 1 to Chapter 9) Material for preparation:  Lecture Slides  Homeworks and Programming assignments  Weiss book

3. Course Overview Math and C++ Review (Chapter 1) Algorithm development and analysis (Chapter 2)  Running time Abstract Data Types (Chapter 3)  Linked Lists, Stacks and Queues  Insert, delete, search, sort Trees (Chapter 4)  Tree Traversals  Binary Trees, Binary Search Trees, AVL Trees, Splay Trees  B-Trees

4. Course Overview Hashing (Chapter 5)  Hash Function  Separate Chaining  Linear Probing, Quadratic probing, Double Hashing Priority Queues/Heaps (Chapter 6)  Binary Heap  Insert, deleteMin, Array representation  Binomial Queues (Insert, deleteMin, Merge)

5. Course Overview Sorting (Chapter 7)  Insertion sort, selection sort, merge sort, quick sort Disjoint Set (Chapter 8)  Find and union Graph Algorithms (Chapter 9)  Topological Sort  Shortest-Path Algorithms  Dijkstra’s Algorithm  Network Flow Problems  Minimum Spanning Tree  Prim’s Algorithm  Kruskal’s Algorithm

6. Math Review (Chapter 1) Floors, ceilings, exponents and logarithms  Definitions and manipulations Series: Arithmetic and Geometric series  Definitions and manipulations Proof Techniques: Read the definition, components, and how to use the following  Proof by induction, Proof by counterexample, Proof by contradiction Recursion  Read the definition and rules  Analyze running time of recursive algorithm

7. C++ Review (Chapter 1) Classes and Objects and Methods  Default Parameters, Initializer List, explicit Constructor, Destructor, Constant Member Function  Copy constructor, operator overloading, operator= Call by value, Call by reference Call by constant reference Templates  Function templates and Class templates STL for different data structures

8. Algorithm Analysis (Chapter 2) Asymptotic analysis of an algorithm Best-case, worst-case and average-case analysis Rate of growth: Definitions and Notation (O, Ω, Θ, o)  Proofs for specific examples

9. Asymptotic Notations O() – upper bound  Definition: T(N) = O(g(N)) if there are positive constants c and n 0 such that T(N)  cg(N) when N ≥ n 0  () – lower bound  Definition: T(N) =  (g(N)) if there are positive constants c and n 0 such that T(N) ≥ cg(N) when N ≥ n 0  () – tight bound  Definition: T(N) =  (g(N)) if and only if T(N) = O(g(N)) and T(N) =  (g(N)) o() – strict upper bound  Definition: T(N) = o(g(N)) if for all constants c there exists an n 0 such that T(N) n 0

10. Asymptotic Analysis O-notation gives an upper bound for a function to within a constant factor  - notation gives an lower bound for a function to within a constant factor  -notation bounds a function to within a constant factor  The value of f(n) always lies between c 1 g(n) and c 2 g(n) inclusive In this diagram; T(n) = f(n)

11. Abstract Data Types (Chapter 3) Lists  Operations: Insert, Delete, Search  Implementations: vectors, singly-linked lists, double-linked lists  Analysis of operations for each implementation Stacks (LIFO)  Operations: Push, Pop, Top  Implementations: linked-list, vector (tradeoffs)  Analysis of operations for each implementation Queues (FIFO)  Operations: Enqueue, dequeue  Implementations: linked-list, vector (tradeoffs)  Analysis of operations for each implementation

12. Trees (Chapter 4) Binary Tree Traversals  Pre-order traversal: root, left, right  In-order traversal: left, root, right  Post-order traversal: left, right, root Binary Search Tree (BST)  Definition  Operations: Insert, Delete, Search, FindMin, FindMax, traversals  Know how to perform these on a BST and show resulting BST  Know worst-case and average-case analysis of performance

13. Insert operation on a BST insert(T, x)  Inserts x into the BST T  E.g. insert 5

14. Trees Balanced BST (AVL trees)  Definition: The height of the left and right subtrees at every node in the BST differ by at most 1  Operations: Rotations & Cases, Insert, Lazy Delete, Search, FindMin, FindMax  Know how to perform these on an AVL tree and show resulting AVL tree  Know worst-case performance Splay Tree  Splaying (Zig-zag and Zig-zig), Insert, Remove  Know how to perform these on Splay tree and show resulting Splay tree B-Trees Sets and Maps STL

15. AVL Trees Which of the following is an AVL tree? AVL tree?

16. AVL Insert Insert can violate the AVL balance condition Can be fixed by a rotation Remember 4 cases (Single rotation and double rotation) Inserting 6 violates AVL balance condition Rotating 7-8 restores balance

17. Splay Tree After a node is accessed, push it to the root via a series of AVL rotations  If a node is deep, there are also many other nodes on the path which are relatively deep  Restructuring helps to make future access cheaper  Studies have shown that if a node is accessed, it is likely to be accessed again Guarantees that any M consecutive operations on an empty tree will take at most O(M log 2 N) time Amortized cost per operation is O(log 2 N) Still, some operations may take O(N) time Does not require maintaining height or balance information

18. Splay Tree Insert Insert node 1 in the following Splay tree CASE: Zig-zig

19. Hashing (Chapter 5) Hashing  Hash Function  Collision resolution strategy: Separate Chaining Linear Probing, Quadratic probing, Double Hashing

20. Hash Function Mapping from key to array index is called a hash function Hash function  h(key) = key mod TableSize with integer keys Collision occurs when hash function maps two keys to same array index h(key) ==> hash table index

21. Collision Resolution by Chaining Load factor of a hash table T  N = number of elements in T  M = size of T  = N / M Key k is stored in list at T[h(k)] E.g. TableSize = 10  h(k) = k mod 10  Insert first 10 perfect squares Insertion sequence = 0, 1, 4, 9, 16, 25, 36, 49, 64, 81

22. Collision Resolution by Open Addressing Probe sequence  Sequence of slots in hash table to search  h 0 (x), h 1 (x), h 2 (x), …  Needs to visit each slot exactly once Hash function  h i (x) = (hash(x) + f(i)) mod TableSize  f(0) = 0 ==> first try Three common collision resolution strategies  Linear Probing: h i (x) = (hash(x) + f(i)) mod TableSize, f(i)=i  Quadratic Probing: h i (x) = (hash(x) + f(i)) mod TableSize; f(i) = i 2  Double Hashing: h i (x) = (hash(x) + f(i)) mod TableSize; f(i) = i * hash2(x)

23. Priority Queues/Heaps (Chapter 6) Priority Queues/Heaps  Binary Heap  Insert, deleteMin, array representation  Leftist Heaps  Binomial Queues (Insert, deleteMin, Merge)

24. Binary Heap A binary heap is a binary tree with two properties  Structure property  Heap-order property A binary heap is a complete binary tree  Each level is completely filled  Bottom level may be partially filled from left to right Height of a complete binary tree with N elements is  log 2 N 

25. Binary Heap Example N = 10 Array representation of the binary heap above: Every level (except last) completely filled

26. Heap DeleteMin: Example

27. Binomial Heap A forest of heap-ordered binomial trees  Structure property  Heap-order property A binomial tree of height k is called B k  It has 2 k nodes  The number of nodes at depth d is the binomial co-efficient B3B3 Depth d = 0 d = 1 d = 2 d = 3 # of nodes

28. Binomial Heap with 31 Nodes B0B0 B1B1 B2B2 B3B3 B4B4 B2B2 B2B2 B3B3 B3B3 B k == B k-1 + B k-1

29. Sorting (Chapter 7) Insertion sort; Worst case: O(N 2 ); Best Case O(N) Selection sort ; Worst-case: O(N 2 ) Shell sort; Worst-case: O(N 3/2 ) Heap sort; Worst-case: O(N log 2 N) Merge sort; Worst-case: O(N log 2 N) Quick sort; Worst-case: O(N 2 )

30. Insertion Sort Algorithm:  Start with empty list S and unsorted list A of N items  For each item x in A Insert x into S, in sorted order Example: 7395 AS 7395 AS 3795 AS 3795 AS 3579 SA

31. Selection Sort Algorithm:  Start with empty list S and unsorted list A of N items  for (i = 0; i < N; i++) x  item in A with smallest key Remove x from A Append x to end of S 7395 AS 3795 AS 3597 AS 3579 AS 3579 SA

32. Shell Sort 13 14 94 33 82 25 59 94 65 23 45 27 73 25 39 10 13 14 94 33 82 25 59 94 65 23 45 27 73 25 39 10 10 14 73 25 23 13 27 94 33 39 25 59 94 65 82 45 Sorting each column 10 14 73 25 23 13 27 94 33 39 25 59 94 65 82 45 5-sorting o Works by comparing the elements that are distant o The distance between comparisons decreases as algorithm runs until its last phase

33. Shell Sort Sorting each column 10 14 13 25 23 33 27 25 59 39 65 73 45 94 82 94 3-sorting 10 14 73 25 23 13 27 94 33 39 25 59 94 65 82 45 10 14 13 25 23 33 27 25 59 39 65 73 45 94 82 94 10 14 73 25 23 13 27 94 33 39 25 59 94 65 82 45

34. Shell Sort Insertion Sort:  Start with empty list S and unsorted list A of N items  For each item x in A Insert x into S, in sorted order Unsorted A 1-sorting/ Insertion Sort 10 13 14 23 25 25 27 33 39 45 59 65 73 82 94 94 10 14 13 25 23 33 27 25 59 39 65 73 45 94 82 94 Sorted S

35. Heap Sort 7395 AS 5 93 7 buildHeap() 3 75 9 9573 AS 3975 AS deleteMin() 5 79 3597 AS 7 9 3579 AS 9 3579 S Initially, toss all items in A onto heap (ignoring heap-order) then call buildHeap() Maintain heap backward at the end of the array

36. Merge Sort 73954801 73954801 73954801 73954801 1 +  log 2 N  levels 37594801 35790148 01345789 Sorted S1 Sorted S2 Unsorted A1 Unsorted A2 Sorted S Divide with O(log n) steps Conquer with O(log n) steps Dividing is trivial Merging is non- trivial

37. Quick Sort Algorithm quicksort (array: S) 1.If size of S is 0 or 1, return 2.Pivot = Pick an element v in S 3.Partition S – {v} into two disjoint groups S1 = {x  (S – {v}) | x < v} S2 = {x  (S – {v}) | x > v} 4.Return {quicksort(S1), followed by v, followed by quicksort(S2)}

38. Quick Sort 4715930 For now, let the pivot v be the first item in the list. 1307594 S1S2 v 013 S1 S2 v 579 S1 S2 v 0134 v 579 S1S2 0134579 O(N log 2 N) Dividing (“Partitioning”) is non-trivial Merging is trivial

39. Comparison of Sorting Algorithms

40. The Disjoint Set Class (Chapter 8) Disjoint Sets  Definition, Equivalence class, Equivalence relation find(x) returns the root element of the tree containing x union(x, y) points root node of tree containing y to root node of tree containing x

41. Simple Union Example After union(4, 5): points root node of tree containing 5 to root node of tree containing 4 After union(6, 7) After union(4, 6): Convention is that new root after union (x,y) is x The array representation

42. Union-by-Size Attach the root of the “smaller” tree to the root of the “larger” tree w.r. t size union(3, 7) So, attach root 3 to root 4. size = 1 size = 4

43. Union-by-Size Example Result of union(3, 7) using union-by-size heuristic Result of union(3, 7) using simple union Using simple union could end up unbalanced like this. Size is 5 Size is 1

44. Union-by-Height Example Attach the root of the “shallower” tree to the root of the “deeper” tree union(3, 7)So, attach root 3 to root 4. height = 0 height = 2

45. Path Compression Example After find(14):

46. Graph Algorithms (Chapter 9) Graph Definition  Directed graph, undirected graph, complete graph Graph Algorithms  Topological sort  Shortest paths  Network flow  Minimum spanning tree

47. Topological Sort Order the vertices in a directed acyclic graph (DAG), such that if (u, v)  E, then u appears before v in the ordering Solution #1 is O(|V| 2 ) Solution #2 is O(|V| + |E|) (queue implementation) Possible topological orderings: v 1, v 2, v 5, v 4, v 3, v 7, v 6 and v 1, v 2, v 5, v 4, v 7, v 3, v 6.

48. Shortest Path Problems Input is a weighted graph where each edge (v i, v j ) has cost c i, j to traverse the edge Cost of a path v 1 v 2 …v N is  Weighted path cost Unweighted shortest path (number of edges on path): O(|E| + |V|) Weighted shortest path (weighted path cost)  Dijkstra’s algorithm : O(|E| log |V|)

49. Dijkstra’s Algorithm

50. Dijkstra’s Adjacency List

51. Network Flow Problems Given a directed graph G = (V, E) with edge capacities c v, w Give two vertices s, called the source, and t, called the sink Through any edge, (v, w), at most c v, w units of “flow” may pass At any vertex v, that is not either s or t, the total flow coming in must equal the total flow coming out

52. Maximum Flow Algorithm Network graphFlow graph (f = 5)Residual graph Augmenting path (s, a, c, t) Augmenting path (s, a, d, t) Augmenting path (s, b, d, t)

53. Minimum Spanning Tree Problem Find a minimum-cost set of edges that connect all vertices of a graph at lowest total cost Solution # 1: Prim’s Algorithm:  O(|V| 2 ) without heap  O(|E| log |V|) using binary heaps Solution # 2: Kruskal’s Algorithm: O(|E| log |V|)

54. Prim’s Algorithm Solution #1 (Prim’s Algorithm (1957))  Start with an empty tree T  T = {x}, where x is an arbitrary node in the input graph  While T is not a spanning tree Find the lowest-weight edge that connects a vertex in T to a vertex not in T Add this edge to T  T will be a minimum spanning tree

55. Prim’s Algorithm: Example

56. Kruskal’s algorithm Solution #2 (Kruskal’s algorithm (1956))  Start with T = V (with no edges)  For each edge in increasing order by weight If adding edge to T does not create a cycle Then add edge to T  T will be a minimum spanning tree

57. Kruskal’s Algorithm: Example Input graph: 1 st 2 nd 3 rd 4 th 5 th 6 th Collection of Trees |V| single-node Trees Algorithm Terminates Now only one Tree It is a MST

58. Tentative Final Exam Structure Short Multiple Choice Questions 20*2 = 40 points Medium Multiple Choice Questions 3*5 = 15 points Large Questions 7*5 + 1*10 = 45 points Total = 100 points When: Tuesday (05/1) 1pm – 3pm Where: In Class

59. Good Luck! Please take a few minutes to complete the online course evaluations available at https://skylight.wsu.edu/student/ Thank you for taking Cpt S 223. https://skylight.wsu.edu/student/