Lecture 17 Review

What I’ve taught ≠ What you’ve learnt

Course goals To become proficient in the application of fundamental algorithm design techniques To gain familiarity with the main theoretical tools used in the analysis of algorithms To study and analyze different algorithms for many of “standard” algorithmic problems To introduce students to some of the prominent subfields of algorithmic study in which they may wish to pursue further study

Course contents Algorithm design techniques ‣ Divide-and-conquer, Greedy, Dynamic Programming Analysis of algorithms ‣ O, Ω,Θ, Worst-case and Average-case, Recurrences, Amortized analysis “Standard” algorithmic problems ‣ Sorting, Graph theory, Network flow Prominent subfields ‣ NP theory, Backtracking, Randomization, Approximation, Lower bound, Computational Geometry, Substring search, Data compression

Divide-and-conquer What: Divide-conquer-combine When: Non-overlapping subproblems Examples: Quicksort, mergesort, Select, Closest pair Test: Given a sorted array of distinct integers A[1,..., n], you want to find out whether there is an index i for which A[i] = i.

What: Step by step, always choose local optimal solution When: local optimal -> global optimal Examples: Kruskal, Prim, Huffman code, Fractional knapsack Test: Show how to find the maximum spanning tree of a graph. Greedy

What: 1. write the paradigm: Problems -> subproblems 2. solve it in a bottom-up way When: subproblems overlap Examples: Dijkstra, Knapsack, LCS, Matrix chain Test: Longest increasing subsequence Dynamic programming

Course contents Algorithm design techniques ‣ Divide-and-conquer, Greedy, Dynamic Programming Analysis of algorithms ‣ O, Ω,Θ, Worst-case and Average-case, Recurrences, Amortized analysis “Standard” algorithmic problems ‣ Sorting, Graph theory, Network flow Prominent subfields ‣ NP theory, Backtracking, Randomization, Approximation, Lower bound, Computational Geometry, Substring search, Data compression

Complexity Analysis Tools Notation: O, Ω, Θ Recurrences and Master theorem

Master theorem Simplified Master Theorem Let a ≥ 1, b > 1 and c,d,w ≥ 0 be constants, and let T(n) be defined on the nonnegative integers by the recurrence Then

Complexity Analysis Tools Notation: O, Ω, Θ Recurrences and Master theorem Average-case, worst-case analysis Amortized analysis Test: n! and 2 n, T(n) = 3T(n/2) + n T(n) = T(n-1) + n

Course contents Algorithm design techniques ‣ Divide-and-conquer, Greedy, Dynamic Programming Analysis of algorithms ‣ O, Ω,Θ, Worst-case and Average-case, Recurrences, Amortized analysis “Standard” algorithmic problems ‣ Sorting, Graph theory, Network flow Prominent subfields ‣ NP theory, Backtracking, Randomization, Approximation, Lower bound, Computational Geometry, Substring search, Data compression

Sorting AverageWorst InsertionO(n 2 ) SelectionO(n 2 ) BubbleO(n 2 ) QuickO(nlogn)O(n 2 ) Bottom-up mergeO(nlogn) MergeO(nlogn) HeapO(nlogn) RadixO(kn) BucketO(n)O(n 2 ) Test: Give an optimal algorithm for sorting

Graph Traversal Adj. MatrixAdj. ListApplication BFSO(n 2 )O(n+m)Shortest path DFSO(n 2 )O(n+m) Acyclicity Topological sort SCC Test: Give a linear-time algorithm to find an odd-length cycle in a undirected graph.

Minimum Spanning Tree WithoutWithDS KruskalO(n 2 )O(mlogm)Disjoint set PrimO(n 2 )O(mlogn)Heap Test: Show Prim or Kruskal is valid for graphs with negative weights.

Shortest path Single-sourceGraph typesComplexity DijkstraPositive weightO(mlogn) Bellman-FordAnyO(mn) Topological sortDAGO(m+n) All-pairsFloyd-WarshallAnyO(n 3 ) Test: Give an algorithm to find the shortest cycle containing some edge e.

Maximum flow Ford–FulkersonO(E |f*|) Edmonds–KarpO(V 2 Elogc*), O(VE 2 ) Test: The value of max flow. Applications: Bipartite Matching, Edge/Vertex Disjoint path......

Course contents Algorithm design techniques ‣ Divide-and-conquer, Greedy, Dynamic Programming Analysis of algorithms ‣ O, Ω,Θ, Worst-case and Average-case, Recurrences, Amortized analysis “Standard” algorithmic problems ‣ Sorting, Graph theory, Network flow Prominent subfields ‣ NP theory, Backtracking, Randomization, Approximation, Lower bound, Computational Geometry, Substring search, Data compression

Prominent Subfields NP-theory Randomized algorithm Approximation algorithm Lower bounds Computational geometry Substring search Data compression

NP-theory P: deterministic TM, Polynomial time NP: Nondeterministic TM, Polynomial time A ∝ p B: Use B to solve A, A is less harder than B. NPC: the hardest problems in NP. 3-SAT, 3-coloring, Hamiltonian, Vertex Cover, TSP Test: Show clique is NPC given vertex cover is NPC. NP P NPC

Backtracking What: A kind of intelligent exhaustive search When: partial solution can be quickly checked Examples: 3-coloring, 8-queen......

Lower bounds What: A problem has lower bounds Ω(f(n)) if all algorithms for it are Ω(f(n)). Optimal algorithms In decision tree model, Sorting, Element uniqueness, Closest pair, Convex hull, and Euclidean MST have lower bound Ω(nlogn).

Randomized algorithm What: algorithms that play dice. Types: Monte Carlo and Las Vegas Examples: Randomized Quicksort, String equivalence check, Primality check Test: Tell the difference between Monte Carlo and Las Vegas algorithm.

Approximation algorithm What: a trade-off between precision and time.(polynomial time algorithm) When: optimization problems Approximation ratio, hardness result Examples: Vertex cover, Knapsack, TSP Test: Show there is no constant ratio approximation algorithms for TSP unless P=NP.

Computational geometry What: algorithms which can be stated in terms of geometry Geometric sweeping Examples: Maximal points, Convex hull, Diameter......

Substring search What: Match a pattern in texts. T (pattern): U N D E R S (text): D O U U N D E R S T A N D M E ? Brute-Force, KMP, AC-automation, Rabin-Karp, suffix tree, …

Data Compression Compression reduces the size of a file: To save space when storing it. To save time when transmitting it. Most files have lots of redundancy. Run length, Huffman, LZW, JPEG...

Course contents Algorithm design techniques ‣ Divide-and-conquer, Greedy, Dynamic Programming Analysis of algorithms ‣ O, Ω,Θ, Worst-case and Average-case, Recurrences, Amortized analysis “Standard” algorithmic problems ‣ Sorting, Graph theory, Network flow Prominent subfields ‣ NP theory, Backtracking, Randomization, Approximation, Lower bound, Computational Geometry, Substring search, Data compression

Course Goals To become proficient in the application of fundamental algorithm design techniques To gain familiarity with the main theoretical tools used in the analysis of algorithms To study and analyze different algorithms for many of “standard” algorithmic problems To introduce students to some of the prominent subfields of algorithmic study in which they may wish to pursue further study

The End