CSE 326: Data Structures Lecture #25 Class Wrap-up Steve Wolfman Winter Quarter 2000

Today’s Outline Finish algorithms Wrap-up of class

Memoizing/ Dynamic Programming Define problem in terms of smaller subproblems Solve and record solution for base cases Build solutions for subproblems up from solutions to smaller subproblems Can improve runtime of divide & conquer algorithms that have shared subproblems with optimal substructure. Usually involves a table of subproblem solutions.

Optimal Binary Search Tree Problem Given: –a set of words {w 1, …, w n } –probabilities of each word’s occurrence {p 1, …, p n } Produce a Binary Search Tree which includes all the words and has the lowest expected cost: Expected cost = (where d i is the depth of word i in the tree)

The Optimal BST Optimal Substructure Can an optimal solution possibly have suboptimal subtrees? Shared Subproblems Millenium Falcon And to Ever First to Meet Falcon And to Ever Millenium to Zaphod

Optimal BST Cost Let C Left,Right be the cost of the optimal subtree between w Left and w Right. Then, C Left,Right is: Let’s maintain a 2-D table to store values of C i,j

Optimal BST Algorithm a… am… and… egg… if… the… two… …a …am …and …egg …if …the …two

Backtracking (a.k.a. Systematic Search) Incrementally establish a solution If complete solution is constructed, succeed! If solution fails, roll back and alter recent choices Usually asymptotically no better than brute force. Key to success is pruning the search space. Key to pruning is domain knowledge and learning!

Backtracking in Action Depth First Search DPLL: Satisfiability Solving  -  Search (Game Search)

Game Search Search space is composed of board positions Transitions are moves Levels alternate between us and them We can evaluate any given board position according to a scoring heuristic How should we decide the next move?

Backtracking Game Search (MiniMax) us them us

 -  Pruned Game Search us them us

Randomized Algorithms Define a property (or subroutine) in an algorithm Sample or randomly modify the property Use altered property as if it were the true property Can transform average case runtimes into expected runtimes (remove input dependency). Sometimes allows substantial speedup in exchange for probabilistic unsoundness.

Randomization in Action Treaps Quicksort Randomized back-off Primality testing

Properties of Primes P is a prime 0 < A < P and 0 < X < P Then: A P-1 = 1 (mod P) And, the only solutions to X 2 = 1 (mod P) are: X = 1 and X = P - 1

Calculating Powers HugeInt pow(HugeInt x, HugeInt n, HugeInt modulo) { if (n == 0) return 1; if (n == 1) return x; HugeInt squared = x * x % modulo; if (isEven(n)) return pow(squared, n / 2); else return (pow(squared, n/2) * x) % modulo; } // If 1 < x < modulo - 1 // but squared == 1, // then modulo isn’t prime!

Checking Primality Systematic algorithm: –For prime P, for all A such that 0 < A < P –Calculate A P-1 mod P using pow –Check at each step of pow and at end for primality conditions Randomized algorithm: use just one random A If the randomized algorithm reports failure, then P really isn’t prime. If the randomized algorithm reports success, then P might be prime. –P is prime with probability > ¾ –Each new A has independent probability of false positive

Evaluating Randomized Primality Testing Your probability of being struck by lightning this year: % Your probability that a number that tests prime 11 times in a row is actually not prime: % Your probability of winning a lottery of 1 million people five times in a row: 1 in Your probability that a number that tests prime 50 times in a row is actually not prime: 1 in 2 100

Randomized Greedy Algorithms: Simulated Annealing

Randomized Backtracking for Heavy-Tailed Distributions Some backtracking algorithms have a few (fruitless) branches that are very large, both deep and broad. Algorithms which choose randomly at a split point will have a small probability of getting caught in one of these branches. Therefore, some runs finish very quickly, most runs take some time, and a few runs take orders of magnitude more time than the median. Solution: cut off long runs and reseed the randomizer.

Data Structures and ADTs Course Overview Dictionary Priority Queue List, Stack, Queue, Multilist Linked lists Arrays BSTs AVL trees Splay trees B-Trees Hash tables Treaps Skip lists Binary heaps d-Heaps Leftist heaps Skew heaps Binomial queues

Data Structures and ADTs Course Overview Up-tree forest k-D trees Quad trees BSP trees Adjacency list Adjacency matrix Disjoint Set Union/Find ADT Graph Multidimensional Dictionary

Algorithms and Algorithm Analysis Analysis of algorithms –proofs of correctness –asymptotic analysis (relative runtimes in the limit) –strict analysis (actual number of operations) –greedy vs. divide&conquer vs. backtracking etc. Actual algorithms –searching (linear search, binary search, tree search, hashing) –sorting (heapsort, mergesort, quicksort, radixsort) –graph algorithms (Dijkstra’s, Kruskal’s) Course Overview

Evaluating Algorithms Use of space and time: –asymptotic analysis for high-level comparison worst case best case average case expected case –stricter analysis, experiments for practical performance Course Overview How fast is fast enough?

Criteria for Good Running Time Your resources –how much time/memory can you afford? Nature of the problem –some problems are just harder than others (e.g., sorting is harder than finding) Characteristics of your application –what problem sizes/types of inputs will you be running on? –how might they change in the future? Ease of Coding/Maintainability –sometimes coding and maintenance time and costs dominate Course Overview

Games Theoreticians Play Sour Cove Review Prove that an algorithm is  (f(n)) by nature –e.g., sorting using only comparisons cannot be done in less than n log n What’s wrong with this claim? “I wrote a FindMin() operation that runs in in O(log n) time on an unsorted list of integers!”

More Games Theoreticians Play Take an operation and –see how fast you can do it (e.g., decreaseKey in amortized O(1)) –see how simple you can make it (e.g., Treaps) –see how you can put it together with other operations Receives Our Vow

Observation All programs manipulate data –programs process, store, display, gather –data can be information, numbers, images, sound Each program must decide how to store data Choice influences program at every level –execution speed –memory requirements –maintenance (debugging, extending, etc.) Or Cues Overview

Goals of the Course Become familiar with some of the fundamental data structures in computer science Improve ability to solve problems abstractly –data structures are the building blocks Improve ability to analyze your algorithms –prove correctness –gauge (and improve) time complexity Become modestly skilled with the UNIX operating system and X-windows (you’ll need this in upcoming courses) Course Overview

To Do Study for the final

Coming Up Fun stuff on Friday Final Exam (March 13 th )