Algorithms and Data Review Fall 2010 Karl Lieberherr 1CS 4800 Fall /7/2010

Problem Solving Polya: How to Solve It 2CS 4800 Fall /7/2010

Algorithmic Patterns Greedy Divide and Conquer Dynamic Programming Network Flow Intractability: Reductions and characterization of NP 3CS 4800 Fall /7/2010

Algorithmic thinking Making claims about algorithms Choosing from a predefined set of claims – claims that are refuted – claims that are defended Used: Scientific Community Game for Algorithmic domains – make learning interactive and second order – students who are better at the game are better at algorithms 12/7/2010CS 4800 Fall 20104

Summary of SCG Definitions Domain Problem Solution valid(Problem, Solution) quality(Problem, Solution) → [0,1] 5CS 4800 Fall 2010 Claim(Domain) Problems: PowerSet(Domain.Problem) q: Quality = [0,1] r: Resource = N + Rules of the Scientific Community: propose and oppose, be an active scholar, rules for reputation accumulation. Tournaments 12/7/2010

Formal: HSR Domain: – Problem: (n,k), k <= n. – Solution: Decision tree to determine highest safe rung. – quality(problem, solution): depth of decision tree / number of rungs – valid(problem, solution): at most k left branches,... 6CS 4800 Fall /7/2010

Formal: HSR Claim(Domain): – Alice claims ({(25,2)},9/25,5 seconds) {(25,2)}: set of problems (singleton) 9/25: quality 5 seconds: resource Refutation Protocol: – Bob refutes: only one problem: (25,2) – Alice: solves problem by providing decision tree t. – predicate: t is a valid decision tree for (25,2) of depth 9 CS 4800 Fall /7/2010

Protocol variations Example: Independent Set Domain – Problem: Directed graph G=(V,E) – Solution: V’ subset of V Claim – Problems: all directed graphs with < 10^6 nodes – quality: |V’|/|V| Protocol – Alice to Admin: G, AlgA(G) – Admin to Bob: G – Bob to Admin: AlgB(G) – Admin to both: AlgB(G), r[G,AlgB(G),AlgA(G)] 12/7/2010CS 4800 Fall Alice claims that her algorithm finds a solution at least as good as Bob’s.

Protocol variations Example: Knapsack Domain – Problem: Knapsack problem K(n,Items(Weights,Values),W) – Solution: subset of Items Claim – Problems: all knapsack problems with W < 10^3 – quality: |V’|/|V| Protocol – Alice to Admin: K, AlgA(K) – Admin to Bob: K – Bob to Admin: AlgB(K) including AlgB(K).resource – Admin to both: AlgB(K) including AlgB(K).resource, r[K,AlgB(K),AlgA(K)] 12/7/2010CS 4800 Fall Alice claims that her algorithm finds a solution at least as good as Bob’s and using fewer resources.

Languages for expressing algorithms Decision trees Finite state machines Pseudo code Programming languages 10CS 4800 Fall /7/2010

Prove properties about algorithm Find an invariant of the algorithm that helps to prove what you want. Example: Ford-Fulkerson Dynamic Programming: formulate recurrence for optimal solution in terms of optimal solutions for sub problems. Formulate algorithm so that it is easier to prove its correctness. 11CS 4800 Fall /7/2010

Intractability Problems that don’t have an algorithmic solution. E.g. is grammar ambiguous? Problems that don’t have an efficient algorithmic solution, e.g., NP-complete problems. Reduction: Y <= X : Y is reducible to X, Y reduces to X: an algorithm for X implies an algorithm for Y Polynomial Reduction: Y <=p X: a polynomial algorithm for X implies a polynomial algorithm for Y. 12CS 4800 Fall /7/2010

NP-complete Problem X 1.X in NP 2.for all Y in NP, Y <=p X. Every problem in NP can be reduced to X. Circuit Satisfiability is NP-complete. 13CS 4800 Fall /7/2010

More NP-complete problems If Y is NP-complete and X is in NP with the property that Y <=p X, then X is NP-complete. 14CS 4800 Fall /7/2010

Strategy for proving problem X to be NP-complete Prove that X in NP. Choose a problem Y that is NP-complete. Prove that Y <=p X. Cook and Karp (translation) reductions (page 473) Example of Cook reduction: search (FSAT) reduces to decision (SAT) 15CS 4800 Fall /7/2010

Interview question Given a set S and a predicate pred defined on subsets of S, find a subset ss of S satisfying pred. – Example pred: false on all subsets not having 3 elements. For 3 element subsets pred({a,b,c}) iff a+b+c=0. 16CS 4800 Fall /7/2010

Order Statistics Given a sequence of n numbers, find the kth largest element. O(n*log(n)) O(n*log(k)) O(n): median of the median 17CS 4800 Fall /7/2010

Expected number of satisfied constraints under random assignment Maximum Satisfiability Maximum Boolean CSP 18CS 4800 Fall /7/2010

Problem Snapshot Boolean CSP: constraint satisfaction problem – Each constraint uses a Boolean relation. – e.g. a Boolean relation 1in3(x y z) is satisfied iff exactly one of its parameters is true. Boolean MAX-CSP a multi-set of constraints. Maximize satisfied fraction. 19CS 4800 Fall /7/2010

3p(1-p) 2 for MAX-CSP({22}) 20CS 4800 Fall /7/2010

all the look-ahead polynomials for T Ball 21CS 4800 Fall /7/2010