UNIT III DYNAMIC PROGRAMMING ALGORITHMS Developing a Dynamic Programming Algorithm-Subtle Points- Question for the Little Bird- Sub instances and Sub solutions-Set of Substances-Decreasing Time and Space-Number of Solutions-Code. Reductions and NP-Completeness-Satisfiability-Proving NP-Completeness- 3-Coloring- Bipartite Matching. Randomized Algorithms-Randomness to Hide Worst Cases- Optimization Problems with a Random Structure.

Dynamic Programming A hard topic. I try to provide a unified way to think of it and a fixed set of steps to follow. Even if you don’t get the details of the algorithm correct, at least get the right structure. I provide analogies (little bird) to make it hopefully more fun & easier to follow.

Non-Deterministic Poly-Time Decision Problems (NP) An optimization problem Each solution is either valid or not (no cost) The output is Yes, it has a valid solution. No, it does not the solution is not returned Eg: Given graph and integer <G,k>, does G have a clique of size k?

an instance I (= <G,k>) and a solution S (= subset of nodes) Key: Given an instance I (= <G,k>) and a solution S (= subset of nodes) there is a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I. Poly-time in |I| not in |S|. |S| can't be too big. Not Valid k=4 Valid Formal definition: Prob  NP iff  poly time Valid such that Prob(I) =  S Valid(I,S)

Valid k=4 Key: If the instance has a valid solution A non-deterministic (fairy god mother) could prove it to you by giving you such a solution as a witness. You could check that it is valid. You could convince your boss. k=4 Valid

Key: If the instance does not have a valid solution A non-deterministic (fairy god mother) could prove it to you by giving you ???? You have no way to convince your boss. k=5

Non-Deterministic Poly-Time (NP) Example: 3-Col: Instance: A graph G. Solution: Colouring C nodes of G with 3 colours such that every edge has two colours. G is a Yes instance if there is such a colouring. Given an instance G and a solution C, there is a poly-time alg Valid(G,C) to test whether or not C is a valid 3- colouring of G.  3-Col  NP.

Non-Deterministic Poly-Time (NP) Example: Airplane Wing: Instance: Requirements I of the wing. Solution: A description S of how to make the wing. I is a Yes instance if there is such a wing. Given an instance I and a proof S, there is a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I. Airplane Wing  NP. I = [weight, lift, cost, …]

Reductions Palg ≤poly Poracle Reduction: Design a fast algorithm for one computational problem, using a supposedly fast algorithm for another problem as a subroutine.

NP vs Co-NP Preserving Reductions Definition is asymmetric. There is a witness that a graph has a 3-Col. There is no known witness that a graph has no 3-Col. Cook Reduction: Design any fast algorithm for Palg using a supposed fast algorithm for Poracle as a subroutine. Karp Reduction: The algorithm for Palg calls that for Poracle only once Yes  Yes & No  No

Karp Reduction: Yes  Yes & No  No We will only consider reductions of this simple form. Because they preserve NP vs Co-NP

Problem Pnew is NP-Complete Pnew not too hard. Pnew  NP

Test in poly-time if a given solution is valid NP-Complete Problems Computable Exp Poly Test in poly-time if a given solution is valid NP Sat complete Pnew Known GCD

NP-Complete Problems Problem Pnew is NP-Complete Pnew not too hard. Pnew  NP Pnew sufficiently hard. PNP, P ≤poly Pnew Easier: Sat ≤poly Pnew Cook: P ≤poly Sat Computable Exp Poly NP Sat complete Pnew Known GCD

K-Clique vs K-Independent Set Clique: Given <G,k>, does G contains a k-clique? Independent Set: Given <G,k>, does G contains a k-Ind Set? A K-independent set is a set of K nodes with no edges between them. A K-clique is a set of K nodes with all edges between them.

K-Clique vs K-Independent Set Clique: Given <G,k>, does G contains a k-clique? Independent Set: Given <G,k>, does G contains a k-Ind Set? Brute Force: Try out all n choose k possible subsets. If k = (n) then 2(n) subsets to check If k=3 then only O(n3)

K-Clique vs K-Independent Set Clique ≤poly Indep Set <G,k> G has a k clique or not GIVEN: Indep. Set Oracle BUILD: Clique Oracle <G*, k> G* has a k Indep. set or not

K-Clique vs K-Independent Set Clique ≤poly Indep Set <G,k> G has a k clique or not GIVEN: Indep. Set Oracle BUILD: Clique Oracle <G*, k> Proof of correctness: Our oracle says yes to <G,k> iff Old oracle says yes to <G*,k> iff G* has a k indep. set iff G has a k clique G* has a k Indep. set or not

K-Clique vs K-Independent Set G* G G* has edge if and only if G does not

K-Clique vs K-Independent Set This graph contains a clique of size 4. G if and only if G* This graph contains an independent set of size 4.

12 Steps Steps for proving that Pnew is NP-Complete 1) I am addicted to solving NP-Complete Problems 2) I trust in my higher power to help A witness

12 Steps Steps for proving that Pnew is NP-Complete 0) Pnew  NP 1) What to reduce it to 2) What is what 3) Direction of reduction & Code 4) Look for similarities 5) Instance Map 6) Solution Map 7) Valid to Valid 8) Reverse Solution Map 9) Valid to Valid 10) Working Algorithm 11) Running Time

Design poly-time algorithm Valid(I,S) 12 Steps Step 0: Pnew  NP Formal definition: Pnew  NP iff  poly time Valid such that Pnew(I) =  S Valid(I,S) Design poly-time algorithm Valid(I,S) which determines whether a given solution S is valid for a given instance I.

Step 1: What to reduce it to 12 Steps Step 1: What to reduce it to Palg ≤poly Poracle Choose a problem Pis NP-comp that is as similar to Pnew as possible and that is already known to be NP-Complete. ..

= Independent Set = Clique k=3 k=4 12 Steps Step 2: What is what Pnew Pis NP-comp = Clique Inew Snew Iis NP-comp Sis NP-comp k=3 k=4 ..

Step 3: Direction of reduction & Code 12 Steps Step 3: Direction of reduction & Code Reduce Pnew to Pis NP-comp or Pis NP-comp to Pnew? Pis NP-comp ≤poly Pnew Palg ≤poly Poracle ..

Step 3: Direction of reduction & Code 12 Steps Step 3: Direction of reduction & Code ..

Step 4: Look for similarities 12 Steps Step 4: Look for similarities Clique Independent Set .. Both instances are graphs: nodes & edges A clique solution is a subset of the nodes with edges. An Ind Set solution is a subset of the nodes without edges.

12 Steps Step 5: Instance Map ..

12 Steps Step 5: Instance Map Ialg Ioracle = InstanceMap(Ialg)

no yes Yes mapped to Yes No to No no yes 12 Steps Step 5: Instance Map Ialg Yes mapped to Yes No to No Ioracle no yes

12 Steps Step 6: Solution Map ..

Potential Solution = SolutionMap(S). Potential Solution S. 12 Steps Step 6: Solution Map Potential Solution = SolutionMap(S). Ialg Ioracle Potential Solution S.

Salg = SolutionMap(Soracle) Valid 12 Steps Step 7: Valid to Valid Salg = SolutionMap(Soracle) Ialg Valid Ioracle = InstanceMap(Ialg) If Soracle is valid solution for Ioracle, then Salg is valid solution for Ialg Soracle Valid

Step 8: Reverse Solution Map 12 Steps Step 8: Reverse Solution Map Potential Solution S. Ialg Potential Solution = ReverseSolutionMap(S). Ioracle Not part of code, but of proof.

Soracle = ReverseSolutionMap(Salg) 12 Steps Step 9: Valid to Valid Salg Ialg Valid Valid Ioracle = InstanceMap(Ialg) If Salg is valid solution for Ialg, then Soracle is valid solution for Ioracle Soracle = ReverseSolutionMap(Salg)

Step 10: Working Algorithm 12 Steps Step 10: Working Algorithm Our Algalg says yes to Ialg iff old Algoracle says yes to Ioracle iff Ioracle has a valid solution Soracle iff Ialg has a valid solution Salg iff Ialg is a yes instance. .. Steps 8&9 Steps 6&7

Salg = SolutionMap(Soracle) 12 Steps Step 11: Running Time Algalg(Ialg) is poly time if the following are poly time: Algoracle(Ioracle) Assumed Ioracle = InstanceMap(Ialg) Your job Salg = SolutionMap(Soracle)

Circuit Satisfiability Palg ≤poly Poracle or Palg Poracle Graph Colouring Scheduling Circuit Satisfiability Any NP-Problem Clique Independent Set This will prove that Cir-SAT is NP-Complete.

Reduction I I don’t even know what problem I am trying to solve!!! ? Any NP-problem ≤poly Cir-SAT I I don’t even know what problem I am trying to solve!!! BUILD: Oracle for arbitrary NP Problem GIVEN: Oracle for Cir-Sat ?

We need to solve some unknown NP-Problem. What do we know about it? Formal definition: Parbitrary  NP iff  poly time Valid such that Parbitrary(I) =  S Valid(I,S) We have a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I. Valid k=4

Reduction I Please, give me a solution S such that Valid(I,S) is true. Any NP-problem ≤poly Cir-SAT I Please, give me a solution S such that Valid(I,S) is true. BUILD: Oracle for arbitrary NP Problem GIVEN: Oracle for Cir-Sat That looks like a Turing Machine. I only know about circuits

The Circuit Satisfiability Problem An instance is a circuit C. A solution is an assignment X = [F,T,F…]. C(X) evaluates to T or F. x3 x2 x1 OR AND NOT F T F T One bit output No feedback

The Circuit Satisfiability Problem An instance is a circuit C. A solution is an assignment X = [F,T,F…]. A valid solution has C(X) = True. x3 x2 x1 OR AND NOT F F T Given a circuit, does it have a satisfying assignment?

The Circuit Satisfiability Problem Turing (and friends) prove that any algorithm computed by a JAVA program in poly-time can be computed by a Turing Machine in poly-time. Very Powerful Cook proves that any algorithm computed by a Turing Machine in time T(n) can be computed by a family of circuits of size [T(n)]2. But clearly, circuits compute.

Reduction I I build a circuit VI(S) equivalent to Valid(I,S) Any NP-problem ≤poly Cir-SAT I I build a circuit VI(S) equivalent to Valid(I,S) Please, give me a solution S such that VI(S) is true. BUILD: Oracle for arbitrary NP Problem GIVEN: Oracle for Cir-Sat Thanks for the circuit. But what is S? I can only find assignments.

Reduction I I build a circuit VI(S) equivalent to Valid(I,S) Any NP-problem ≤poly Cir-SAT I I build a circuit VI(S) equivalent to Valid(I,S) I decode X into S and am done. Please, give me an assignment X such that VI(X) is true. BUILD: Oracle for arbitrary NP Problem GIVEN: Oracle for Cir-Sat My pleasure. Here: X

12 Step Program Let's be more formal. And flow the 12 steps.

Step 0: Cir-SAT  NP Design poly-time algorithm ValidCir-SAT(C,X) which determines whether a given assignment X is valid for a given circuit C. Easy: Evaluate C(X).

Step 1: What to reduce it to Palg ≤poly Cir-Sat Choose a problem Pis NP-comp that is as similar to Cir-Sat as possible and that is already known to be NP-Complete. Have none. .. Any NP-problem ≤poly Cir-SAT We have a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I.

Step 2: What is what x3 x2 x1 = Circuit-Sat = I = S F AND NOT OR Pnew Parbitrary = some NP problem Inew Snew Iarbitrary Sarbitrary = I = S x3 x2 x1 OR AND NOT F ..

Step 3: Direction of reduction & Code Given oracle for Cir-Sat, we need to be able to solve any NP-Complete problem. Parbitrary .. Cir-Sat

Step 4: Look for similarities x3 x2 x1 OR AND NOT F .. ?

Step 5: Instance Map ..

Step 5: Instance Map I  circuit We have a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I. Let Validn(I,S) be a circuit: I is a bit string representing an instance I. S is a bit string representing a solution S. Outputs T if S is an encoding of a valid solution of I. G,k Eg: Clique S I {u.v}  E OR uS OR vS |S|  k And over pairs of nodes

Outputs T if S is an encoding of a valid solution S of I Step 5: Instance Map I  circuit Given an instance I Circuit VI(S) = Valid(I,S) I S hard wired Outputs T if S is an encoding of a valid solution S of I Validn(I,S) I VI(S)

Step 6: Solution Map ..

Step 6: Solution Map S  assignment X=[T,F,F,T,F,T] X is viewed as a bit string S representing a solution S. If X is not a bit string representing a solution then “what ever” Outputs T if S is an encoding of a valid solution S of I I S hard wired VI(S)

Outputs T if S is an encoding of a valid solution S of I Step 6: Solution Map S  assignment X=[T,F,F,T,F,T] X is viewed as a bit string S representing a solution S. solution S Outputs T if S is an encoding of a valid solution S of I I S hard wired VI(S)

Outputs T if S is an encoding of a valid solution S of I Step 7: Valid  Valid VI(X) = T Valid(I,S) = Validn(I,S) = VI(X) = T S is a valid solution of I. Outputs T if S is an encoding of a valid solution S of I I S hard wired VI(S)

Outputs T if S is an encoding of a valid solution S of I Step 8: Rev. Sol. Map solution  assignment solution S S bit string representing a solution S X=[T,F,F,T,F,T] Outputs T if S is an encoding of a valid solution S of I I S hard wired VI(S)

Outputs T if S is an encoding of a valid solution S of I Step 9: Valid  Valid S is a valid solution of I. VI(X) = Validn(I,S) = Valid(I,S) = T VI(X) = T Outputs T if S is an encoding of a valid solution S of I I S hard wired VI(S)

GIVEN: Alg for circuit problem Reduction Any NP-problem ≤poly Cir-SAT I Yes/No i.e.  S, S is a valid solution for I BUILD: Opt. problem GIVEN: Alg for circuit problem VI satisfiable or not i.e.  X, VI(X)

Circuit Satisfiability Palg ≤poly Poracle or Palg Poracle 3-Col ? Graph Colouring Circuit Satisfiability Scheduling Any NP-Problem Clique Independent Set

Graph Col: Given graph G & k can G be coloured with k colours? can G be coloured with 3 colours? 3-Col ≤poly Graph Col If you can decide whether you can colour with k colours, then you can decide whether you can colour with 3 colours. G,k C G C

Circuit Satisfiability Palg ≤poly Poracle or Palg Poracle 3-Col 3-SAT ? Graph Colouring Circuit Satisfiability Scheduling Any NP-Problem Clique Independent Set

Cir Sat: Given circuit C does it have a satisfying assignment X? 3-SAT: Given an expression: A circuit consisting of a big AND of clauses Each clause is the OR of at most 3 literals Each literal is a variable or its negation. F T F T xoryorz AND xorwora AND …

Cir Sat: Given circuit C does it have a satisfying assignment X? 3-SAT: Given an expression: 3-Sat ≤poly Cir Sat If you can decide whether any circuit is satifiable, then you can decide whether a circuit that happens to be an expression is satisfiable. F T F T xoryorz AND xorwora AND …

Circuit Satisfiability Palg ≤poly Poracle or Palg Poracle 3-Col 3-SAT ? Graph Colouring Circuit Satisfiability Scheduling Any NP-Problem Clique Independent Set 3-Col ≤poly 3-SAT Graph Col ≤poly Cir SAT »

Outputs T if C is an encoding of a valid 3-colouring C of G Step 5: Instance Map Graph G  Expression V Given an instance G Circuit VG(C) = Valid(G,C) Outputs T if C is an encoding of a valid 3-colouring C of G hard wired C G Encoding C of C: u is a node r is a colour x<u,r> = T if node u is colour r.

Step 5: Instance Map Given an instance G Graph G  Expression V G C Circuit VG(C) = Valid(G,C) hard wired C G Encoding C of C: u is a node r is a colour x<u,r> = T if node u is colour r. x<u, r> = F OR x<v, r> = F And over edges <u,v> and colours r clauses with 2 literals

x<u, r> =T OR x<u, g>=T OR x<u, b>=T Step 5: Instance Map Graph G  Expression V Given an instance G Circuit VG(C) = Valid(G,C) hard wired C G Encoding C of C: u is a node r is a colour x<u,r> = T if node u is colour r. x<u, r> =T OR x<u, g>=T OR x<u, b>=T And over node u clauses with 3 literals