2. UNIT III DYNAMIC PROGRAMMING ALGORITHMS
Developing a Dynamic Programming Algorithm-Subtle Points- Question for the Little Bird- Sub instances and Sub solutions-Set of Substances-Decreasing Time and Space-Number of Solutions-Code. Reductions and NP-Completeness-Satisfiability-Proving NP-Completeness- 3-Coloring- Bipartite Matching. Randomized Algorithms-Randomness to Hide Worst Cases- Optimization Problems with a Random Structure.

3. Dynamic Programming A hard topic.
I try to provide a unified way to think of it and a fixed set of steps to follow.
Even if you don’t get the details of the algorithm correct, at least get the right structure.
I provide analogies (little bird) to make it hopefully more fun & easier to follow.

4. Non-Deterministic Poly-Time Decision Problems (NP)
An optimization problem
Each solution is either valid or not
(no cost)
The output is
Yes, it has a valid solution.
No, it does not
the solution is not returned
Eg: Given graph and integer <G,k>,
does G have a clique of size k?

5. an instance I (= <G,k>) and a solution S (= subset of nodes)
Key: Given
an instance I (= <G,k>)
and a solution S (= subset of nodes)
there is a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I.
Poly-time in |I| not in |S|. |S| can't be too big.
Not Valid
k=4
Valid
Formal definition:
Prob  NP iff  poly time Valid
such that Prob(I) =  S Valid(I,S)

6. Valid k=4 Key: If the instance has a valid solution
A non-deterministic (fairy god mother) could prove it to you by giving you
such a solution as a witness.
You could check that it is valid.
You could convince your boss.
k=4
Valid

7. Key:
If the instance does not have a valid solution
A non-deterministic (fairy god mother) could prove it to you by giving you
????
You have no way to convince your boss.
k=5

8. Non-Deterministic Poly-Time (NP)
Example: 3-Col:
Instance: A graph G.
Solution: Colouring C nodes of G with
3 colours such that every edge has two
colours.
G is a Yes instance if there is such a
colouring.
Given an instance G and a solution C, there is a poly-time alg Valid(G,C) to
test whether or not C is a valid 3-
colouring
of G.
 3-Col  NP.

9. Non-Deterministic Poly-Time (NP)
Example: Airplane Wing:
Instance: Requirements I of the wing.
Solution: A description S of how to make the wing.
I is a Yes instance if there is such a wing.
Given an instance I and a proof S, there is a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I.
Airplane Wing  NP.
I = [weight, lift, cost, …]

10. Reductions Palg ≤poly Poracle
Reduction: Design a fast algorithm for one computational problem, using a supposedly fast algorithm for another problem as a subroutine.

11. NP vs Co-NP Preserving Reductions
Definition is asymmetric.
There is a witness that a graph has a 3-Col.
There is no known witness that a graph has no 3-Col.
Cook Reduction:
Design any fast algorithm for Palg using a supposed fast algorithm for Poracle as a subroutine.
Karp Reduction:
The algorithm for Palg calls that for Poracle only once
Yes  Yes & No  No

12. Karp Reduction: Yes  Yes & No  No
We will only consider reductions of this simple form. Because they preserve NP vs Co-NP

13. Problem Pnew is NP-Complete Pnew not too hard.
Pnew  NP

14. Test in poly-time if a given solution is valid
NP-Complete Problems
Computable
Exp
Poly
Test in poly-time if a given solution is valid NP
Sat
complete
Pnew
Known
GCD

15. NP-Complete Problems Problem Pnew is NP-Complete Pnew not too hard.
Pnew  NP
Pnew sufficiently hard.
PNP, P ≤poly Pnew
Easier: Sat ≤poly Pnew
Cook: P ≤poly Sat
Computable
Exp
Poly NP
Sat
complete
Pnew
Known
GCD

16. K-Clique vs K-Independent Set
Clique: Given <G,k>, does G contains a k-clique?
Independent Set: Given <G,k>, does G contains a k-Ind Set?
A K-independent set is a set of K nodes with no edges between them.
A K-clique is a set of K nodes with all edges between them.

17. K-Clique vs K-Independent Set
Clique: Given <G,k>, does G contains a k-clique?
Independent Set: Given <G,k>, does G contains a k-Ind Set?
Brute Force: Try out all n choose k possible subsets.
If k = (n) then 2(n) subsets to check
If k=3 then
only O(n3)

18. K-Clique vs K-Independent Set
Clique ≤poly Indep Set
<G,k>
G has a k clique
or not
GIVEN: Indep. Set Oracle
BUILD: Clique Oracle
<G*, k>
G* has a k Indep. set
or not

19. K-Clique vs K-Independent Set
Clique ≤poly Indep Set
<G,k>
G has a k clique
or not
GIVEN: Indep. Set Oracle
BUILD: Clique Oracle
<G*, k>
Proof of correctness:
Our oracle says yes to <G,k>
iff Old oracle says yes to <G*,k>
iff G* has a k indep. set
iff G has a k clique
G* has a k Indep. set
or not

20. K-Clique vs K-Independent SetG* G
G* has edge if and only if
G does not

21. K-Clique vs K-Independent Set
This graph contains a clique of size 4. G
if and only if G*
This graph contains an independent set of size 4.

22. 12 Steps Steps for proving that Pnew is NP-Complete
1) I am addicted to solving NP-Complete Problems
2) I trust in my higher power to help
A witness

23. 12 Steps Steps for proving that Pnew is NP-Complete 0) Pnew  NP
1) What to reduce it to
2) What is what
3) Direction of reduction & Code
4) Look for similarities
5) Instance Map
6) Solution Map
7) Valid to Valid
8) Reverse Solution Map
9) Valid to Valid
10) Working Algorithm
11) Running Time

24. Design poly-time algorithm Valid(I,S)
12 Steps
Step 0: Pnew  NP
Formal definition:
Pnew  NP iff  poly time Valid
such that Pnew(I) =  S Valid(I,S)
Design poly-time algorithm Valid(I,S)
which determines whether a given solution S
is valid for a given instance I.

25. Step 1: What to reduce it to
12 Steps
Step 1: What to reduce it to
Palg ≤poly Poracle
Choose a problem Pis NP-comp
that is as similar to Pnew as possible and
that is already known to be NP-Complete. ..

26. = Independent Set = Clique k=3 k=4 12 Steps Step 2: What is what Pnew
Pis NP-comp
= Clique
Inew
Snew
Iis NP-comp
Sis NP-comp
k=3
k=4 ..

27. Step 3: Direction of reduction & Code
12 Steps
Step 3: Direction of reduction & Code
Reduce Pnew to Pis NP-comp or Pis NP-comp to Pnew?
Pis NP-comp ≤poly Pnew
Palg ≤poly Poracle ..

28. Step 3: Direction of reduction & Code
12 Steps
Step 3: Direction of reduction & Code ..

29. Step 4: Look for similarities
12 Steps
Step 4: Look for similarities
Clique
Independent Set ..
Both instances are graphs: nodes & edges
A clique solution is a subset of the nodes with edges.
An Ind Set solution is a subset of the nodes without edges.

30. 12 Steps
Step 5: Instance Map ..

31. 12 Steps
Step 5: Instance Map
Ialg
Ioracle = InstanceMap(Ialg)

32. no yes Yes mapped to Yes No to No no yes 12 Steps Step 5: Instance Map
Ialg
Yes mapped to Yes
No to No
Ioracle no
yes

33. 12 Steps
Step 6: Solution Map ..

34. Potential Solution = SolutionMap(S). Potential Solution S. 12 Steps
Step 6: Solution Map
Potential Solution
= SolutionMap(S).
Ialg
Ioracle
Potential Solution S.

35. Salg = SolutionMap(Soracle) Valid
12 Steps
Step 7: Valid to Valid
Salg = SolutionMap(Soracle)
Ialg
Valid
Ioracle = InstanceMap(Ialg)
If Soracle is valid solution for Ioracle, then Salg is valid solution for Ialg
Soracle
Valid

36. Step 8: Reverse Solution Map
12 Steps
Step 8: Reverse Solution Map
Potential Solution S.
Ialg
Potential Solution
= ReverseSolutionMap(S).
Ioracle
Not part of code,
but of proof.

37. Soracle = ReverseSolutionMap(Salg)
12 Steps
Step 9: Valid to Valid
Salg
Ialg
Valid
Valid
Ioracle = InstanceMap(Ialg)
If Salg is valid solution for Ialg, then Soracle is valid solution for Ioracle
Soracle = ReverseSolutionMap(Salg)

38. Step 10: Working Algorithm
12 Steps
Step 10: Working Algorithm
Our Algalg says yes to Ialg
iff old Algoracle says yes to Ioracle
iff Ioracle has a valid solution Soracle
iff Ialg has a valid solution Salg
iff Ialg is a yes instance. ..
Steps 8&9
Steps 6&7

39. Salg = SolutionMap(Soracle)
12 Steps
Step 11: Running Time
Algalg(Ialg) is poly time if the following are poly time:
Algoracle(Ioracle)
Assumed
Ioracle = InstanceMap(Ialg)
Your job
Salg = SolutionMap(Soracle)

40. Circuit Satisfiability
Palg ≤poly Poracle
or Palg Poracle
Graph Colouring
Scheduling
Circuit Satisfiability
Any NP-Problem
Clique
Independent Set
This will prove that
Cir-SAT is NP-Complete.

41. Reduction I I don’t even know what problem I am trying to solve!!! ?
Any NP-problem ≤poly Cir-SAT I
I don’t even know what problem I am trying to solve!!!
BUILD: Oracle for arbitrary NP Problem
GIVEN: Oracle for Cir-Sat ?

42. We need to solve some unknown NP-Problem.
What do we know about it?
Formal definition:
Parbitrary  NP iff  poly time Valid
such that Parbitrary(I) =  S Valid(I,S)
We have a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I.
Valid
k=4

43. Reduction I Please, give me a solution S such that Valid(I,S) is true.
Any NP-problem ≤poly Cir-SAT I
Please, give me a solution S such that Valid(I,S) is true.
BUILD: Oracle for arbitrary NP Problem
GIVEN: Oracle for Cir-Sat
That looks like a Turing Machine.
I only know about
circuits

44. The Circuit Satisfiability Problem
An instance is a circuit C.
A solution is an assignment X = [F,T,F…].
C(X) evaluates to T or F. x3 x2 x1 OR
AND
NOT F T F T
One bit output No feedback

45. The Circuit Satisfiability Problem
An instance is a circuit C.
A solution is an assignment X = [F,T,F…].
A valid solution has C(X) = True. x3 x2 x1 OR
AND
NOT F F T
Given a circuit, does it have a
satisfying assignment?

46. The Circuit Satisfiability Problem
Turing (and friends) prove that any algorithm computed by a
JAVA program in poly-time
can be computed by a Turing Machine in poly-time.
Very Powerful
Cook proves that any algorithm
computed by a Turing Machine
in time T(n) can be computed
by a family of circuits of size [T(n)]2.
But clearly, circuits compute.

47. Reduction I I build a circuit VI(S) equivalent to Valid(I,S)
Any NP-problem ≤poly Cir-SAT I
I build a circuit VI(S)
equivalent to Valid(I,S)
Please, give me a solution S such that VI(S) is true.
BUILD: Oracle for arbitrary NP Problem
GIVEN: Oracle for Cir-Sat
Thanks for the circuit.
But what is S?
I can only find assignments.

48. Reduction I I build a circuit VI(S) equivalent to Valid(I,S)
Any NP-problem ≤poly Cir-SAT I
I build a circuit VI(S)
equivalent to Valid(I,S)
I decode X into S and am done.
Please, give me an assignment X
such that VI(X) is true.
BUILD: Oracle for arbitrary NP Problem
GIVEN: Oracle for Cir-Sat
My pleasure.
Here: X

49. 12 Step Program
Let's be more formal.
And flow the 12 steps.

50. Step 0: Cir-SAT  NP Design poly-time algorithm ValidCir-SAT(C,X)
which determines whether a given assignment X
is valid for a given circuit C.
Easy: Evaluate C(X).

51. Step 1: What to reduce it to
Palg ≤poly Cir-Sat
Choose a problem Pis NP-comp
that is as similar to Cir-Sat as possible and
that is already known to be NP-Complete.
Have none. ..
Any NP-problem ≤poly Cir-SAT
We have a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I.

52. Step 2: What is what x3 x2 x1 = Circuit-Sat = I = S F AND NOT OR Pnew
Parbitrary
= some NP problem
Inew
Snew
Iarbitrary
Sarbitrary
= I
= S x3 x2 x1 OR
AND
NOT F ..

53. Step 3: Direction of reduction & Code
Given oracle for Cir-Sat,
we need to be able to solve any NP-Complete problem.
Parbitrary ..
Cir-Sat

54. Step 4: Look for similaritiesx3 x2 x1 OR
AND
NOT F .. ?

55. Step 5: Instance Map ..

56. Step 5: Instance Map I  circuit
We have a poly-time alg Valid(I,S) to test whether or not S is a valid solution for I.
Let Validn(I,S) be a circuit:
I is a bit string representing an instance I.
S is a bit string representing a solution S.
Outputs T if S is an encoding of a valid solution of I.
G,k
Eg: Clique S I
{u.v}  E OR uS OR vS
|S|  k
And over pairs of nodes

57. Outputs T if S is an encoding of a valid solution S of I
Step 5: Instance Map
I  circuit
Given an instance I
Circuit VI(S) = Valid(I,S) I S
hard wired
Outputs T if S is an encoding of a valid solution S of I
Validn(I,S) I
VI(S)

58. Step 6: Solution Map ..

59. Step 6: Solution Map S  assignment X=[T,F,F,T,F,T]
X is viewed as a bit string S representing a solution S.
If X is not a bit string representing a solution then “what ever”
Outputs T if S is an encoding of a valid solution S of I I S
hard wired
VI(S)

60. Outputs T if S is an encoding of a valid solution S of I
Step 6: Solution Map
S  assignment
X=[T,F,F,T,F,T]
X is viewed as a bit string S representing a solution S.
solution S
Outputs T if S is an encoding of a valid solution S of I I S
hard wired
VI(S)

61. Outputs T if S is an encoding of a valid solution S of I
Step 7: Valid  Valid
VI(X) = T
Valid(I,S) = Validn(I,S) = VI(X) = T
S is a valid solution of I.
Outputs T if S is an encoding of a valid solution S of I I S
hard wired
VI(S)

62. Outputs T if S is an encoding of a valid solution S of I
Step 8: Rev. Sol. Map
solution  assignment
solution S
S bit string representing a solution S
X=[T,F,F,T,F,T]
Outputs T if S is an encoding of a valid solution S of I I S
hard wired
VI(S)

63. Outputs T if S is an encoding of a valid solution S of I
Step 9: Valid  Valid
S is a valid solution of I.
VI(X) = Validn(I,S) = Valid(I,S) = T
VI(X) = T
Outputs T if S is an encoding of a valid solution S of I I S
hard wired
VI(S)

64. GIVEN: Alg for circuit problem
Reduction
Any NP-problem ≤poly Cir-SAT I
Yes/No
i.e.  S, S is a valid solution for I
BUILD: Opt. problem
GIVEN: Alg for circuit problem VI
satisfiable or not
i.e.  X, VI(X)

65. Circuit Satisfiability
Palg ≤poly Poracle
or Palg Poracle
3-Col ?
Graph Colouring
Circuit Satisfiability
Scheduling
Any NP-Problem
Clique
Independent Set

66. Graph Col: Given graph G & k can G be coloured with k colours?
can G be coloured with 3 colours?
3-Col ≤poly Graph Col
If you can decide whether you can colour with k colours,
then you can decide whether you can colour with 3 colours.
G,k C G C

67. Circuit Satisfiability
Palg ≤poly Poracle
or Palg Poracle
3-Col
3-SAT ?
Graph Colouring
Circuit Satisfiability
Scheduling
Any NP-Problem
Clique
Independent Set

68. Cir Sat: Given circuit C does it have a satisfying assignment X?
3-SAT: Given an expression:
A circuit consisting of a big AND of clauses
Each clause is the OR of at most 3 literals
Each literal is a variable or its negation. F T F T
xoryorz AND xorwora AND …

69. Cir Sat: Given circuit C does it have a satisfying assignment X?
3-SAT: Given an expression:
3-Sat ≤poly Cir Sat
If you can decide whether any circuit is satifiable, then you can decide whether a circuit that happens to be an expression is satisfiable. F T F T
xoryorz AND xorwora AND …

70. Circuit Satisfiability
Palg ≤poly Poracle
or Palg Poracle
3-Col
3-SAT ?
Graph Colouring
Circuit Satisfiability
Scheduling
Any NP-Problem
Clique
Independent Set
3-Col ≤poly 3-SAT
Graph Col ≤poly Cir SAT

71. Outputs T if C is an encoding of a valid 3-colouring C of G
Step 5: Instance Map
Graph G  Expression V
Given an instance G
Circuit VG(C) = Valid(G,C)
Outputs T if C is an encoding of a valid 3-colouring C of G
hard wired C G
Encoding C of C:
u is a node
r is a colour
x<u,r> = T if node u is colour r.

72. Step 5: Instance Map Given an instance G Graph G  Expression V G C
Circuit VG(C) = Valid(G,C)
hard wired C G
Encoding C of C:
u is a node
r is a colour
x<u,r> = T if node u is colour r.
x<u, r> = F OR x<v, r> = F
And over edges <u,v> and colours r
clauses with 2 literals

73. x<u, r> =T OR x<u, g>=T OR x<u, b>=T
Step 5: Instance Map
Graph G  Expression V
Given an instance G
Circuit VG(C) = Valid(G,C)
hard wired C G
Encoding C of C:
u is a node
r is a colour
x<u,r> = T if node u is colour r.
x<u, r> =T OR x<u, g>=T OR x<u, b>=T
And over node u
clauses with 3 literals