1. COMP 482: Design and Analysis of Algorithms
Spring 2013
Lecture 20
Prof. Swarat Chaudhuri

2. Recap: Project Selection
can be positive or negative
Projects with prerequisites.
Set P of possible projects. Project v has associated revenue pv.
some projects generate money: create interactive e-commerce interface, redesign web page
others cost money: upgrade computers, get site license
Set of prerequisites E. If (v, w)  E, can't do project v and unless also do project w.
A subset of projects A  P is feasible if the prerequisite of every project in A also belongs to A.
Project selection. Choose a feasible subset of projects to maximize revenue.
project selection = max weight closure problem

3. Recap: Project Selection: Prerequisite Graph
Include an edge from v to w if can't do v without also doing w.
{v, w, x} is feasible subset of projects.
{v, x} is infeasible subset of projects. w w v x v x
feasible
infeasible

4. 7.10 Image Segmentation

5. Image Segmentation Image segmentation.
Central problem in image processing.
Divide image into coherent regions.
Ex: Three people standing in front of complex background scene. Identify each person as a coherent object.

6. Image Segmentation Foreground / background segmentation.
Label each pixel in picture as belonging to foreground or background.
V = set of pixels, E = pairs of neighboring pixels.
ai  0 is likelihood pixel i in foreground.
bi  0 is likelihood pixel i in background.
pij  0 is separation penalty for labeling one of i and j as foreground, and the other as background.
Goals.
Accuracy: if ai > bi in isolation, prefer to label i in foreground.
Smoothness: if many neighbors of i are labeled foreground, we should be inclined to label i as foreground.
Find partition (A, B) that maximizes:
foreground
background

7. Image Segmentation Properties of the problem. Maximization.
No source or sink.
Undirected graph.
Turn into minimization problem.
Maximizing is equivalent to minimizing
or alternatively

8. Q1
…Can you solve this problem using max flow or min-cut?

9. Image Segmentation Formulate as min cut problem. G' = (V', E').
Add source to correspond to foreground; add sink to correspond to background
Use two anti-parallel edges instead of undirected edge.
pij
pij
pij aj
pij s i j t bi G'

10. Image Segmentation Consider min cut (A, B) in G'. A = foreground.
Precisely the quantity we want to minimize.
if i and j on different sides,
pij counted exactly once aj
pij s i j t bi A G'

11. 7.12 Baseball Elimination

12. Baseball Elimination
Which teams have a chance of finishing the season with most wins?
Montreal eliminated since it can finish with at most 80 wins, but Atlanta already has 83.
wi + ri < wj  team i eliminated.
Only reason sports writers appear to be aware of.
Sufficient, but not necessary!
Team i
Wins wi
Losses li
To play ri
Against = rij
Atl
Phi NY
Mon
Atlanta 83 71 8 - 1 6 1
Philly 80 79 3 1 - 2
New York 78 78 6 6 -
Montreal 77 82 3 1 2 -
As season unfolds, sports fans are naturally interested in how their favorite teams are doing. Is there some outcome of remaining games in which your favorite team finishes the season with the most (or tied for most) wins?

13. Baseball Elimination
Which teams have a chance of finishing the season with most wins?
Philly can win 83, but still eliminated . . .
If Atlanta loses a game, then some other team wins one.
Remark. Answer depends not just on how many games already won and left to play, but also on whom they're against.
Team i
Wins wi
Losses li
To play ri
Against = rij
Atl
Phi NY
Mon
Atlanta 83 71 8 - 1 6 1
Philly 80 79 3 1 - 2
New York 78 78 6 6 -
Montreal 77 82 3 1 2 -

14. Baseball Elimination Baseball elimination problem. Set of teams S.
Distinguished team s  S.
Team x has won wx games already.
Teams x and y play each other rxy additional times.
Is there any outcome of the remaining games in which team s finishes with the most (or tied for the most) wins?

15. Baseball Elimination: Max Flow Formulation
Can team 3 finish with most wins?
Assume team 3 wins all remaining games  w3 + r3 wins.
Divvy remaining games so that all teams have  w3 + r3 wins.
1-2 1
team 4 can still win this many more games
1-4
games left 2 
1-5 s
2-4
r24 = 7  4
w3 + r3 - w4 t
2-5 5
game nodes
4-5
team nodes

16. Baseball Elimination: Max Flow Formulation
Theorem. Team 3 is not eliminated iff max flow saturates all edges leaving source.
Integrality theorem  each remaining game between x and y added to number of wins for team x or team y.
Capacity on (x, t) edges ensure no team wins too many games.
1-2 1
team 4 can still win this many more games
1-4
games left 2 
1-5 s
2-4
r24 = 7  4
w3 + r3 - w4 t
2-5 5
game nodes
4-5
team nodes

17. Ad analytics
Suppose you are running a website that is visited by the same set of people every day. Each visitor claims membership in one or more demographic groups; for example, a visitor might describe himself as male, years old, married, a resident of Houston, and an academic. Your site is supported by advertisers. Each advertiser has told you which demographic groups should see its ads and how many of its ads you must show each day. Altogether, there are n visitors, k demographic groups, and m advertisers. Describe an efficient algorithm to determine, given all the data described in the previous paragraph, whether you can show each visitor exactly one ad per day, so that every advertiser has its desired number of ads displayed, and every ad is seen by someone in an appropriate demographic group.

18. NP-completeness and computational intractability

19. Algorithm Design Patterns and Anti-Patterns
Algorithm design patterns. Ex.
Greed. O(n log n) interval scheduling.
Divide-and-conquer. O(n log n) FFT.
Dynamic programming. O(n2) edit distance.
Duality. O(n3) bipartite matching.
Reductions.
Local search.
Randomization.
Algorithm design anti-patterns.
NP-completeness. O(nk) algorithm unlikely.
PSPACE-completeness. O(nk) certification algorithm unlikely.
Undecidability. No algorithm possible.

20. 8.1 Polynomial-Time Reductions

21. Classify Problems According to Computational Requirements
Q. Which problems will we be able to solve in practice?
A working definition. [Cobham 1964, Edmonds 1965, Rabin 1966] Those with polynomial-time algorithms.
Yes
Probably no
Shortest path
Longest path
Matching
3D-matching
Min cut
Max cut
2-SAT
3-SAT
Planar 4-color
Planar 3-color
Bipartite vertex cover
Vertex cover
Primality testing
Factoring

22. Classify Problems
Desiderata. Classify problems according to those that can be solved in polynomial-time and those that cannot.
Provably requires exponential-time.
Given a Turing machine, does it halt in at most k steps?
Given a board position in an n-by-n generalization of chess, can black guarantee a win?
Frustrating news. Huge number of fundamental problems have defied classification for decades.
This chapter. Show that these fundamental problems are "computationally equivalent" and appear to be different manifestations of one really hard problem.

23. Polynomial-Time Reduction
Desiderata'. Suppose we could solve X in polynomial-time. What else could we solve in polynomial time?
Reduction. Problem X polynomial reduces to problem Y if arbitrary instances of problem X can be solved using:
Polynomial number of standard computational steps, plus
Polynomial number of calls to oracle that solves problem Y.
Notation. X  P Y.
Remarks.
We pay for time to write down instances sent to black box  instances of Y must be of polynomial size.
Note: Cook reducibility.
don't confuse with reduces from
Cook reduction = polynomial time Turing reduction
Karp reduction = polynomial many-one reduction = polynomial transformation
computational model supplemented by special piece of hardware that solves instances of Y in a single step
in contrast to Karp reductions

24. Polynomial-Time Reduction
Purpose. Classify problems according to relative difficulty.
Design algorithms. If X  P Y and Y can be solved in polynomial-time, then X can also be solved in polynomial time.
Establish intractability. If X  P Y and X cannot be solved in polynomial-time, then Y cannot be solved in polynomial time.
Establish equivalence. If X  P Y and Y  P X, we use notation X  P Y.
up to cost of reduction

25. Reduction By Simple Equivalence
Basic reduction strategies.
Reduction by simple equivalence.
Reduction from special case to general case.
Reduction by encoding with gadgets.

26. Independent Set Ex. Is there an independent set of size  6? Yes.
INDEPENDENT SET: Given a graph G = (V, E) and an integer k, is there a subset of vertices S  V such that |S|  k, and for each edge at most one of its endpoints is in S?
Ex. Is there an independent set of size  6? Yes.
Ex. Is there an independent set of size  7? No.
Application: find set of mutually non-conflicting points
independent set

27. Vertex Cover Ex. Is there a vertex cover of size  4? Yes.
VERTEX COVER: Given a graph G = (V, E) and an integer k, is there a subset of vertices S  V such that |S|  k, and for each edge, at least one of its endpoints is in S?
Ex. Is there a vertex cover of size  4? Yes.
Ex. Is there a vertex cover of size  3? No.
TOY application = art gallery problem: place guards within an art gallery so that all corridors are visible at any time
vertex cover

28. Vertex Cover and Independent Set
Claim. VERTEX-COVER P INDEPENDENT-SET.
Pf. We show S is an independent set iff V  S is a vertex cover.
independent set
vertex cover

29. Vertex Cover and Independent Set
Claim. VERTEX-COVER P INDEPENDENT-SET.
Pf. We show S is an independent set iff V  S is a vertex cover. 
Let S be any independent set.
Consider an arbitrary edge (u, v).
S independent  u  S or v  S  u  V  S or v  V  S.
Thus, V  S covers (u, v). 
Let V  S be any vertex cover.
Consider two nodes u  S and v  S.
Observe that (u, v)  E since V  S is a vertex cover.
Thus, no two nodes in S are joined by an edge  S independent set. ▪

30. Reduction from Special Case to General Case
Basic reduction strategies.
Reduction by simple equivalence.
Reduction from special case to general case.
Reduction by encoding with gadgets.

31. Set Cover Sample application. m available pieces of software.
SET COVER: Given a set U of elements, a collection S1, S2, , Sm of subsets of U, and an integer k, does there exist a collection of  k of these sets whose union is equal to U?
Sample application.
m available pieces of software.
Set U of n capabilities that we would like our system to have.
The ith piece of software provides the set Si  U of capabilities.
Goal: achieve all n capabilities using fewest pieces of software.
Ex:
U = { 1, 2, 3, 4, 5, 6, 7 } k = 2
S1 = {3, 7} S4 = {2, 4}
S2 = {3, 4, 5, 6} S5 = {5} S3 = {1} S6 = {1, 2, 6, 7}

32. Vertex Cover Reduces to Set Cover
Claim. VERTEX-COVER  P SET-COVER.
Pf. Given a VERTEX-COVER instance G = (V, E), k, we construct a set cover instance whose size equals the size of the vertex cover instance.
Construction.
Create SET-COVER instance:
k = k, U = E, Sv = {e  E : e incident to v }
Set-cover of size  k iff vertex cover of size  k. ▪
SET COVER
U = { 1, 2, 3, 4, 5, 6, 7 } k = 2
Sa = {3, 7} Sb = {2, 4}
Sc = {3, 4, 5, 6} Sd = {5} Se = {1} Sf= {1, 2, 6, 7}
VERTEX COVER a b e7 e2 e4 e3 f e6 c e1 e5
k = 2 e d

33. Q2: Hitting set Show that SET COVER polynomial reduces to HITTING SET.
HITTING SET: Given a set U of elements, a collection S1, S2, , Sm of subsets of U, and an integer k, does there exist a subset of U of size  k such that U overlaps with each of the sets S1, S2, , Sm?
Show that SET COVER polynomial reduces to HITTING SET.