Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon

Basic Optimization Problems Minimum-Balanced-Separator: Given G=(V,E), partition V into 2 parts, each of size at least n/3, minimize # of edges crossing partition.

Basic Optimization Problems Minimum-Balanced-Separator: Minimum-Vertex-Cover: Given G=(V,E), partition V into 2 parts, each of size at least n/3, minimize # of edges crossing partition. Given G=(V,E), choose the smallest subset S ⊆ V such that each edge touches S.

Both are NP-hard n-vbl 3SAT formula F O(n)-vtx graph G poly(n) time, β F satisfiable F unsatisfiable ⇒ ⇒ Min-BS(G) = β Min-BS(G) > β Distinguishing requires* at least 2 Ω(n) time. ⇒

Approximate Optimization “C-approximation algorithm” Guaranteed to find a solution with value at most C times the minimum. “C-certification algorithm” Output form: “I certify the minimum is ≥ α”. Must always be correct. Guaranteed that α ≥ (true minimum) / C. Stronger

Minimum Balanced-Separator Is there a 1.01-approximation algorithm running in O(n) time? Is there a certification algorithm running in 2 n.99 time? DON’T KNOW [AMS’11]: Cannot* certify in poly(n) time.

Minimum Vertex-Cover Can 2-approximate in linear time. Cannot* 1.17-certify even in 2 n time. Cannot* 1.36-certify even in 2 n time. Can you 1.5-certify in polynomial time? DON’T KNOW

How could you show that you can’t 1.5-certify Min-VC in poly time? n-vbl 3SAT formula F O(n)-vtx graph G poly(n) time, β F satisfiable F unsatisfiable ⇒ ⇒ Min-VC(G) ≤ β Min-VC(G) > 1.5 β This would show 1.5-certifying Min-VC requires* superpolynomial time. DON’T KNOW HOW

How could you show that you can’t 1.5-certify Min-VC in poly time? give evidence that Show that known powerful poly-time optimization techniques fail to do it.

Prehistory:Linear programming can’t certify Min-VC. [ABL’02]: Lovász-Schrijver d Super-LP can’t certify Min-VC. [GK’95]: Semidefinite programming can’t certify Min-VC. [GMPT’07]: Lovász-Schrijver d Super-SDP can’t certify Min-VC. [BCGM’10]: Sherali-Adams d Super-Duper-SDP can’t certify Min-VC. +++ n O(d) time

[KS’09], [RS’09]: Sherali-Adams d Super-Duper-SDP can’t certify Min-Bal-Sep. For Min-Balanced-Separator, a similar situation:

Prehistory:Linear programming can’t certify Min-VC. I.e., there are graphs G on n vertices such that: Min-VC(G) ≥ n LP(G) = “I certify Min-VC(G) ≥ n” α = minimize: ∑ v ∈ V X v subject to: X v ∈ {0,1} for all v ∈ V X u + X v ≥ 1 for all (u,v) ∈ E [0,1] LP certif. alg. for Min-VC outputs α, where

I.e., there are graphs G on n vertices such that: Min-VC(G) ≥ n SA d (G) = “I certify Min-VC(G) ≥ n” [BCGM’10]: Sherali-Adams d Super-Duper-SDP can’t certify Min-VC. Specifically, this is true for “Frankl-Rödl graphs” [FR’87] : V = {0,1} m, E = {(x,y) : ∆(x,y)=.999 m}

I.e., there are graphs G on n vertices such that: Min-BS(G) ≥ β SA k (G) = “I certify Min-BS(G) ≥ ”. Specifically, this is true for “Khot-Vishnoi graphs” [KV’05]. [KS’09], [RS’09]: Sherali-Adams d Super-Duper-SDP can’t certify Min-Bal-Sep.

These are tough instances. We, the mathematicians, can analyze their opt. But our strongest poly-time algorithms cannot.

Actually…

There is one more algorithm…

It’s even stronger, but hard to analyze…

The “Lasserre d Super-Duper-Ultra-SDP”…

Also known as… SOS d n O(d) time

Our Results [OZ’13]:SOS 4 is a C-certification algorithm (for some small C, maybe 5) for Min-BS on Khot-Vishnoi graphs. SOS d is also pretty good for Max-Cut on Khot-Vishnoi graphs. [KOTZ’13]:SOS d is essentially a 1-certif. alg. for Min-VC on all but the ‘hardest’ Frankl-Rödl graphs.

So your whole result is that one particular algorithm does well on one particular instance?

An Old Joke Q:Why did the complexity theorist work on algorithms? A:To get lower bounds on his lower bounds. SOS d is a dozen years old, but hard to analyze. The Dream: it’s great certification alg. not just for these known hard graphs, but for all graphs.

Our Inspiration: STOC’12 paper of Barak, Brandão, Harrow, Kelner, Steurer, and Zhou. Showed SOS 4 is good certification alg. on known hard instances of “Unique-Games”. Somewhat demystified analysis of SOS d.

So what is SOS d ?

“Min-Balanced-Separator(G) > α” ⇔ has no real solutions ” “

infeasibility certificate: identity −1 = Q 0 + Q 1 P 1 + Q 2 P 2 + +Q m P m where each Q i is a “sum of squares”: Q i = R i R ik 2

Positivstellensatz Subject to some mild technical conditions, every infeasible system has such a certificate. Caveat: Q i ’s might need to have high degree. SOS d algorithm: [Shor’87,Lasserre’00,Parrilo’00] If there exists an infeasibility certificate where all the Q i ’s have degree ≤ d, finds it in time n O(d).

E.g.: SOS d for Min-VC(G) “Min-VC(G) > α” ⇔ X v 2 = X v for all v ∈ V, X u +X v ≥ 1 for all (u,v) ∈ E, ∑ v X v ≤ α infeasible −1 = Q 0 + Q 1 (α−∑ X v ) + ∑ Q uv (X u +X v −1) + existence of sum-of-squares Q’s such that Find largest α such that degree-d Q’s exist. ⇐

Our Results [OZ’13]:SOS 4 is a C-certification algorithm (for some small C, maybe 5) for Min-BS on Khot-Vishnoi graphs. I.e., for Khot-Vishnoi graphs G, there are degree-4 SOS Q’s certifying “Min-Bal-Sep(G) > α” for some α > (true Min-Bal-Sep) / C.

One Slide How-To Thm: Min-VC in this graph is ≥.999n Proof: … vertex isoperimetry… … inductive argument… Thm: Min-BS in this graph is ≥ blah Proof: … hypercontractivity… “Check out these polynomials.”

Tiny Taste A bit of the analysis for Max-Cut: Lemma: Let a,b,c ∈ {−1,1}. If a ≠ c then either a ≠ b or b ≠ c. Formalization with polynomials: SOS Proof:

Open Problems Can you give an SOS proof of… Vertex Isoperimetric Theorem in {0,1} n : If A, B ⊆ {0,1} n, |A|,|B| ≥.1·2 n, then ∃ x ∈ A,y ∈ B with ∆(x,y) ≤ Central Limit Theorem

Thanks! +