Approximability & Proof Complexity Ryan O’Donnell Carnegie Mellon

Approximability & Proof Complexity Ryan O’Donnell Carnegie Mellon of optimization problems

Minimum Vertex-Cover M IN -VC(G) = min {|S| : S ⊆ V, S touches all edges of E}

Minimum Vertex-Cover M IN -VC(G) = min {|S| : S ⊆ V, S touches all edges of E}

“2-approximating” Min-VC Choose any maximal matching M ⊆ E MIN-VC(G) ≥ |M| Let S = {all endpoints in M}. It’s a vertex-cover (why?) satisfying |S| = 2|M| ≤ 2MIN-VC(G)

“Factor 2-certifying” Min-VC Choose any maximal matching M ⊆ E Output “MIN-VC(G) ≥ |M|” 1.Output bound is always correct. 2.Bound is always within factor α of truth. A “factor α-certification” algorithm:

Linear programming (LP) relaxation k = minimize: ∑ v ∈ V X v subject to: 0 ≤ X v ≤ 1for all v ∈ V X u + X v ≥ 1 for all (u,v) ∈ E Output “MIN-VC(G) ≥ k”

Matching algorithm, LP algorithm are both factor 2-certification algorithms. Are they also 1.99-certification algorithms? No. M IN -VC(K n ) =n−1 maximal |M| = n/2 LP value = n/2 KnKn

Matching algorithm, LP algorithm are both factor 2-certification algorithms. Are they also 1.99-certification algorithms? No. M IN -VC(K n ) =n−1 maximal |M| = n/2 LP value = n/2 Is there any poly-time 1.99-certification alg? KnKn

We don’t know. Best we know is: ∃ 1.36-certification alg (unless P=NP) [Dinur-Safra’02] Is 1.99-certifying Min-VC NP-hard?

Approximability & Proof Complexity

Resolution Refutes statements encoded… by boolean disjunctions Cutting planesby integer inequalities Nullstellensatz/ Polynomial Calculus [BIKPP’96,CEI’96] by polynomial equations Positivstellensatz/ Sum-of-Squares (SOS) [Grigoriev-Vorobjov’99] by polynomial inequalities ZFC (“Frege”)in ordinary math language

is infeasible if −1 = Q 0 + Q 1 P 1 + Q 2 P 2 + +Q m P m there exist “certifying polynomials” Q 0, …, Q m, each a sum of squares, s.t. we have the identity Positivstellensatz [Krivine’64,Stengle’73,Schmüdgen’91,Putinar’93,Wörmann’98] and only if assuming a mild technical condition

“M IN -VC(G) > k” X v 2 ≥ X v X v 2 ≤ X v X u +X v ≥ 1 for all (u,v) ∈ E ∑ v X v ≤ k infeasible −1 = Q 0 + Q 1 (k −∑ X v ) + ∑ Q uv (X u +X v −1) + ∃ certifying SOS polynomials Q such that for all v ∈ V ⇔ ⇔

Positivstellensatz / SOS proof system Suggested by Grigoriev and Vorobjov in The complexity of an SOS proof is the maximum degree of any Q i P i or Q 0. SOS d denotes the proof system restricted to degree d. (No longer complete.)

Example proof Theorem: The following system is infeasible: {X 2 ≤ 1, Y 2 ≤ 1, Z 2 ≤ 1, XY+YZ+ZX ≤ −2}. ZFC proof: Let f(X,Y,Z) = XY+YZ+ZX. Suppose X 2 ≤ 1; i.e., X ∈ [−1,1]. Since f is linear in X, it’s maximized if X = ±1. Similarly for Y and Z. Suffices to show f(±1,±1,±1) > −2. If all three inputs same, f is 3. If not all three inputs same, f is −1.

Example proof Theorem: The following system is infeasible: {X 2 ≤ 1, Y 2 ≤ 1, Z 2 ≤ 1, XY+YZ+ZX ≤ −2}. SOS d=4 proof: +

Show SOS d=4 certifies MIN-VC(K n ) ≥ n−1 (i.e., refutes MIN-VC(K n ) ≤ n−2). Exercise

SOS d is ‘automatizable’! Theorem: [Lasserre’00,Parrilo’00, cf. N.Shor’87] If a polynomial inequality system can be refuted in the SOS d proof system, the certifying Q i ’s can be found in n O(d) time (using semidefinite programming).

The strongest(?) automatizable proof system that we know SOS d is stronger than: Width-d Resolution Degree-d Nullstellensatz Basic LP relaxations Basic SDP relaxations d/2 rounds of Lovász-Schrijver LP/SDP hierarchy d/2 rounds of Sherali-Adams LP/SDP hierarchy (Doesn’t seem to be stronger than degree-d Polynomial Calculus.)

A very powerful poly-time algorithm for Vertex-Cover certification: Output the largest k ∈ [n] such that SOS d=1000 certifies “M IN -VC(G) > k”. Could this be a 1.99-certification algorithm? I.e., is it true that whenever MIN-VC(G) = β, ∃ degree-1000 Q i ’s certifying Min-VC(G) ≥ β/1.99?

Partial history of upper and lower bounds for SOS d

SOS d upper bounds, Nothing that we didn’t already know by other means. E.g., SDP is a.878-certification alg for Max-Cut ∵ SDP ≤ SOS d=4 ∴ SOS d=4 also.878-certifies Max-Cut

SOS d lower bounds,

[Grigoriev’99]: (indep. [Schoenbeck’08]) Consider a random system of O(n) 3-variable equations modulo 2. With very high probability… No assignment sats > 51% of equations Unless d = Ω(n), SOS d cannot refute “the system is totally satisfiable”. Tseitin Tautologies / 3Lin(mod 2)

[Grigoriev’01]: See also [Laurent’02], [Cheung’05] “If n is odd and X 1, …, X n satisfy X i 2 = 1, then X X n cannot be 0.” Not provable in SOS d unless d ≥ n+1. ‘Knapsack’ (Essentially equivalent: “MAX-CUT(K n ) ≥ ”) Open Problem: Give a pleasant proof that d needs to be at least, say, 6.

(A corollary of the 3Lin(mod 2) lower bound.) (Not rigorously proven, but seems true in all cases.) [Tulsiani’09] Rule of Thumb For any factor-α certification problem which we know is NP-hard, there exists instances which require degree-n Ω(1) SOS proofs.

A very powerful poly-time algorithm for Vertex-Cover certification: Output the largest k ∈ [n] such that SOS d=1000 certifies “M IN -VC(G) > k”. Could this be a 1.99-certification algorithm?

Integrality Gaps [GK’95] SDP does not 2-certify Min-VC:Frankl-Rödl graphs [FS’00] SDP does not.879-certify Max-Cut:Noisy-sphere graphs [KV’05] SDP+∆ does not.879-certify Max-Cut or solve Unique-Games: KV noisy-hypercube graphs [DKSV’06] SDP+∆ does not O(1)-certify Sparsest-Cut (Balanced-Separator): DKSV noisy-hypercube graphs [KS’09,RS’09] Sherali-Adams +, degree-O(1), does not.879-certify Max-Cut or solve Unique-Games: KV noisy-hypercube graphs [BCGM’11] Sherali-Adams +, degree-6, (and probably degree-O(1)) does not 2-certify Min-VC Frankl-Rödl graphs

These tricky instances aren’t so hard for SOS!

[BBHKSZ’12]: SOS d=4 solves the KV Unique-Games instances! [OZ’13]: SOS d=4 solves the DKSV Balanced-Separator instances. [OZ’13]: SOS d=O(1).95-certifies the KV Max-Cut instances.

[BBHKSZ’12]: SOS d=4 solves the KV Unique-Games instances! [OZ’13]: SOS d=4 solves the DKSV Balanced-Separator instances. [DMN’13]: SOS d=O(1) solves the KV Max-Cut instances. [KV’05]: used ZFC to show “MAX-CUT(KV) ≈ 85%” [KS’09,RS’09]: SA + d=O(1) only certify “MAX-CUT(KV) ≥ 75%” [DMN’13] SOS d certifies “MAX-CUT(KV) ≥ 85% − o d (1)”

[BBHKSZ’12]: SOS d=4 solves the KV Unique-Games instances! [OZ’13]: SOS d=4 solves the DKSV Balanced-Separator instances. [DMN’13]: SOS d=O(1) solves the KV Max-Cut instances. [KOTZ’13]: SOS d=O(1) solves “most of” the Frankl-Rödl Min-VC instances.

The whole result is just that one particular algorithm does well on one particular instance? I have 3 responses.

Response 1: an old joke Q:Why did the complexity theorist work on algorithms? A:To get lower bounds on his lower bounds. We basically no longer know any “hard instances”.

Response 2: Evidence for algorithmic optimism? [BBHKSZ+’13] points out that as far as we know, SOS d=4 solves the Unique-Games problem (i.e., refutes the UGC). Perhaps SOS d is the killer algorithm for combinatorial optimization.

Response 3: New proofs Proving known theorems in restricted proof systems can lead to new insights and proofs. [Razborov’93]: New Switching Lemma proof [BBHKSZ’12, Hypercontractivity insights OZ’13,KOTZ’13]: [BHM’12,KOTZ’13]: New Frankl-Rödl Thm. proof [MN’13,DMN’13]:New Maj. is Stablest proof

Let’s take stock Approximation Algs ≤ Proof Complexity: “Is there an efficient algorithm for 100-coloring a 3-colorable graph?”

Let’s take stock Approximation Algs ≤ Proof Complexity: “Given a graph that is not 100-colorable, how hard is it to prove that it’s not 3-colorable?” SOS d is a quirky yet strong proof system, automatizable in time n O(d). SOS d=O(1) solves all the trickiest instances we know of Unique-Games, Max-Cut.

Three closing thoughts regarding proof complexity 1.Frankl-Rödl graphs and SOS lower bounds 2.The Dynamic SOS proof system 3.My favorite algorithm for Unique-Games

Frankl-Rödl graphs FR m (γ):V = {0,1} m E = {(x,y) : Δ(x,y) ≥ (1−γ) m} [Frankl-Rödl’87]: MIN-VC(FR m (γ)) ≥ (1−o(1)) 2 m [KOTZ’13]: SOS d=O(1/γ) can prove this. But perhaps SOS d=O(1) cannot handle

A simpler open problem Theorem: (a corollary of [Harper’66]’s Vertex-Isoperimetric Inequality) Let A, B ⊆ {0,1} m satisfy dist(A,B) ≥ Then |A|, |B| aren’t both large: Conjecture: SOS d=4 cannot prove this.

Dynamic SOS Lines of the proof are of form P(X 1, …, X n ) ≥ 0. From P ≥ 0 and Q ≥ 0 can derive P + Q ≥ 0 and P Q ≥ 0. Can always derive R 2 ≥ 0. To refute a system {P 1 ≥ 0, …, P m ≥ 0}, derive −1 ≥ 0. Complexity: max degree of any line

Dynamic SOS Facts [Grigoriev-Hirsch-Pasechnik’01]: Dynamic SOS d=3 refutes Knapsack Dynamic SOS d=5 refutes any unsatisfiable 3XOR instance Open problem 1 [GHP’01] : Suggest an explicit unsatisfiable boolean formula which SOS d=O(1) might not refute. Open problem 2: Give negative evidence re automatizability.

Unique-Games [Khot’02] conjectured that for the “UG” CSP, it’s NP-hard to distinguish -satisfiable instances from (1−)-satisfiable instances. [BBHKSZ’12]: Perhaps SOS d=4 can actually do it. ⇒ UGC is false (assuming NP ≠ P) Perhaps SOS d=log(n) can do it. ⇒ UGC is false (assuming NP ⊈ TIME[n log n ]) Why be so concerned about automatizability?

Unique-Games My favorite UG algorithm: Given an -satisfiable instance, nondeterministically guess a poly-length ZFC proof that instance is ≤ (1−)-satisfiable. [Khot’02] conjectured that for the “UG” CSP, it’s NP-hard to distinguish -satisfiable instances from (1−)-satisfiable instances. If this algorithm works, UGC is false. (assuming NP ≠ coNP)

ありがとうございました！ Thank you!