Hypercontractivity & Sums of Squares

Slides:



Advertisements
Similar presentations
Iterative Rounding and Iterative Relaxation
Advertisements

Hypercontractive inequalities via SOS, and the Frankl-Rödl graph Manuel Kauers (Johannes Kepler Universität) Ryan ODonnell (Carnegie Mellon University)
Linear Round Integrality Gaps for the Lasserre Hierarchy Grant Schoenebeck.
Approximability & Proof Complexity Ryan O’Donnell Carnegie Mellon.
Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon.
Incremental Linear Programming Linear programming involves finding a solution to the constraints, one that maximizes the given linear function of variables.
1 Matching Polytope x1 x2 x3 Lecture 12: Feb 22 x1 x2 x3.
On the Density of a Graph and its Blowup Raphael Yuster Joint work with Asaf Shapira.
Approximation Algorithms Chapter 14: Rounding Applied to Set Cover.
C&O 355 Mathematical Programming Fall 2010 Lecture 22 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A.
1/17 Optimal Long Test with One Free Bit Nikhil Bansal (IBM) Subhash Khot (NYU)
Dictator tests and Hardness of approximating Max-Cut-Gain Ryan O’Donnell Carnegie Mellon (includes joint work with Subhash Khot of Georgia Tech)
Approximation Algoirthms: Semidefinite Programming Lecture 19: Mar 22.
A Linear Round Lower Bound for Lovasz-Schrijver SDP relaxations of Vertex Cover Grant Schoenebeck Luca Trevisan Madhur Tulsiani UC Berkeley.
Totally Unimodular Matrices Lecture 11: Feb 23 Simplex Algorithm Elliposid Algorithm.
Semidefinite Programming
Approximation Algorithm: Iterative Rounding Lecture 15: March 9.
Semidefinite Programming Based Approximation Algorithms Uri Zwick Uri Zwick Tel Aviv University UKCRC’02, Warwick University, May 3, 2002.
Matching Polytope, Stable Matching Polytope Lecture 8: Feb 2 x1 x2 x3 x1 x2 x3.
Linear Programming Relaxations for MaxCut Wenceslas Fernandez de la Vega Claire Kenyon -Mathieu.
(work appeared in SODA 10’) Yuk Hei Chan (Tom)
Tight Integrality Gaps for Lovász-Schrijver LP relaxations of Vertex Cover Grant Schoenebeck Luca Trevisan Madhur Tulsiani UC Berkeley.
Finding Almost-Perfect
Dana Moshkovitz, MIT Joint work with Subhash Khot, NYU.
Fractional decompositions of dense hypergraphs Raphael Yuster University of Haifa.
C&O 355 Lecture 2 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A.
C&O 355 Mathematical Programming Fall 2010 Lecture 2 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A.
The Polynomial Time Algorithm for Testing Primality George T. Gilbert.
Edge Covering problems with budget constrains By R. Gandhi and G. Kortsarz Presented by: Alantha Newman.
Edge-disjoint induced subgraphs with given minimum degree Raphael Yuster 2012.
Yuan Zhou Carnegie Mellon University Joint works with Boaz Barak, Fernando G.S.L. Brandão, Aram W. Harrow, Jonathan Kelner, Ryan O'Donnell and David Steurer.
CSM Workshop 1: Zeros of Graph Polynomials Enumeration of Spanning Subgraphs with Degree Constraints Dave Wagner University of Waterloo.
Semidefinite Programming
1 Asymptotically optimal K k -packings of dense graphs via fractional K k -decompositions Raphael Yuster University of Haifa.
1/19 Minimizing weighted completion time with precedence constraints Nikhil Bansal (IBM) Subhash Khot (NYU)
Linear Program Set Cover. Given a universe U of n elements, a collection of subsets of U, S = {S 1,…, S k }, and a cost function c: S → Q +. Find a minimum.
1.1 Chapter 3: Proving NP-completeness Results Six Basic NP-Complete Problems Some Techniques for Proving NP-Completeness Some Suggested Exercises.
Flipping letters to minimize the support of a string Giuseppe Lancia, Franca Rinaldi, Romeo Rizzi University of Udine.
Shorter Long Codes and Applications to Unique Games 1 Boaz Barak (MSR, New England) Parikshit Gopalan (MSR, SVC) Johan Håstad (KTH) Prasad Raghavendra.
Unique Games Approximation Amit Weinstein Complexity Seminar, Fall 2006 Based on: “Near Optimal Algorithms for Unique Games" by M. Charikar, K. Makarychev,
Dense graphs with a large triangle cover have a large triangle packing Raphael Yuster SIAM DM’10.
Yuan Zhou, Ryan O’Donnell Carnegie Mellon University.
Iterative Rounding in Graph Connectivity Problems Kamal Jain ex- Georgia Techie Microsoft Research Some slides borrowed from Lap Chi Lau.
Boaz Barak (MSR New England) Fernando G.S.L. Brandão (Universidade Federal de Minas Gerais) Aram W. Harrow (University of Washington) Jonathan Kelner (MIT)
C&O 355 Lecture 19 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A A A A A A A.
TU/e Algorithms (2IL15) – Lecture 12 1 Linear Programming.
Non-LP-Based Approximation Algorithms Fabrizio Grandoni IDSIA
Yuan Zhou Carnegie Mellon University Joint works with Boaz Barak, Fernando G.S.L. Brandão, Aram W. Harrow, Jonathan Kelner, Ryan O'Donnell and David Steurer.
TU/e Algorithms (2IL15) – Lecture 10 1 NP-Completeness, II.
Theory of Computational Complexity Probability and Computing Chapter Hikaru Inada Iwama and Ito lab M1.
P & NP.
Planar Graphs Hubert Chan (Chapter 9.7) [O2 Proof Techniques]
Lap Chi Lau we will only use slides 4 to 19
Polynomial integrality gaps for
Topics in Algorithms Lap Chi Lau.
Sum of Squares Lower Bounds for Refuting Any CSP
Polynomial Norms Amir Ali Ahmadi (Princeton University) Georgina Hall
Sum of Squares, Planted Clique, and Pseudo-Calibration
Log-Sobolev Inequality on the Multislice (and what those words mean)
Joint work with Avishay Tal (IAS) and Jiapeng Zhang (UCSD)
NP-Completeness Yin Tat Lee
Structural Properties of Low Threshold Rank Graphs
SOS is not obviously automatizable,
3.5 Minimum Cuts in Undirected Graphs
Miniconference on the Mathematics of Computation
NP-Completeness Yin Tat Lee
Instructor: Aaron Roth
Instructor: Aaron Roth
Integer and fractional packing of graph families
Presentation transcript:

Hypercontractivity & Sums of Squares Ryan O’Donnell Carnegie Mellon

Motivation: Optimization Max-Independent-Set Min-Vertex-Cover Colouring Sparsest-Cut Balanced-Separator Max-Cut Unique-Games Small-Set-Expansion

Max-IS(G) = max {|A| : A ⊆ V, no edge internal to A} Max Independent-Set: LP relaxation Max-IS(G) = max {|A| : A ⊆ V, no edge internal to A} For each v ∈ V, have variable… [0,1] f(v) ∈ {0,1} Independent-Set constraints… f(v)+f(w) ≤ 1 ∀ (v,w)∈E LP-Opt(G) = max

“I certify the max. independent set is ≤ LP-Opt(G)” Value Certification Max Independent-Set: “I certify the max. independent set is ≤ LP-Opt(G)” LP G

I certify the max. independent set is ≤ Value Certification Max Independent-Set: N-vertex complete graph I certify the max. independent set is ≤ N/2 LP KN Truth: Max-IS(KN) = 1

Value Certification Max Independent-Set: Recall: Max-IS(G) = N − Min-Vertex-Cover(G) N-vertex complete graph I certify the max. independent set is ≤ N/2 LP KN Truth: Max-IS(KN) = 1

Value Certification Min Vertex-Cover: N-vertex complete graph I certify the min. vertex cover set is ≥ N/2 LP KN Truth: Min-VC(KN) = N−1 This is the famous factor-2 integrality gap for Vertex-Cover.

I certify the max. independent set is ≤ Value Certification Max Independent-Set: N-vertex complete graph I certify the max. independent set is ≤ 1 KN SDP LP Truth: Max-IS(KN) = 1

I certify the max. independent set is ≤ Value Certification Max Independent-Set: I certify the max. independent set is ≤ Frankl–Rödl graph N/3 FR ¼ N SDP [KG’95] Given a general gamma, the SDP thinks the indep set is of size (1-2gamma)/(2-2gamma) * N. Truth: Max-IS(FR ) ≤ N.99 [FR’87] FR ¼ N

Value Certification Max Independent-Set: ¼ N I certify the max. independent set is ≤ Frankl–Rödl graph N/3 FR ¼ N SDP [KG’95] Shoot, it might even be 3-colourable…

Frankl–Rödl Graph ¼ N FR defined assuming N = 2n (and ¼ n is even) V = {0,1}n E = { (v,w) : Δ(v,w) = ¾ n } {0,1}n

Frankl–Rödl Graph γ N FR defined assuming N = 2n (and γ n is even) V = {0,1}n E = { (v,w) : Δ(v,w) = (1−γ) n } Neighbours of v: all ways of flipping a γ-fraction of coords 0 < γ < ½ {0,1}n

I certify the max. independent set is ≤ Value Certification Max Independent-Set: I certify the max. independent set is ≤ Frankl–Rödl graph N/3 FR ¼ N SDP [KG’95] Given a general gamma, the SDP thinks the indep set is of size (1-2gamma)/(2-2gamma) * N. Truth: Max-IS(FR ) ≤ N.99 [FR’87] FR ¼ N

I certify the max. independent set is ≤ Value Certification Max Independent-Set: I certify the max. independent set is ≤ Frankl–Rödl graph FR γ N SDP [KG’95] Truth: Max-IS(FR ) ≤ o(N) FR γ N (provided γ ≥ )

I certify the max. independent set is ≤ Value Certification Max Independent-Set: I certify the max. independent set is ≤ Frankl–Rödl graph Lovász– Schrijver LP FR γ N [ABL’02] Truth: Max-IS(FR ) ≤ o(N) FR γ N (provided γ ≥ )

I certify the max. independent set is ≤ Value Certification Max Independent-Set: I certify the max. independent set is ≤ Frankl–Rödl graph Lovász– Schrijver SDP FR γ N [GMPT’07] Truth: Max-IS(FR ) ≤ o(N) FR γ N (provided γ ≥ )

I certify the max. independent set is ≤ Value Certification Max Independent-Set: I certify the max. independent set is ≤ Frankl–Rödl graph Sherali– Adams SDP FR γ N [BCGM’11] Truth: Max-IS(FR ) ≤ o(N) FR γ N (provided γ ≥ )

Value Certification Max Independent-Set: γ N γ N Frankl–Rödl graph How did they certify it?! FR γ N Truth: Max-IS( ) ≤ o(N) FR γ N [FR’87]

How Frankl & Rödl proved it It wasn’t particularly easy. Erdős gave them $250 for it. The central lemma at the very core of the proof was Harper’s Hypercube Vertex-Isoperimetry Theorem.

Hypercube Vertex-Isoperimetry Let A ⊆ {0,1}n be not too small, say size ≥ N.99. Then {w : Δ(w,A) ≤ ¼ n} is almost all of {0,1}n. [Harper’66] gave the precise statement: For any γ in place of ¼, and any fixed size for A, the “worst” A is a Hamming ball.

Hypercube Vertex-Isoperimetry Harper’s proof uses “combinatorial shifting”. So arguably we can say that, as proof systems, LP SDP Lovász–Schrijver LP Lovász–Schrijver SDP Sherali–Adams SDP cannot “do” combinatorial shifting.

Balanced-Separator, Unique-Games, Sparsest-Cut, Small-Set Expansion… I think there’s a small set where most edges touching it are internal. Khot–Vishnoi graph SDP SDP+Δ Sherali– Adams… KV γ N [KV’05] [DKSV’06] [KS’09] [RS’09] … Truth: ∀ small sets, almost all edges touching it are crossing.

Hypercube Small-Set Expansion Core theorem: Hypercube Small-Set Expansion [KKL’88] Let A ⊆ {0,1}n be not too large. Pick v∈A at random, then pick w by flipping each coordinate with probability γ. Then almost surely, w∉A. (Asymptotically, Hamming balls are worst.) Proof technique: Hypercontractive Inequality.

Hypercube SSE: 2 versions V = {0,1}n, E = {(v,w) : Δ(v,w) ≈ γn}, 0 < γ < ½ I. Almost all edges escape II. Practically everywhere sees at least one edge A not too big ⇒ almost all touching edges go out A not too small ⇒ ∃ edge to practically everywhere A A

Hypercube SSE: 2 versions V = {0,1}n, E = {(v,w) : Δ(v,w) ≈ γn}, 0 < γ < ½ I. Almost all edges escape II. Practically everywhere sees at least one edge A not too big ⇒ almost all touching edges go out A not too small ⇒ ∃ edge to practically everywhere Proof: Hypercontractive Inequality Proof: Combinatorial shifting

Hypercube SSE: 2 versions V = {0,1}n, E = {(v,w) : Δ(v,w) ≈ γn}, 0 < γ < ½ I. Almost all edges escape II′. Most places get at least a few edges A not too big ⇒ almost all touching edges go out A not too small ⇒ random edges will go almost everywhere Proof: Hypercontractive Inequality Proof: Reverse Hyper- contractive Inequality [MORSS’05], [BHM’12] for density Frankl–Rödl

Hypercube SSE: 2 versions V = {0,1}n, E = {(v,w) : Δ(v,w) ≈ γn}, 0 < γ < ½ I. Almost all edges escape II′. Most places get at least a few edges A A Two core principles for “SDP integrality gaps”. LP, SDP, LS+, SA+, etc. cannot prove them.

What about SOS? SOS (“Sum of Squares”) is another SDP hierarchy / proof system. The “constant round/degree” version is poly-time. Known that SOS ≫ LP, SDP, LS+, SA+ [LRS’15]: Indeed, for “Max-CSPs”, SOS ≫ any poly-size SDP relaxation. Can SOS “do” the two Hypercube SSE principles?

What is the SOS proof system? It’s not hard to describe, but I won’t do so fully, for lack of time.

Polynomial formulation Max Indep.-Set: Polynomial formulation For each v ∈ V, have variable… f(v)2 = f(v) f(v) ∈ {0,1} Independent-Set constraints… f(v)f(w) = 0 ∀ (v,w)∈E ≥ B ?

Replace a polynomial inequality condition like The SOS idea Replace a polynomial inequality condition like “P(f1, …, fN) ≥ 0” (modulo constraints) with “P(f1, …, fN) is a sum of squares of polynomials of constant degree” (modulo constraints)

SOS and Hypercontractivity Can the Hypercontractive Inequality and the Reverse Hypercontractive Inequality be given a polynomial formulation, and then proved by the SOS method? Answers: Yes [BBHKSZ’12], [OZ’13], [KOTZ’14] and Yes [KOTZ’14], though the “degree” depends inversely on the γ parameter.

SOS and Hypercontractivity Corollaries: SOS gives satisfactory certifications for the Khot–Vishnoi-based hard instances of Unique-Games [BBHKSZ’12] and Balanced-Separator [OZ’13]. SOS gives somewhat satisfactory certifications for the Frankl–Rödl-based hard instances of Independent-Set & Vertex-Cover [KOTZ’14].

A little bit about the SOS formulation and proof for Reverse Hypercontractivity

Hypercube SSE: 2 versions V = {0,1}n, E = {(v,w) : Δ(v,w) ≈ γn}, 0 < γ < ½ I. Almost all edges escape II′. Most places get at least a few edges A not too big ⇒ almost all touching edges go out A not too small ⇒ random edges will go almost everywhere Proof: Hypercontractive Inequality Proof: Reverse Hyper- contractive Inequality [MORSS’05], [BHM’12] for density Frankl–Rödl

Hypercube SSE: 2 versions V = {0,1}n, E = {(v,w) : Δ(v,w) ≈ γn}, 0 < γ < ½ II′. Most places get at least a few edges A not too small ⇒ random edges will go almost everywhere A Proof: Reverse Hyper- contractive Inequality [MORSS’05], [BHM’12] for density Frankl–Rödl

Proof by generalization. Theorem: cor. of [MORSS’05] Let A, B ⊆ {0,1}n have “volume” α each. (I.e., |A|/2n = |B|/2n = α.) Pick v ~ {0,1}n uniformly, then flip each coord. with probability ¼ to form w∈{0,1}n. Then Pr [v∈A and w∈B] ≥ α4 I.e., Pr [w∈B | v∈A] ≥ α3 How to SOS-prove this? How to prove this at all? Proof by generalization.

1/γ Generalization: [MORSS’05] Let A, B ⊆ {0,1}n have “volume” α each. (I.e., |A|/2n = |B|/2n = α.) Pick v ~ {0,1}n uniformly, then flip each coord. with probability ¼ to form w∈{0,1}n. Then Pr [v∈A and w∈B] ≥ α4 γ 1/γ

Further generalization: [MORSS’05] Let A, B ⊆ {0,1}n have “volume” α respectively. Pick v ~ {0,1}n uniformly, then flip each coord. with probability ¼ to form w∈{0,1}n. Then Pr [v∈A and w∈B] ≥ 4 where p(α,β,γ) is a nice explicit expression. , β γ p(α,β,γ)

The “(2-function) Reverse Hypercontractive Inequality” Further further generalization: [B’82,MORSS’05] Let f, g : {0,1}n → ℝ≥0. Let r, s ≥ 0 satisfy . Pick v ~ {0,1}n uniformly, then flip each coord. with probability ¼ to form w∈{0,1}n. Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s γ The “(2-function) Reverse Hypercontractive Inequality” Implies previous slide by taking f = 1A, g = 1B (and choosing r, s appropriately).

But only because we have a 2-function version! Further further generalization: [B’82,MORSS’05] Let f, g : {0,1}n → ℝ≥0. Let r, s ≥ 0 satisfy . Pick v ~ {0,1}n uniformly, then flip each coord. with probability ¼ to form w∈{0,1}n. Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s γ Given the n = 1 case, the general n case follows by an utterly trivial induction. But only because we have a 2-function version!

Equivalent n=1 specialization: [B’82,MORSS’05] Let f, g : {0,1} → ℝ≥0. Let r, s ≥ 0 satisfy . Pick v ~ {0,1}n uniformly, then flip it coord. with probability ¼ to form w∈{0,1} Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s γ Fixing the “parameters” γ, r, s, this is just an inequality about 4 real numbers, namely f(0), f(1), g(0), g(1).

Final statement: [B’82,MORSS’05] Let f, g : {0,1} → ℝ≥0. Let r, s ≥ 0 satisfy . Pick v ~ {0,1}n uniformly, then flip it coord. with probability ¼ to form w∈{0,1} Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s γ Still not easy! Reverse Hölder inequality and homogeneity tricks reduce to an inequality about 1 real number. Then proved by Taylor series.

Final statement: [B’82,MORSS’05] Let f, g : {0,1} → ℝ≥0. Let r, s ≥ 0 satisfy . Pick v ~ {0,1}n uniformly, then flip it coord. with probability ¼ to form w∈{0,1} Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s γ Now how do we prove this using SOS?! Not even a polynomial statement, due to non-integer norms.

Final statement: [B’82,MORSS’05] Let f, g : {0,1} → ℝ≥0. Let r, s ≥ 0 satisfy . Pick v ~ {0,1}n uniformly, then flip it coord. with probability ¼ to form w∈{0,1} Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s γ For the Independent-Set application, it’s enough to consider A, B of equal volume α. Implies it’s enough to consider r = s = 1−2γ.

Replace f, g with f1/(2γ), g1/(2γ). Final statement: [B’82,MORSS’05] Let f, g : {0,1} → ℝ≥0. Let r, s ≥ 0 satisfy . Pick v ~ {0,1}n uniformly, then flip it coord. with probability ¼ to form w∈{0,1} Then E [f(v) g(w)] ≥ ||f||2γ ||g||2γ γ Replace f, g with f1/(2γ), g1/(2γ).

Final statement: [B’82,MORSS’05] Let f, g : {0,1} → ℝ≥0. Let r, s ≥ 0 satisfy . Pick v ~ {0,1}n uniformly, then flip it coord. with probability ¼ to form w∈{0,1} Then E[f(v)1/(2γ)g(w)1/(2γ)] ≥ E[f]1/(2γ)E[g]1/(2γ) γ Assume 1/(2γ) = 2k for some integer k. E.g., γ = ¼ corresponds to k = 1.

By homogeneity, reduces to ineq. on 2 variables. Final statement: [B’82,MORSS’05] Let f, g : {0,1} → ℝ Let k∈ℕ. Let r, s ≥ 0 satisfy . Pick v ~ {0,1}n uniformly, then flip it coor with probability ¼ to form w∈{0,1} Then E[f(v)2k g(w)2k] ≥ E[f]2k E[g]2k 1/(4k) For each fixed k, this is a polynomial inequality in 4 real variables: f(0), f(1), g(0), g(1). By homogeneity, reduces to ineq. on 2 variables.

We implicitly know this statement is true. Final final statement: [B’82,MORSS’05] Let k∈ℕ. Then ≥ 0. We implicitly know this statement is true. What we want to show is that that polynomial is a sum of squares of polynomials in a and b.

For k = 1, the polynomial is equal to… Final final statement: [B’82,MORSS’05] Let k∈ℕ. Then ≥ 0. For k = 1, the polynomial is equal to… (ab)2 + (a+b)2

For k = 2, the polynomial is equal to… Final final statement: [B’82,MORSS’05] Let k∈ℕ. Then ≥ 0. For k = 2, the polynomial is equal to…

For k = 3, the polynomial is equal to… Final final statement: [B’82,MORSS’05] Let k∈ℕ. Then ≥ 0. For k = 3, the polynomial is equal to…

Final final statement: [B’82,MORSS’05] Let k∈ℕ. Then ≥ 0. So we were pretty sure it was SOS for all k. But we could find no pattern.

univariate polynomials are all SOS. One month later… We reduced it to showing, for each k, that a certain infinite sequence of ugly univariate polynomials are all SOS. Fact: A univariate polynomial is SOS iff it is nonnegative. So we just had to show all those univariate polynomials are nonnegative.

We showed all those univariate polynomials are nonnegative. Another month later… We showed all those univariate polynomials are nonnegative. More accurately, using “automated combinatorial identity proving” tricks, we coaxed our computer to assist us in producing incomprehensible proofs of nonnegativity.

We showed all those univariate polynomials are nonnegative. Another month later… We showed all those univariate polynomials are nonnegative.

Can the Hypercontractive Inequality and the Summary Can the Hypercontractive Inequality and the Reverse Hypercontractive Inequality be given a polynomial formulation, and then proved by the SOS method? Answers: Yes [BBHKSZ’12], [OZ’13], [KOTZ’14] and Yes [KOTZ’14], though the “degree” depends inversely on the γ parameter.

Summary Corollaries: SOS gives satisfactory certifications for the Khot–Vishnoi-based hard instances of Unique-Games [BBHKSZ’12] and Balanced-Separator [OZ’13]. SOS gives somewhat satisfactory certifications for the Frankl–Rödl-based hard instances of Independent-Set & Vertex-Cover [KOTZ’14].

Open Problems 1 & 2 Each corollary would be much more satisfactory if we could SOS-prove a certain simple Hypercube SSE statement. For each of the two desired statements, I can tell you an extremely extremely simple “analytic” proof. (Ask me after the talk.) Prove or disprove for either statement: There is a degree-4 SOS proof.

Open Problem 3 Give a human-comprehensible SOS-proof of that family of 2-variable polynomial inequalities!

Thank you! 谢谢 ! Terima kasih! நன்றி!