1. Hypercontractivity & Sums of Squares
Ryan O’Donnell
Carnegie Mellon

2. Motivation: Optimization
Max-Independent-Set
Min-Vertex-Cover
Colouring
Sparsest-Cut
Balanced-Separator
Max-Cut
Unique-Games
Small-Set-Expansion

3. Max-IS(G) = max {|A| : A ⊆ V, no edge internal to A}
Max Independent-Set:
LP relaxation
Max-IS(G) = max {|A| : A ⊆ V, no edge internal to A}
For each v ∈ V,
have variable…
[0,1]
f(v) ∈ {0,1}
Independent-Set
constraints…
f(v)+f(w) ≤ 1 ∀ (v,w)∈E
LP-Opt(G) =
max

4. “I certify the max. independent set is ≤ LP-Opt(G)”
Value Certification
Max Independent-Set:
“I certify the max. independent set is ≤ LP-Opt(G)” LP G

5. I certify the max. independent set is ≤
Value Certification
Max Independent-Set:
N-vertex complete
graph
I certify the max. independent set is ≤
N/2 LP KN
Truth:
Max-IS(KN) = 1

6. Value Certification Max Independent-Set:
Recall: Max-IS(G) = N − Min-Vertex-Cover(G)
N-vertex complete
graph
I certify the max. independent set is ≤
N/2 LP KN
Truth:
Max-IS(KN) = 1

7. Value Certification Min Vertex-Cover: N-vertex complete
graph
I certify the min. vertex cover set is ≥
N/2 LP KN
Truth:
Min-VC(KN) = N−1
This is the famous factor-2 integrality gap for Vertex-Cover.

8. I certify the max. independent set is ≤
Value Certification
Max Independent-Set:
N-vertex complete
graph
I certify the max. independent set is ≤ 1 KN
SDP LP
Truth:
Max-IS(KN) = 1

9. I certify the max. independent set is ≤
Value Certification
Max Independent-Set:
I certify the max. independent set is ≤
Frankl–Rödl graph
N/3 FR N
SDP
[KG’95]
Given a general gamma, the SDP thinks the indep set is of size (1-2gamma)/(2-2gamma) * N.
Truth:
Max-IS(FR ) ≤ N.99
[FR’87] FR N

10. Value Certification Max Independent-Set: ¼ N
I certify the max. independent set is ≤
Frankl–Rödl graph
N/3 FR N
SDP
[KG’95]
Shoot, it might even be 3-colourable…

11. Frankl–Rödl Graph ¼ N FR defined assuming N = 2n (and ¼ n is even)
V = {0,1}n
E = { (v,w) : Δ(v,w) = ¾ n }
{0,1}n

12. Frankl–Rödl Graph γ N FR defined assuming N = 2n (and γ n is even)
V = {0,1}n
E = { (v,w) : Δ(v,w) = (1−γ) n }
Neighbours of v:
all ways of flipping a γ-fraction
of coords
0 < γ < ½
{0,1}n

13. I certify the max. independent set is ≤
Value Certification
Max Independent-Set:
I certify the max. independent set is ≤
Frankl–Rödl graph
N/3 FR N
SDP
[KG’95]
Given a general gamma, the SDP thinks the indep set is of size (1-2gamma)/(2-2gamma) * N.
Truth:
Max-IS(FR ) ≤ N.99
[FR’87] FR N

14. I certify the max. independent set is ≤
Value Certification
Max Independent-Set:
I certify the max. independent set is ≤
Frankl–Rödl graph FR γ N
SDP
[KG’95]
Truth:
Max-IS(FR ) ≤ o(N) FR γ N
(provided γ ≥ )

15. I certify the max. independent set is ≤
Value Certification
Max Independent-Set:
I certify the max. independent set is ≤
Frankl–Rödl graph
Lovász– Schrijver LP FR γ N
[ABL’02]
Truth:
Max-IS(FR ) ≤ o(N) FR γ N
(provided γ ≥ )

16. I certify the max. independent set is ≤
Value Certification
Max Independent-Set:
I certify the max. independent set is ≤
Frankl–Rödl graph
Lovász– Schrijver
SDP FR γ N
[GMPT’07]
Truth:
Max-IS(FR ) ≤ o(N) FR γ N
(provided γ ≥ )

17. I certify the max. independent set is ≤
Value Certification
Max Independent-Set:
I certify the max. independent set is ≤
Frankl–Rödl graph
Sherali–
Adams
SDP FR γ N
[BCGM’11]
Truth:
Max-IS(FR ) ≤ o(N) FR γ N
(provided γ ≥ )

18. Value Certification Max Independent-Set: γ N γ N Frankl–Rödl graph
How did they certify it?! FR γ N
Truth:
Max-IS( ) ≤ o(N) FR γ N
[FR’87]

19. How Frankl & Rödl proved it
It wasn’t particularly easy.
Erdős gave them $250 for it.
The central lemma at the very core of the proof was Harper’s Hypercube Vertex-Isoperimetry Theorem.

20. Hypercube Vertex-Isoperimetry
Let A ⊆ {0,1}n be not too small, say size ≥ N.99.
Then {w : Δ(w,A) ≤ ¼ n} is almost all of {0,1}n.
[Harper’66] gave the precise statement:
For any γ in place of ¼, and any fixed size for A,
the “worst” A is a Hamming ball.

21. Hypercube Vertex-Isoperimetry
Harper’s proof uses “combinatorial shifting”.
So arguably we can say that, as proof systems, LP
SDP
Lovász–Schrijver LP
Lovász–Schrijver SDP
Sherali–Adams SDP
cannot “do” combinatorial shifting.

22. Balanced-Separator, Unique-Games, Sparsest-Cut, Small-Set Expansion…
I think there’s a small set where most edges touching it are internal.
Khot–Vishnoi graph
SDP
SDP+Δ
Sherali–
Adams… KV γ N
[KV’05]
[DKSV’06]
[KS’09]
[RS’09] …
Truth: ∀ small sets, almost all edges touching it are crossing.

23. Hypercube Small-Set Expansion
Core theorem:
Hypercube Small-Set Expansion
[KKL’88]
Let A ⊆ {0,1}n be not too large.
Pick v∈A at random, then pick w by flipping
each coordinate with probability γ.
Then almost surely, w∉A.
(Asymptotically, Hamming balls are worst.)
Proof technique: Hypercontractive Inequality.

24. Hypercube SSE: 2 versions
V = {0,1}n, E = {(v,w) : Δ(v,w) ≈ γn}, 0 < γ < ½
I. Almost all edges escape
II. Practically everywhere
sees at least one edge
A not too big ⇒ almost all touching edges go out
A not too small ⇒ ∃ edge to practically everywhere A A

25. Hypercube SSE: 2 versions
V = {0,1}n, E = {(v,w) : Δ(v,w) ≈ γn}, 0 < γ < ½
I. Almost all edges escape
II. Practically everywhere
sees at least one edge
A not too big ⇒ almost all touching edges go out
A not too small ⇒ ∃ edge to practically everywhere
Proof:
Hypercontractive Inequality
Proof:
Combinatorial shifting

26. Hypercube SSE: 2 versions
V = {0,1}n, E = {(v,w) : Δ(v,w) ≈ γn}, 0 < γ < ½
I. Almost all edges escape
II′. Most places get at
least a few edges
A not too big ⇒ almost all touching edges go out
A not too small ⇒ random edges will go almost everywhere
Proof:
Hypercontractive
Inequality
Proof: Reverse Hyper- contractive Inequality
[MORSS’05], [BHM’12] for density Frankl–Rödl

27. Hypercube SSE: 2 versions
V = {0,1}n, E = {(v,w) : Δ(v,w) ≈ γn}, 0 < γ < ½
I. Almost all edges escape
II′. Most places get at
least a few edges A A
Two core principles for “SDP integrality gaps”. LP, SDP, LS+, SA+, etc. cannot prove them.

28. What about SOS?
SOS (“Sum of Squares”) is another SDP hierarchy / proof system.
The “constant round/degree” version is poly-time.
Known that SOS ≫ LP, SDP, LS+, SA+
[LRS’15]: Indeed, for “Max-CSPs”, SOS ≫ any poly-size SDP relaxation.
Can SOS “do” the two Hypercube SSE principles?

29. What is the SOS proof system?
It’s not hard to describe, but I won’t do so fully, for lack of time.

30. Polynomial formulation
Max Indep.-Set:
Polynomial formulation
For each v ∈ V,
have variable…
f(v)2 = f(v)
f(v) ∈ {0,1}
Independent-Set
constraints…
f(v)f(w) = 0 ∀ (v,w)∈E
≥ B ?

31. Replace a polynomial inequality condition like
The SOS idea
Replace a polynomial inequality condition like
“P(f1, …, fN) ≥ 0” (modulo constraints)
with
“P(f1, …, fN) is a sum of squares of polynomials of constant degree” (modulo constraints)

32. SOS and Hypercontractivity
Can the Hypercontractive Inequality and the
Reverse Hypercontractive Inequality be given a polynomial formulation, and then proved by the SOS method?
Answers: Yes [BBHKSZ’12], [OZ’13], [KOTZ’14]
and Yes [KOTZ’14], though the “degree” depends inversely on the γ parameter.

33. SOS and Hypercontractivity
Corollaries:
SOS gives satisfactory certifications for the Khot–Vishnoi-based hard instances
of Unique-Games [BBHKSZ’12] and Balanced-Separator [OZ’13].
SOS gives somewhat satisfactory certifications for the Frankl–Rödl-based hard instances of Independent-Set & Vertex-Cover [KOTZ’14].

34. A little bit about the SOS formulation and proof for Reverse Hypercontractivity

35. Hypercube SSE: 2 versions
V = {0,1}n, E = {(v,w) : Δ(v,w) ≈ γn}, 0 < γ < ½
I. Almost all edges escape
II′. Most places get at
least a few edges
A not too big ⇒ almost all touching edges go out
A not too small ⇒ random edges will go almost everywhere
Proof:
Hypercontractive
Inequality
Proof: Reverse Hyper- contractive Inequality
[MORSS’05], [BHM’12] for density Frankl–Rödl

36. Hypercube SSE: 2 versions
V = {0,1}n, E = {(v,w) : Δ(v,w) ≈ γn}, 0 < γ < ½
II′. Most places get at
least a few edges
A not too small ⇒ random edges will go almost everywhere A
Proof: Reverse Hyper- contractive Inequality
[MORSS’05], [BHM’12] for density Frankl–Rödl

37. Proof by generalization.
Theorem: cor. of [MORSS’05]
Let A, B ⊆ {0,1}n have “volume” α each.
(I.e., |A|/2n = |B|/2n = α.)
Pick v ~ {0,1}n uniformly, then flip each coord. with probability ¼ to form w∈{0,1}n.
Then Pr [v∈A and w∈B] ≥ α4
I.e., Pr [w∈B | v∈A] ≥ α3
How to SOS-prove this?
How to prove this at all?
Proof by generalization.

38. 1/γ Generalization: [MORSS’05] Let A, B ⊆ {0,1}n have “volume” α each.
(I.e., |A|/2n = |B|/2n = α.)
Pick v ~ {0,1}n uniformly, then flip each coord. with probability ¼ to form w∈{0,1}n.
Then Pr [v∈A and w∈B] ≥ α4 γ
1/γ

39. Further generalization: [MORSS’05]
Let A, B ⊆ {0,1}n have “volume” α
respectively.
Pick v ~ {0,1}n uniformly, then flip each coord. with probability ¼ to form w∈{0,1}n.
Then Pr [v∈A and w∈B] ≥ 4
where p(α,β,γ) is a nice explicit expression.
, β γ
p(α,β,γ)

40. The “(2-function) Reverse Hypercontractive Inequality”
Further further generalization: [B’82,MORSS’05]
Let f, g : {0,1}n → ℝ≥ Let r, s ≥ 0 satisfy
Pick v ~ {0,1}n uniformly, then flip each coord. with probability ¼ to form w∈{0,1}n.
Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s γ
The “(2-function) Reverse Hypercontractive Inequality”
Implies previous slide by taking f = 1A, g = 1B (and choosing r, s appropriately).

41. But only because we have a 2-function version!
Further further generalization: [B’82,MORSS’05]
Let f, g : {0,1}n → ℝ≥ Let r, s ≥ 0 satisfy
Pick v ~ {0,1}n uniformly, then flip each coord. with probability ¼ to form w∈{0,1}n.
Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s γ
Given the n = 1 case, the general n case follows by an utterly trivial induction.
But only because we have a 2-function version!

42. Equivalent n=1 specialization: [B’82,MORSS’05]
Let f, g : {0,1} → ℝ≥ Let r, s ≥ 0 satisfy
Pick v ~ {0,1}n uniformly, then flip it coord. with probability ¼ to form w∈{0,1}
Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s γ
Fixing the “parameters” γ, r, s, this is just an inequality about 4 real numbers, namely f(0), f(1), g(0), g(1).

43. Final statement: [B’82,MORSS’05]
Let f, g : {0,1} → ℝ≥ Let r, s ≥ 0 satisfy
Pick v ~ {0,1}n uniformly, then flip it coord. with probability ¼ to form w∈{0,1}
Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s γ
Still not easy! Reverse Hölder inequality and homogeneity tricks reduce to an inequality about 1 real number. Then proved by Taylor series.

44. Final statement: [B’82,MORSS’05]
Let f, g : {0,1} → ℝ≥ Let r, s ≥ 0 satisfy
Pick v ~ {0,1}n uniformly, then flip it coord. with probability ¼ to form w∈{0,1}
Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s γ
Now how do we prove this using SOS?!
Not even a polynomial statement, due to non-integer norms.

45. Final statement: [B’82,MORSS’05]
Let f, g : {0,1} → ℝ≥ Let r, s ≥ 0 satisfy
Pick v ~ {0,1}n uniformly, then flip it coord. with probability ¼ to form w∈{0,1}
Then E [f(v) g(w)] ≥ ||f||1−r ||g||1−s γ
For the Independent-Set application, it’s enough to consider A, B of equal volume α. Implies it’s enough to consider r = s = 1−2γ.

46. Replace f, g with f1/(2γ), g1/(2γ).
Final statement: [B’82,MORSS’05]
Let f, g : {0,1} → ℝ≥ Let r, s ≥ 0 satisfy
Pick v ~ {0,1}n uniformly, then flip it coord. with probability ¼ to form w∈{0,1}
Then E [f(v) g(w)] ≥ ||f||2γ ||g||2γ γ
Replace f, g with f1/(2γ), g1/(2γ).

47. Final statement: [B’82,MORSS’05]
Let f, g : {0,1} → ℝ≥ Let r, s ≥ 0 satisfy
Pick v ~ {0,1}n uniformly, then flip it coord. with probability ¼ to form w∈{0,1}
Then E[f(v)1/(2γ)g(w)1/(2γ)] ≥ E[f]1/(2γ)E[g]1/(2γ) γ
Assume 1/(2γ) = 2k for some integer k.
E.g., γ = ¼ corresponds to k = 1.

48. By homogeneity, reduces to ineq. on 2 variables.
Final statement: [B’82,MORSS’05]
Let f, g : {0,1} → ℝ
Let k∈ℕ. Let r, s ≥ 0 satisfy
Pick v ~ {0,1}n uniformly, then flip it coor with probability ¼ to form w∈{0,1}
Then E[f(v)2k g(w)2k] ≥ E[f]2k E[g]2k
1/(4k)
For each fixed k, this is a polynomial inequality in 4 real variables: f(0), f(1), g(0), g(1).
By homogeneity, reduces to ineq. on 2 variables.

49. We implicitly know this statement is true.
Final final statement: [B’82,MORSS’05]
Let k∈ℕ. Then
≥ 0.
We implicitly know this statement is true.
What we want to show is that that polynomial is a sum of squares of polynomials in a and b.

50. For k = 1, the polynomial is equal to…
Final final statement: [B’82,MORSS’05]
Let k∈ℕ. Then
≥ 0.
For k = 1, the polynomial is equal to…
(ab)2 + (a+b)2

51. For k = 2, the polynomial is equal to…
Final final statement: [B’82,MORSS’05]
Let k∈ℕ. Then
≥ 0.
For k = 2, the polynomial is equal to…

52. For k = 3, the polynomial is equal to…
Final final statement: [B’82,MORSS’05]
Let k∈ℕ. Then
≥ 0.
For k = 3, the polynomial is equal to…

53. Final final statement: [B’82,MORSS’05] Let k∈ℕ. Then
≥ 0.
So we were pretty sure it was SOS for all k.
But we could find no pattern.

54. univariate polynomials are all SOS.
One month later…
We reduced it to showing, for each k, that a certain infinite sequence of ugly
univariate polynomials are all SOS.
Fact: A univariate polynomial is SOS
iff it is nonnegative.
So we just had to show all those univariate polynomials are nonnegative.

55. We showed all those univariate polynomials are nonnegative.
Another month later…
We showed all those univariate polynomials are nonnegative.
More accurately, using “automated combinatorial identity proving” tricks, we coaxed our computer to
assist us in producing incomprehensible proofs of nonnegativity.

56. We showed all those univariate polynomials are nonnegative.
Another month later…
We showed all those univariate polynomials are nonnegative.

57. Can the Hypercontractive Inequality and the
Summary
Can the Hypercontractive Inequality and the
Reverse Hypercontractive Inequality be given a polynomial formulation, and then proved by the SOS method?
Answers: Yes [BBHKSZ’12], [OZ’13], [KOTZ’14]
and Yes [KOTZ’14], though the “degree” depends inversely on the γ parameter.

58. Summary
Corollaries:
SOS gives satisfactory certifications for the Khot–Vishnoi-based hard instances
of Unique-Games [BBHKSZ’12] and Balanced-Separator [OZ’13].
SOS gives somewhat satisfactory certifications for the Frankl–Rödl-based hard instances of Independent-Set & Vertex-Cover [KOTZ’14].

59. Open Problems 1 & 2
Each corollary would be much more satisfactory if we could SOS-prove a certain
simple Hypercube SSE statement.
For each of the two desired statements, I can tell you an extremely extremely simple “analytic” proof. (Ask me after the talk.)
Prove or disprove for either statement: There is a degree-4 SOS proof.

60. Open Problem 3
Give a human-comprehensible SOS-proof of that family of 2-variable polynomial inequalities!

61. Thank you!
谢谢 !
Terima kasih!
நன்றி!