1. Sum of Squares Lower Bounds for Refuting Any CSP
Ryan O’Donnell
Carnegie Mellon
joint work with Pravesh Kothari, Ryuhei Mori, & David Witmer

2. Let’s say you’re a cruel professor.
And you need to make some
hard homework problems, ASAP.

3. I’ll just put some random 3SAT problems on there.

5. I ⋮ 3SAT instance I x1 ∨ x2 ∨ x5 Δn variables x1, …, xn
to be assigned vals in {0,1}
Δn constraints;
each an OR of 3 literals
Random 3SAT:
all constraints chosen uniformly, independently

6. But what Δ to choose?

7. Δ But what Δ to choose? I is satisfiable (w.v.h.p.)
I is unsatisfiable (w.v.h.p.) Δ
4.2667
x1 ∨ x2 ∨ x5
x1 ∨ x̅3 ∨ x9
x̅3 ∨ x̅4 ∨ x8
x5 ∨ x7 ∨ x̅12
x8 ∨ x9 ∨ x̅n−3 ⋮ I
x1 ∨ x2 ∨ x5
x1 ∨ x̅3 ∨ x9
x̅3 ∨ x̅4 ∨ x8
x5 ∨ x7 ∨ x̅12
x8 ∨ x9 ∨ x̅n−3 ⋮ I
“Prove the
following I is satisfiable:”
“Prove the
following I is unsatisfiable:”
We don’t want the situation on the left.
A proof will typically just be a satisfying assignment.
Too easy for the students to copy off each other.
Too easy for the TAs to grade (we want to be cruel to them too!). Also, in seriousness, kind of easy. Statistical physics based ideas give fairly effective algorithms.

8. Δ But what Δ to choose? For Δ ≥ 10 (say),
it’s ridiculously easy to prove that I will be unsatisfiable (w.v.h.p.)
I is unsatisfiable (w.v.h.p.) Δ
4.2667
x1 ∨ x2 ∨ x5
x1 ∨ x̅3 ∨ x9
x̅3 ∨ x̅4 ∨ x8
x5 ∨ x7 ∨ x̅12
x8 ∨ x9 ∨ x̅n−3 ⋮ I
“Prove the
following I is unsatisfiable:”
But once you quench I,
proving that particular I
is unsatisfiable may be hard.

9. I ⋮ General CSP instance I P1(x7,x5,x9,xn,x2) Δn variables x1, …, xn
P2(x6,x4,xn−3,x2,x1)
P3(x42,x3,x4,x7,x2)
P4(x5,x3,x11,x6,x8)
PΔn(x4,x8,x9,x3,x5)
General CSP instance I
Δn variables x1, …, xn
to be assigned values in some small domain Ω
Δn constraints;
each Pi from small set P
Random CSP(P ):
constraints chosen randomly

10. “Prove it’s not 3-colorable.”
Example:
General CSP instance I
Ω = {red, green, blue}
Δn variables x1, …, xn
to be assigned values in some small domain Ω
Δn constraints;
each Pi from small set P
P = {≠}
x1≠x7 ⋮ I
x2≠x4
x2≠x9
x6≠x25
x9≠xn
Fixed I ~ G(n,m), m = Δn.
“Prove it’s not 3-colorable.”
Random CSP(P ):
constraints chosen randomly

11. I There’s not gonna be. (unless Δ ≥ n1/3 !)
I hope there’s a K4 in here!
There’s not gonna be. (unless Δ ≥ n1/3 !)
Example:
Ω = {red, green, blue}
P = {≠}
x1≠x7 ⋮ I
x2≠x4
x2≠x9
x6≠x25
x9≠xn
Fixed I ~ G(n,m), m = Δn.
“Prove it’s not 3-colorable.”

12. to be assigned values in some small domain Ω
Example 2:
General CSP instance I
Ω = {0,1}
Δn variables x1, …, xn
to be assigned values in some small domain Ω
Δn constraints;
each Pi from small set P
P = {TSA5 of literals}
TSA5(p,q,r,s,t)
= p⊕q⊕r⊕(s∧t)
Important for highly-efficient
cryptography [Goldreich’00].
(Δ = PRG stretch factor.)
Random CSP(P ):
constraints chosen randomly

13. so that, if we form a random CSP(P) instance I,
What we want to know:
For a given predicate P,
how large can we set Δ
so that, if we form a random CSP(P) instance I,
the task “Prove I is unsatisfiable” will (w.h.p.) be impossible for any poly(n)-time student?
algorithm

14. SOS Δ What’s known for 3SAT ??
“Prove the following I is unsatisfiable:”
efficient “spectral” algorithm [FG’01]
efficient combinatorial algorithm [BKPS’98]
trivial refutation algorithm
x1 ∨ x2 ∨ x5
x1 ∨ x̅3 ∨ x9
x̅3 ∨ x̅4 ∨ x8
x5 ∨ x7 ∨ x̅12
x8 ∨ x9 ∨ x̅n−3 ⋮ I or
SOS ?? Δ
4.2667
n.5 n n2

15. SOS (The “Sum of Squares” Algorithm)
The polynomial-time algorithm all the kids these days are using.
Can use it to try to refute any CSP(P) instance.
(Or indeed, most any mathematical statement.)
Encapsulates all* known techniques.

16. If I is a random instance of CSP(P),
Theorem:
[AOW’15]
If I is a random instance of CSP(P),
then (w.h.p.) SOS will certify it’s unsatisfiable,
provided Δ ≳ ncmplx(P)−2. 2
cmplx(P) is an easy-to-calculate integer, measuring some “pseudorandomness” of P.
For kSAT (that is, P = ORk), cmplx(P) = k.

17. For full credit, prove a statement of the form
“there is no assignment satisfying ≥ β fraction of constraints in I” for the smallest β you can.

18. β
Random 3SAT
w.h.p., max fraction of satisfiable constraints
what [AOW’15] shows SOS can certify 1 7 8 Δ
const
n.5

19. ∙∙∙ ∙∙∙ β Δ Random CSP(P) const n.5 n n1.5 n(k−2)/2
w.h.p., max fraction of satisfiable constraints
what [AOW’15] shows SOS can certify 1
∙∙∙
σ2(P)
σ3(P)
σ4(P)
σ5(P)
E[P] Δ
const
n.5 n
n1.5
n(k−2)/2
∙∙∙

20. what [AOW’15] shows SOS can certify in polynomial timeβ
Random CSP(P)
what [AOW’15] shows SOS can certify in polynomial time 1
∙∙∙
[RRS’16]: if you allow SOS 2nϵ time, replace each “n” on the horizontal axis with n1−ϵ
σ2(P)
σ3(P)
σ4(P)
σ5(P)
E[P] Δ
const
n.5 n
n1.5
n(k−2)/2
∙∙∙

21. ∙∙∙ ∙∙∙ β Δ Random CSP(P) Theorem [KMOW’17]:
what [AOW’15] shows SOS can certify in polynomial time 1
∙∙∙
[RRS’16]: if you allow SOS 2nϵ time, replace each “n” on the horizontal axis with n1−ϵ
σ2(P)
σ3(P)
σ4(P)
σ5(P)
Theorem [KMOW’17]:
Everything on this slide is tight. (Up to lower-order terms.)
SOS does no better than this.
E[P] Δ
const
n.5 n
n1.5
n(k−2)/2
∙∙∙

22. Perhaps this tells us how secure the associated crypto objects are.
Perhaps no algorithm does better than that.
Theorem [KMOW’17]:
Everything on that slide is tight. (Up to lower-order terms.)
SOS does no better than that.

23. ξ is a t-wise uniform distribution on {0,1}k
σt(P) = max Pr [w satisfies P]
σt(P)
w ~ ξ
ξ is a t-wise uniform distribution on {0,1}k
E.g., 3SAT, where P = OR3
σ2(P) = 1
001
010
011
100
101
110
111

24. ξ is a t-wise uniform distribution on {0,1}k
σt(P) = max Pr [w satisfies P]
σt(P)
w ~ ξ
ξ is a t-wise uniform distribution on {0,1}k
E.g., 3SAT, where P = OR3
σ2(P) = 1
001
010
011
100
101
110
111 7 8
σ3(P) =
Because: let ξ put probability ¼ on
each of these w’s
Because: the only 3-wise uniform distribution is the full uniform distrib. on {0,1}3

25. what [AOW’15] shows SOS can certify in polynomial timeβ
Random CSP(P)
what [AOW’15] shows SOS can certify in polynomial time 1
∙∙∙
σ2(3SAT) = 1
σ2(P)
σ3(P)
σ4(P)
σ5(P)
σ3(3SAT) = 7 8
E[P] Δ
const
n.5 n
n1.5
n(k−2)/2
∙∙∙

26. β
Random 3SAT
w.h.p., max fraction of satisfiable constraints
what [AOW’15] shows SOS can certify 1 7 8 Δ
const
n.5

27. If there were time: 2 slides for the experts
To show “degree-d SOS” fails to refute I,
you need to show that I is “degree-d pseudo-satisfiable”.
That is, there’s a degree-d pseudo-distribution (signed measure) Ẽ on global assignments x such that Ẽ[1] = 1, Ẽ[xi2 p(x)] = Ẽ[xi p(x)], and Ẽ[p(x)2] ≥ 0 for all n-variate polynomials p(x) of degree at most d/2,
and, such that Ẽ “satisfies” I, meaning that if I contains, e.g., “x1 ∨ x̅3 ∨ x9”, then Ẽ[(1−x1) x3 (1−x9)] = 0.

28. If there were time: 2 slides for the experts
To show “degree-d SOS” fails to refute I,
you need to show that I is “degree-d pseudo-satisfiable”.
That is, there’s a degree-d pseudo-distribution (signed measure) Ẽ on global assignments x such that Ẽ[1] = 1, Ẽ[xi2 p(x)] = Ẽ[xi p(x)], and Ẽ[p(x)2] ≥ 0 for all n-variate polynomials p(x) of degree at most d/2,
and, such that Ẽ “satisfies” I, meaning that if I contains, e.g., “x1 ∨ x̅3 ∨ x9”, then Ẽ[(1−x1) x3 (1−x9)] = 0.
We’ll even get this to be Prξ[(0,1,0)].
Ẽ[(1−x1) x3 (1−x9)]

29. If there were time: 2 slides for the experts
Previous work […, BGMT’12, BCK’15]:
Define notion of “closure” in constraint hypergraph.
Prove closures are “small” in random hypergraphs.
Define Ẽ[xS] based on “planted ξ-distribution” on closure(S).
Pain and suffering to show Ẽ[p(x)2] ≥ 0 for p(x) of degree at most d/2.
One of our contributions: I think we defined the “right” notion of “closure”.

30. Summary: Algorithmic thresholds?
I is satisfiable (w.v.h.p.)
I is unsatisfiable (w.v.h.p.) Δ
Can efficiently certify I is satisfiable (w.h.p.)
Can efficiently certify I is unsatisfiable (w.h.p.)
satisfiability threshold (e.g., )
SOS pseudo- satisfiability threshold?
Open problem: Drill down into
constants, for, e.g., k-Colorability.

31. PS: I actually don’t care for Harry Potter.
I read the first book and thought it was OK.