1. Sum of Squares, Planted Clique, and Pseudo-Calibration
Joint work with Boaz Barak, Sam Hopkins, Pravesh Kothari, Jonathan Kelner, and Ankur Moitra

2. Planted Clique Lower Bound
Theorem [BHKKMP, FOCS 2016]: If the input graph 𝐺 is chosen randomly from the Erdös- Rényi distribution 𝐺(𝑛, 1 2 ), then with high probability the degree 𝑑 sum of squares hierarchy gives a value of at least 𝑛 1 2 −𝑐 𝑑 𝑙𝑜𝑔𝑛 for the size of the largest clique in 𝐺.

3. Talk Outline Part I: The Sum of Squares Hierarchy
Part II: Planted Clique
Part III: Pseudo-calibration

4. Part I: The Sum of Squares Hierarchy

5. The Sum of Squares Hierarchy
Developed independently by Shor, Nesterov, Parrillo, and Lasserre.
Generalization of linear and semidefinite programming
Strictly stronger than the Lovasz-Schrijver hierarchy and the Sherali-Adams hierarchy
Captures the Goemans-Williamson algorithm for MAX-CUT, the Goemens-Linial algorithm for sparsest cut (analyzed by Arora, Rao, Vazirani), and the Arora-Barak-Steurer subexponential time algorithm for Unique Games
Leading candidate for refuting Khot’s Unique Games Conjecture.

6. Positivstellensatz Proofs
Setup: Want to know if equations
𝑠 1 𝑥 1 ,…, 𝑥 𝑛 =0, 𝑠 2 𝑥 1 ,…, 𝑥 𝑛 =0, …
can be solved simultaneously over ℝ.
Degree 𝑑 Positivstellensatz proof of infeasibility: Equation of the form
−1= 𝑖 𝑓 𝑖 𝑠 𝑖 + 𝑗 𝑔 𝑗 2 where for all 𝑖, deg 𝑓 𝑖 + deg 𝑠 𝑖 ≤𝑑 and for all 𝑗, deg 𝑔 𝑗 ≤ 𝑑 2

7. Degree 𝑑 Sum of Squares
Stengle’s Positivstellensatz: If equations are infeasible, there exists a Positivstellensatz proof of infeasibility.
However, degree could be very high
Degree 𝑑 Sum of Squares hierarchy: Returns NO if there is a degree 𝑑 Positivstellensatz proof of infeasibility, otherwise returns YES.
Fundamental questions: At what degree is there a Positivstellesatz proof of infeasibility? If there is no degree 𝑑 proof, how do we show it?

8. Pseudo-expectation values
Pseudo-expectation values: linear mapping Ẽ from polynomials of degree ≤𝑑 to values in ℝ satisfying the following conditions:
Ẽ 1 =1
Ẽ 𝑓 𝑠 𝑖 =0 whenever deg 𝑓 + deg 𝑠 𝑖 ≤𝑑
Ẽ 𝑔 2 ≥0 whenever deg 𝑔 ≤ 𝑑 2
Intuition: Ẽ “looks like” the expected values over a distribution of solutions.

9. Duality Degree 𝑑 Positivstellensatz proof: −1= 𝑖 𝑓 𝑖 𝑠 𝑖 + 𝑗 𝑔 𝑗 2
−1= 𝑖 𝑓 𝑖 𝑠 𝑖 + 𝑗 𝑔 𝑗 2
Pseudo-expectation values:
Ẽ 1 =1
Ẽ 𝑓 𝑖 𝑠 𝑖 =0
Ẽ 𝑔 𝑗 2 ≥0
Cannot both exist, otherwise
−1=Ẽ −1 = 𝑖 Ẽ[𝑓 𝑖 𝑠 𝑖 ] + 𝑗 Ẽ[𝑔 𝑗 2 ] ≥0
If strong duality holds (which we will assume), one or the other will exist

10. The Moment Matrix 𝑞
𝑝,𝑞 are monomials of degree at most 𝑑 2 . 𝑝
Ẽ[𝑝q] 𝑀
Each 𝑔 of degree ≤ 𝑑 2 can be viewed as a vector in the basis of monomials
Ẽ[ 𝑔 2 ] =𝑔𝑇𝑀𝑔
Ẽ 𝑔 2 ≥0 whenever deg 𝑔 ≤ 𝑑 2 ⇔M is PSD (positive semi-definite)

11. Lower Bounds Strategy for SOS
To prove a lower bound on SOS, we must
Construct pseudo-expectation values Ẽ and the corresponding moment matrix 𝑀
Show that Ẽ obeys the required equalities and that 𝑀 is PSD (this is the hard part).

12. Semidefinite Programs for SOS
Can search for the moment matrix 𝑀 (and the corresponding pseudo-expectation values Ẽ) with a semidefinite program of size 𝑛 𝑂(𝑑) .
The conditions that Ẽ 1 =1 and Ẽ 𝑓 𝑠 𝑖 =0 whenever deg 𝑓 + deg 𝑠 𝑖 ≤𝑑 give linear constraints on entries of 𝑀.
𝑀 must be PSD
SOS gives a hierarchy of increasingly powerful (and large) semidefinite programs

13. Optimization with SOS Often have equations with parameter(s)
Want to optimize over green region, SOS optimizes over the blue and green regions.
Equations are feasible
Infeasible but no proof
Positivstellensatz proof of infeasibility
As we increase the degree, the blue region shrinks.

14. Approximation Algorithms with SOS
For many problems, there is a method for rounding the pseudo-expectation values Ẽ into an actual solution (with worse parameters). This gives an approximation algorithm.
Optimal Solution
Equations are feasible Ẽ
A Solution
Infeasible but no proof
Positivstellensatz proof of infeasibility

15. For more info on SOS
Princeton sum of squares seminar website
Harvard sum of squares seminar website

16. Part II: Planted Clique

17. The Planted Clique Problem
G(n,1/2) + clique(ω)
Jerrum 92, Kucera 95: For which ω can we find the planted clique?
Best- Alon et al. 98: ω= Ω( 𝑛 ) a j b i c h d g e f
Can you find the 5-clique?

18. The Planted Clique Problem
G(n,1/2) + clique(ω)
Jerrum 92, Kucera 95: For which ω can we find the planted clique?
Best- Alon et al. 98: ω= Ω( 𝑛 ) a j b i c h d g e f
This 5-clique was planted by adding the red edge.

19. Equations for 𝜔-Clique
Variable 𝑥 𝑖 for each vertex i in G.
Want 𝑥 𝑖 =0 if i is not in the clique
Want 𝑥 𝑖 =1 if i is in the clique.
Equations:
𝑥 𝑖 2 = 𝑥 𝑖 for all i.
𝑥 𝑖 𝑥 𝑗 = 0 if 𝑖,𝑗 ∉𝐸(𝐺)
𝑖 𝑥 𝑖 = 𝜔
These equations are feasible precisely when G contains a 𝜔 -clique.

20. Part III: Pseudo-Calibration

21. Choosing Ẽ: First attempt
How should we choose Ẽ?
First attempt: give every clique of size 𝑑 the same weight.
Definition: Define 𝑥 𝑉 = 𝑖∈𝑉 𝑥 𝑖
𝐸 𝑥 𝑉 ≈ 2 |𝑉| 𝜔 |𝑉| 𝑛 |𝑉| if 𝑉 is a clique, 𝐸 𝑥 𝑉 =0 otherwise.
This gives non-trivial lower bounds, but cannot give the full lower bound

22. Choosing Ẽ: Second attempt
Second attempt: See what went wrong and fix it
This works for degree 𝑑=4 but is ad-hoc and would be very complicated for higher degrees.

23. Choosing Ẽ: Pseudo-calibration
Planted distribution: Each vertex is in the clique with probability 𝜔 𝑛 , 𝑥 𝑖 =1 if 𝑖∈𝑐𝑙𝑖𝑞𝑢𝑒, 0 otherwise
Idea: 𝐸 should match the planted distribution on low degree tests

24. Fourier Characters 𝜒 𝐸
Definition: Given a set 𝐸 of possible edges of 𝐺, define 𝜒 𝐸 𝐺 =− 1 |𝐸∖𝐸(𝐺)|
𝑥 1
𝑥 2
𝑥 4
𝑥 3 𝐺
Example: If 𝐸={ 𝑥 1 , 𝑥 2 , 𝑥 1 , 𝑥 3 ,( 𝑥 1 , 𝑥 4 )} then 𝜒 𝐸 𝐺 =−1 as 𝐸∖𝐸 𝐺 =1

25. Pseudo-calibration
For all 𝑉,𝐸, 𝐸 𝐺∼𝐺 𝑛, 𝜒 𝐸 𝐸 [ 𝑥 𝑉 ] = 𝐸 𝐺,𝑥∼𝐺 𝑛, 𝐾 𝜔 [ 𝑥 𝑉 𝜒 𝐸 ]
Right hand side is 0 (over the random part of G) unless 𝑉∪𝑉 𝐸 ⊆𝑐𝑙𝑖𝑞𝑢𝑒, in which case the right hand side is 1.
Each vertex is in the clique with probability 𝜔 𝑛 , so the right hand side has value ω 𝑛 𝑉 𝐸 ∪𝑉

26. Pseudo-calibrated moments
∀𝑉,𝐸, 𝐸 𝐺∼𝐺 𝑛, 𝜒 𝐸 𝐸 [ 𝑥 𝑉 ] = 𝜔 𝑛 |𝑉∪𝑉(𝐸)|
𝐸 𝑥 𝑉 = 𝐸: 𝑉 𝐸 ∪𝑉 <τ ω 𝑛 𝑉 𝐸 ∪𝑉 𝜒 𝐸 𝐺
Note: We only have 𝑖 𝑥 𝑖 ≈ω with these moments.

27. Properties of 𝜒 𝐸 Recall: 𝜒 𝐸 𝐺 =− 1 |𝐸∖𝐸(𝐺)|
If we define the inner product 𝑓,𝑔 = Ε 𝐺 [ 𝑓 𝐺 𝑔 𝐺 ] where 𝑓,𝑔 are functions of the input graph 𝐺, then
𝜒 𝐸 1 𝜒 𝐸 2 = 𝜒 𝐸 1 Δ 𝐸 2 (Δ = symmetric difference)

28. Current/Future work
Can we apply these techniques to prove sum of squares lower bounds for other planted problems?
Can we find general conditions for the pseudo-calibrated 𝐸 which imply a sum of squares lower bound?
Can we prove our lower bound with an 𝐸 which fully respects the equation 𝑥 𝑥 𝑖 =ω ?

29. Acknowledgements
Thanks to the National Science Foundation, Microsoft Research, and the Simons Foundation for supporting this research.

30. Thank you!