SOS is not obviously automatizable, even approximately Ryan O’Donnell Carnegie Mellon

this is Linear Programming, exactly solvable in P. [Ellipsoid, K’79]. minimize P(x1, …, xn) subject to Q1(x1, …, xn) ≥ 0 ··· Qm(x1, …, xn) ≥ 0 If degree of P, Q1, …, Qm is =1, this is Linear Programming, exactly solvable in P. [Ellipsoid, K’79].

can encode NP-hard problems. minimize P(x1, …, xn) subject to Q1(x1, …, xn) ≥ 0 ··· Qm(x1, …, xn) ≥ 0 If degree of P, Q1, …, Qm is =1, can encode NP-hard problems. ≥2

can encode NP-hard problems. Max-Cut: minimize subject to ∀ i ∈ V If degree of P, Q1, …, Qm is =1, can encode NP-hard problems. ≥2

can encode NP-hard problems. Max-Cut: Bisection minimize subject to ∀ i ∈ V x1+ ··· +xn = 0 If degree of P, Q1, …, Qm is =1, can encode NP-hard problems. ≥2

can encode NP-hard problems. Min-Vertex-Cover minimize x1+ ··· +xn subject to ∀ i ∈ V (1−xi)(1−xj) = 0 ∀ ij ∈ E If degree of P, Q1, …, Qm is =1, can encode NP-hard problems. ≥2

SOS(d) hierarchy of SDP relaxations So hot right now: SOS(d) hierarchy of SDP relaxations “Degree / round” parameter d ∈ ℕ As d increases, so do quality and run-time For Max-Cut, SOS(2) = Goemans-Williamson

“The SoS hierarchy… can be solved in time nO(d) for degree parameter d [BHKKMP16] “The SOS algorithm… with… parameter ℓ takes nO(ℓ) time to run.” [Bar14] “If a [degree-d SOS] proof exists it can be found in time nO(d).” [OZ12] “The r round [SOS] relaxation can be solved in nO(r) time on instances of size n.” [MPW15] “By semidefinite programming finding such a [degree ℓ dual solution] can be done in nO(ℓ) time.” [Bar14] “There is an algorithm running in time nO(k) to optimize [in the degree-k SOS proof system].” [Ats13] “If there is a degree-ℓ SOS proof… it can be found in mnO(ℓ) time.” [BS14] “If there is a degree-d SoS proof with bit complexity nO(d), then one can be found in time nO(d).” [RRS16] “… this refutation can also be found efficiently by solving a semidefinite program of size nO(d).” [KOTZ13] “By using semidefinite programming, [SoS algorithm of degree q] run[s] in time nO(q).” [BGGLT16] “n-variable degree-d SOS proofs can be found in nO(d) time whenever they exist.” [AOW15] “[If there is an SOS] proof of infeasibility of degree d, then it can be found in time O(m·nO(d)).” [DMN12] “Hence the r-round SoS relaxation can be solved up to accuracy ϵ in time (mn·log(1/ϵ))O(r).” [BBHKSZ12] “We can find in (n polylog(M/ϵ))O(k) time [an ϵ-accurate, degree-k SOS dual solution].” [BKS14] “For any fixed t, [the SOS relaxation of order t] can be computed in polynomial time (to any precision).” [Lau10] “We can compute the SOS(6) [relaxation] within arbitrary accuracy in polynomial time.” [BM15] “For each fixed t, [the degree-t SOS relaxation] may be computed… in poly. time to any fixed accuracy.” [deKL10] “One [can] optimize over [t-round SOS polytope] in time nO(t)·mO(1) up to numerical errors.” [Rot13]

“The SoS hierarchy… can be solved in time nO(d) for degree parameter d [BHKKMP16] “The SOS algorithm… with… parameter ℓ takes nO(ℓ) time to run.” [Bar14] “If a [degree-d SOS] proof exists it can be found in time nO(d).” [OZ12] “The r round [SOS] relaxation can be solved in nO(r) time on instances of size n.” [MPW15] “By semidefinite programming finding such a [degree ℓ dual solution] can be done in nO(ℓ) time.” [Bar14] “There is an algorithm running in time nO(k) to optimize [in the degree-k SOS proof system].” [Ats13] “If there is a degree-ℓ SOS proof… it can be found in mnO(ℓ) time.” [BS14] “If there is a degree-d SoS proof with bit complexity nO(d), then one can be found in time nO(d).” [RRS16] “… this refutation can also be found efficiently by solving a semidefinite program of size nO(d).” [KOTZ13] “By using semidefinite programming, [SoS algorithm of degree q] run[s] in time nO(q).” [BGGLT16] “n-variable degree-d SOS proofs can be found in nO(d) time whenever they exist.” [AOW15] “[If there is an SOS] proof of infeasibility of degree d, then it can be found in time O(m·nO(d)).” [DMN12] “Hence the r-round SoS relaxation can be solved up to accuracy ϵ in time (mn·log(1/ϵ))O(r).” [BBHKSZ12] “We can find in (n polylog(M/ϵ))O(k) time [an ϵ-accurate, degree-k SOS dual solution].” [BKS14] “For any fixed t, [the SOS relaxation of order t] can be computed in polynomial time (to any precision).” [Lau10] “We can compute the SOS(6) [relaxation] within arbitrary accuracy in polynomial time.” [BM15] “For each fixed t, [the degree-t SOS relaxation] may be computed… in poly. time to any fixed accuracy.” [deKL10] “One [can] optimize over [t-round SOS polytope] in time nO(t)·mO(1) up to numerical errors.” [Rot13]

“The SoS hierarchy… can be solved in time nO(d) for degree parameter d [BHKKMP16] “The SOS algorithm… with… parameter ℓ takes nO(ℓ) time to run.” [Bar14] “If a [degree-d SOS] proof exists it can be found in time nO(d).” [OZ12] “The r round [SOS] relaxation can be solved in nO(r) time on instances of size n.” [MPW15] “By semidefinite programming finding such a [degree ℓ dual solution] can be done in nO(ℓ) time.” [Bar14] “There is an algorithm running in time nO(k) to optimize [in the degree-k SOS proof system].” [Ats13] “If there is a degree-ℓ SOS proof… it can be found in mnO(ℓ) time.” [BS14] “If there is a degree-d SoS proof with bit complexity nO(d), then one can be found in time nO(d).” [RRS16] “… this refutation can also be found efficiently by solving a semidefinite program of size nO(d).” [KOTZ13] “By using semidefinite programming, [SoS algorithm of degree q] run[s] in time nO(q).” [BGGLT16] “n-variable degree-d SOS proofs can be found in nO(d) time whenever they exist.” [AOW15] “[If there is an SOS] proof of infeasibility of degree d, then it can be found in time O(m·nO(d)).” [DMN12] “Hence the r-round SoS relaxation can be solved up to accuracy ϵ in time (mn·log(1/ϵ))O(r).” [BBHKSZ12] “We can find in (n polylog(M/ϵ))O(k) time [an ϵ-accurate, degree-k SOS dual solution].” [BKS14] “For any fixed t, [the SOS relaxation of order t] can be computed in polynomial time (to any precision).” [Lau10] “We can compute the SOS(6) [relaxation] within arbitrary accuracy in polynomial time.” [BM15] “For each fixed t, [the degree-t SOS relaxation] may be computed… in poly. time to any fixed accuracy.” [deKL10] “One [can] optimize over [t-round SOS polytope] in time nO(t)·mO(1) up to numerical errors.” [Rot13]

“The SoS hierarchy… can be solved in time nO(d) for degree parameter d [BHKKMP16] “The SOS algorithm… with… parameter ℓ takes nO(ℓ) time to run.” [Bar14] “If a [degree-d SOS] proof exists it can be found in time nO(d).” [OZ12] “The r round [SOS] relaxation can be solved in nO(r) time on instances of size n.” [MPW15] “By semidefinite programming finding such a [degree ℓ dual solution] can be done in nO(ℓ) time.” [Bar14] “There is an algorithm running in time nO(k) to optimize [in the degree-k SOS proof system].” [Ats13] “If there is a degree-ℓ SOS proof… it can be found in mnO(ℓ) time.” [BS14] “If there is a degree-d SoS proof with bit complexity nO(d), then one can be found in time nO(d).” [RRS16] “… this refutation can also be found efficiently by solving a semidefinite program of size nO(d).” [KOTZ13] “By using semidefinite programming, [SoS algorithm of degree q] run[s] in time nO(q).” [BGGLT16] “n-variable degree-d SOS proofs can be found in nO(d) time whenever they exist.” [AOW15] “[If there is an SOS] proof of infeasibility of degree d, then it can be found in time O(m·nO(d)).” [DMN12] “Hence the r-round SoS relaxation can be solved up to accuracy ϵ in time (mn·log(1/ϵ))O(r).” [BBHKSZ12] “We can find in (n polylog(M/ϵ))O(k) time [an ϵ-accurate, degree-k SOS dual solution].” [BKS14] “For any fixed t, [the SOS relaxation of order t] can be computed in polynomial time (to any precision).” [Lau10] “We can compute the SOS(6) [relaxation] within arbitrary accuracy in polynomial time.” [BM15] “For each fixed t, [the degree-t SOS relaxation] may be computed… in poly. time to any fixed accuracy.” [deKL10] “One [can] optimize over [t-round SOS polytope] in time nO(t)·mO(1) up to numerical errors.” [Rot13]

“The SoS hierarchy… can be solved in time nO(d) for degree parameter d [BHKKMP16] “The SOS algorithm… with… parameter ℓ takes nO(ℓ) time to run.” [Bar14] “If a [degree-d SOS] proof exists it can be found in time nO(d).” [OZ12] “The r round [SOS] relaxation can be solved in nO(r) time on instances of size n.” [MPW15] “By semidefinite programming finding such a [degree ℓ dual solution] can be done in nO(ℓ) time.” [Bar14] “There is an algorithm running in time nO(k) to optimize [in the degree-k SOS proof system].” [Ats13] “If there is a degree-ℓ SOS proof… it can be found in mnO(ℓ) time.” [BS14] “If there is a degree-d SoS proof with bit complexity nO(d), then one can be found in time nO(d).” [RRS16] “… this refutation can also be found efficiently by solving a semidefinite program of size nO(d).” [KOTZ13] “By using semidefinite programming, [SoS algorithm of degree q] run[s] in time nO(q).” [BGGLT16] “n-variable degree-d SOS proofs can be found in nO(d) time whenever they exist.” [AOW15] “[If there is an SOS] proof of infeasibility of degree d, then it can be found in time O(m·nO(d)).” [DMN12] “Hence the r-round SoS relaxation can be solved up to accuracy ϵ in time (mn·log(1/ϵ))O(r).” [BBHKSZ12] “We can find in (n polylog(M/ϵ))O(k) time [an ϵ-accurate, degree-k SOS dual solution].” [BKS14] “For any fixed t, [the SOS relaxation of order t] can be computed in polynomial time (to any precision).” [Lau10] “We can compute the SOS(6) [relaxation] within arbitrary accuracy in polynomial time.” [BM15] “For each fixed t, [the degree-t SOS relaxation] may be computed… in poly. time to any fixed accuracy.” [deKL10] “One [can] optimize over [t-round SOS polytope] in time nO(t)·mO(1) up to numerical errors.” [Rot13]

*More accurately: It is not known to be true. You would be forgiven for thinking that… “The SOS(d) relaxation can be approximately computed in nO(d) time by SDP.” * This statement is false. *More accurately: It is not known to be true. And I would bet on it being false.

This statement is still false. * You would be forgiven for thinking that… “The SOS(d) relaxation can be approximately computed in nO(d) time by SDP.” This statement is still false. * (Still false even when the relaxation is “Archimedean / explicitly bounded”, meaning we have for all i.)

Say P(x,y,z) = x4+xy3+y4−3x2yz−4xy2z+2x2z2+xz3+yz3+z4 Optimization: Minimize P(x,y,z) on ℝ3. Decision: Is P(x,y,z) ≥ 0 on ℝ3? SOS(4) relaxation: Is P(x,y,z) a sum of squared polynomials?

⇔ Say P(x,y,z) = x4+xy3+y4−3x2yz−4xy2z+2x2z2+xz3+yz3+z4 SOS(4) relaxation: Is P(x,y,z) a sum of squared polynomials? ⇔ Is the following SDP feasible? M

SDP in P, by Ellipsoid Method [GLS’88]. In general, n-variable SOS(d) is equivalent to feasibility of an SDP with ~nd variables. SDP in P, by Ellipsoid Method [GLS’88]. ? ∴ SOS(d) in time nO(d). SDP-Feasibility ∈ P: unknown SDP-Feasibility ∈ NP: unknown Deciding if a poly is sum of squares ∈ NP: unknown ✔ SDP-Feasibility ∈ PSPACE:

M

Ellipsoid Algorithm

R

r

r issue: Almost always a problem in theory. Maybe not a problem if it’s OK to not satisfy constraints exactly. R issue: Typically unclear. (Despite constraints.)

Example from the paper: minimize subject to SOS(2) relaxation: Mathematically exact. Computationally: Any ±.01 accurate SOS(2) solution requires exp(n) many bits to write down.

then you have to take some care. “The SoS hierarchy… can be solved in time nO(d) for degree parameter d.” [BHKKMP16] “The SOS algorithm… with… parameter ℓ takes nO(ℓ) time to run.” [Bar14] “If a [degree-d SOS] proof exists it can be found in time nO(d).” [OZ12] “The r round [SOS] relaxation can be solved in nO(r) time on instances of size n.” [MPW15] “By semidefinite programming finding such a [degree ℓ dual solution] can be done in nO(ℓ) time.” [Bar14] “There is an algorithm running in time nO(k) to optimize [in the degree-k SOS proof system].” [Ats13] If your motivation is to use SOS to show that certain approximation ratios are achievable in P, in the Turing Machine model, then you have to take some care. “If there is a degree-ℓ SOS proof… it can be found in mnO(ℓ) time.” [BS14] “If there is a degree-d SoS proof with bit complexity nO(d), then one can be found in time nO(d).” [RRS16] “… this refutation can also be found efficiently by solving a semidefinite program of size nO(d).” [KOTZ13] “By using semidefinite programming, [SoS algorithm of degree q] run[s] in time nO(q).” [BGGLT16] “n-variable degree-d SOS proofs can be found in nO(d) time whenever they exist.” [AOW15] “[If there is an SOS] proof of infeasibility of degree d, then it can be found in time O(m·nO(d)).” [DMN12] “Hence the r-round SoS relaxation can be solved up to accuracy ϵ in time (mn·log(1/ϵ))O(r).” [BBHKSZ12] “We can find in (n polylog(M/ϵ))O(k) time [an ϵ-accurate, degree-k SOS dual solution].” [BKS14] “For any fixed t, [the SOS relaxation of order t] can be computed in polynomial time (to any precision).” [Lau10] “We can compute the SOS(6) [relaxation] within arbitrary accuracy in polynomial time.” [BM15] “For each fixed t, [the degree-t SOS relaxation] may be computed… in poly. time to any fixed accuracy.” [deKL10] “One [can] optimize over [t-round SOS polytope] in time nO(t)·mO(1) up to numerical errors.” [Rot13]

Takeaways I love SOS. All of those papers are great. In many (but not all) cases, it’s not too hard to make everything okay. I don’t know a general theory for when it’s okay. One case that’s definitely okay: Constraint Set = Can indeed solve to accuracy 2−n in nO(d) time. (Goemans−Williamson is okay!)

Nice(?) open problem Okay when Constraint Set ⊇ ? That is, can SOS(d) be solved in nO(d) time to accuracy 2−n when all variables constrained to be Boolean, but there are additional constraints?

Thanks!