1. Georgina Hall Princeton, ORFE Joint work with Amir Ali Ahmadi
LP, SOCP, and Optimization-Free Approaches to Polynomial Optimization and Difference of Convex Programming
Georgina Hall
Princeton, ORFE
Joint work with
Amir Ali Ahmadi

2. Two problems of interest
Lower bounds for
polynomial optimization problems
Writing a polynomial as a difference of two convex polynomials
Can we provide lower bounds on the polynomial optimization problem (POP)
min 𝑥∈ ℝ 𝑛 𝑝(𝑥)
s.t. 𝑔 𝑖 𝑥 ≥0, 𝑖=1,…,𝑚
where 𝑝, 𝑔 𝑖 , 𝑖=1,…,𝑚 are polynomials?
Given a polynomial 𝑓 𝑥 ,
can we find
convex polynomials 𝑔 𝑥 and ℎ(𝑥)
such that
𝑓 𝑥 =𝑔 𝑥 −ℎ 𝑥 ?
First part of the talk
Second part of the talk

3. Part I: Computing lower bounds for POPs

4. Lower bounds on POPs: applications (1/2)
Polynomial optimization problems
min 𝑥∈ ℝ 𝑛 𝑝(𝑥)
𝑠.𝑡. 𝑔 𝑖 𝑥 ≥0
Optimal power flow problem
Economics and game theory
Sensor network localization

5. Lower bounds on POPs: applications (2/2)
Infeasibility of a set of polynomial inequalities
The system
𝑔 𝑖 𝑥 ≥0, 𝑖=1,…,𝑚
is infeasible.
0 is a strict lower bound on
min 𝑥 −𝑔 1 (𝑥)
s.t. 𝑔 2 𝑥 ≥0,…, 𝑔 𝑚 𝑥 ≥0 ⇔
If 0 is not a strict upper bound on g_1(x) then it means that there exists x such that g2(x)>=0,…,gm(x)>=0 and g1(x)>=0 which means the system is feasible
Automated theorem proving
Optimality certificates for discrete optimization

6. Lower bounds on POPs: the SOS approach (1/2)
A polynomial 𝑝 is a sum of squares (SOS) if it can be written as
𝑝 𝑥 = 𝑖 𝑞 𝑖 2 (𝑥) ,
where 𝑞 𝑖 are polynomials.
Example:
Sufficient condition for nonnegativity.
A polynomial 𝑝(𝑥) of degree 2d is SOS if and only if ∃𝑄≽0 such that
where 𝑧= 1, 𝑥 1 ,…, 𝑥 𝑛 , 𝑥 1 𝑥 2 ,…, 𝑥 𝑛 𝑑 𝑇 is the vector of monomials of degree up to 𝑑.

7. Lower bounds on POPs: the SOS approach (2/2)
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾≥0,
∀𝑥∈ 𝑥 𝑔 𝑖 𝑥 ≥0, 𝑖=1,…,𝑚}
min 𝑥 𝑝(𝑥)
𝑠.𝑡. 𝑔 𝑖 𝑥 ≥0, 𝑖=1,…,𝑚 =
opt. val.
𝜸 ∗
Under compactness assumptions, for large enough degree =
opt. val.
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾= 𝜎 0 𝑥 + 𝑖 𝜎 𝑖 𝑥 ⋅ 𝑔 𝑖 𝑥 ,
where 𝜎 𝑖 , 𝑖=0,…,𝑚, are SOS.
For fixed degrees of the SOS polynomials, solving this problem is an SDP
This is an SOS certificate that 𝛾 is a lower bound for POP.
There are many other such SOS certificates.

8. The first part of the talk
Goal: avoid using SOS polynomials and SDP to compute lower bounds for POPs
LP and SOCP-based alternatives to
SOS polynomials (applies to any SOS-based certificate)
Using dsos/sdsos polynomials and improvements
New certificate that 𝛾 is a lower bound on the POP that only requires checking if coefficients of a given polynomial are ≥0
Providing a new certificate that does not use optimization at all
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾= 𝜎 0 𝑥 + 𝑖 𝜎 𝑖 𝑥 ⋅ 𝑔 𝑖 𝑥 ,
where 𝜎 𝑖 , 𝑖=0,…,𝑚, are SOS.
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾= 𝜎 0 𝑥 + 𝑖 𝜎 𝑖 𝑥 ⋅ 𝑔 𝑖 𝑥 ,
where 𝜎 𝑖 , 𝑖=0,…,𝑚, are SOS.
dsos/sdsos + improvements
New certificate

9. Part I: Computing lower bounds for POPs
Using dsos/sdsos polynomials and improvements
Providing an optimization-free certificate for lower bounds on POPs

10. LP and SOCP alternatives to sos: dsos and sdsos
Sum of squares (sos)
𝑝 𝑥 =𝑧 𝑥 𝑇 𝑄𝑧 𝑥 , 𝑄≽0
SDP
DD cone ≔ 𝑸 𝑸 𝒊𝒊 ≥ 𝒋 𝑸 𝒊𝒋 , ∀𝒊}
PSD cone≔ 𝑸 𝑸≽𝟎}
SDD cone ≔ 𝑸 ∃ diagonal 𝑫 with 𝑫 𝒊𝒊 >𝟎 s.t. 𝑫𝑸𝑫 𝒅𝒅}
Diagonally dominant sum of squares (dsos)
𝑝 𝑥 =𝑧 𝑥 𝑇 𝑄𝑧 𝑥 , 𝑄 𝑑𝑖𝑎𝑔𝑜𝑛𝑎𝑙𝑙𝑦 𝑑𝑜𝑚𝑖𝑛𝑎𝑛𝑡 (dd) LP
Scaled diagonally dominant sum of squares (sdsos)
𝑝 𝑥 =𝑧 𝑥 𝑇 𝑄𝑧 𝑥 , 𝑄 𝑠𝑐𝑎𝑙𝑒𝑑 𝑑𝑖𝑎𝑔𝑜𝑛𝑎𝑙𝑙𝑦 𝑑𝑜𝑚𝑖𝑛𝑎𝑛𝑡 (sdd)
SOCP
Ahmadi, Majumdar

11. where 𝜎 𝑖 , 𝑖=0,…,𝑚, are dsos/sdsos.
LP and SOCP alternatives to sos: dsos and sdsos
Problem: computing lower bounds for POPs
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾= 𝜎 0 𝑥 + 𝑖 𝜎 𝑖 𝑥 ⋅ 𝑔 𝑖 𝑥 ,
where 𝜎 𝑖 , 𝑖=0,…,𝑚, are sos.
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾= 𝜎 0 𝑥 + 𝑖 𝜎 𝑖 𝑥 ⋅ 𝑔 𝑖 𝑥 ,
where 𝜎 𝑖 , 𝑖=0,…,𝑚, are dsos/sdsos. ≥
opt. val. b
Example:
For a parametric family of polynomials:
𝑝 𝑥 1 , 𝑥 2 =2 𝑥 𝑥 2 4 +𝑎 𝑥 1 3 𝑥 2 +(1−𝑎) 𝑥 1 2 𝑥 2 2 +𝑏 𝑥 1 𝑥 2 3 a

12. Improvements on dsos and sdsos (1/2)
Replacing sos polynomials by dsos/sdsos polynomials:
+: fast bounds
- : not always as good quality (compared to sos)
Iteratively construct a sequence of improving LP/SOCPs
Initialization: Start with the dsos/sdsos polynomials
Column Generation method

13. Improvements on dsos/sdsos (2/2)
Problem of interest: computing lower bounds for POPs Example considered : computing lower bounds for unconstrained POPs
Compactness assumptions + large degree
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾= 𝜎 0 𝑥 + 𝑖 𝜎 𝑖 𝑥 ⋅ 𝑔 𝑖 𝑥 ,
where 𝜎 𝑖 , 𝑖=0,…,𝑚, are SOS.
min 𝑥 𝑝(𝑥)
𝑠.𝑡. 𝑔 𝑖 𝑥 ≥0 =
opt. val.
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾= 𝜎 0 𝑥 ,
where 𝜎 0 is SOS. ≥
opt. val.
min 𝑥 𝑝(𝑥)

14. Method 2: Column generation (1/3)
Focus on LP-based version of this method (SOCP is similar).
Sos problem
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾 SOS
Replace
with
Dsos problem
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾 dsos
Dsos problem
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾=𝒛 𝒙 𝑻 𝑸𝒛 𝒙 , 𝑸 𝒅𝒅
Two different ways of characterizing 𝑸 𝒅𝒅:
② 𝑄 dd ⇔∃ 𝛼 𝑖 ≥0 s.t. 𝑄= 𝑖 𝛼 𝑖 𝑣 𝑖 𝑣 𝑖 𝑇 ,
where 𝑣 𝑖 fixed vector with
at most two nonzero components =±1
① 𝑄 dd ⇔ 𝑄 𝑖𝑖 ≥ 𝑗 |𝑄 𝑖𝑗 | , ∀ 𝑖
[In collaboration with Sanjeeb Dash, IBM Research]

15. Method 2: Column generation (2/3)
Dsos problem
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾= 𝒊 𝜶 𝒊 𝒗 𝒊 𝑻 𝒛 𝒙 𝟐 , 𝜶 𝒊 ≥𝟎
Dsos problem
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾=𝒛 𝒙 𝑻 𝑸𝒛 𝒙 , 𝑸 𝒅𝒅
Using ②
Idea behind the algorithm:
Expand feasible space at each iteration by adding a new vector 𝑣 and variable 𝛼
max 𝛾 𝛾
𝒑 𝒙 −𝜸= 𝒊 𝜶 𝒊 𝒗 𝒊 𝑻 𝒛 𝒙 𝟐 +𝜶 𝒗 𝑻 𝒛 𝒙 𝟐 𝜶 𝒊 ≥𝟎, 𝒂≥𝟎
Question: How to pick 𝒗? Use the dual!

16. Method 2: Column Generation (3/3)
DSOS
+ 5 iterations
SDSOS
+ 5 iterations
Example: minimizing a degree-4 polynomial
𝒏=𝟏𝟓
𝒏=𝟐𝟎
𝒏=𝟐𝟓
𝒏=𝟑𝟎
𝒏=𝟒𝟎 bd
t(s)
DSOS
-10.96
0.38
-18.01
0.74
-26.45
15.51
-36.52
7.88
-62.30
10.68
𝐷𝑆𝑂 𝑆 𝑘
-5.57
31.19
-9.02
471.39
-20.08
600
-32.28
-35.14
SOS
-3.26
5.60
-3.58
82.22
-3.71 NA

17. Part I: Avoiding SDP to compute lower bounds on POPs
Using dsos/sdsos polynomials and improvements
Providing an optimization-free certificate for lower bounds on POPs

18. An optimization free method for lower bounds (1/3)
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾≥0,
∀𝑥∈ 𝑥 𝑔 𝑖 𝑥 ≥0, 𝑖=1,…,𝑚}
min 𝑥∈ ℝ 𝑛 𝑝(𝑥)
𝑠.𝑡. 𝑔 𝑖 𝑥 ≥0, 𝑖=1,…,𝑚 =
opt. val.
2𝑑= maximum degree of 𝑝, 𝑔 𝑖
Under compactness assumptions,
i.e., 𝑥 𝑔 𝑖 𝑥 ≥0}⊆𝐵(0,𝑅) =
opt. val.
For large enough 𝑟
sup 𝛾 𝛾
s.t. 𝑓 𝛾 𝑣 2 − 𝑤 2 − 1 𝑟 𝑖 (𝑣 𝑖 2 − 𝑤 𝑖 2 ) 2 𝑑 + 1 2𝑟 𝑖 𝑣 𝑖 4 + 𝑤 𝑖 𝑑 ⋅ 𝑖 𝑣 𝑖 2 + 𝑖 𝑤 𝑖 𝑟 2
has nonnegative coefficients,
where 𝑓 𝛾 is a form in 𝑛+𝑚+3 variables and of degree 4𝑑
which explicitely depends on 𝑝, 𝑔 𝑖 and 𝑅
Theorem:

19. An optimization free method for lower bounds (2/3)
𝛾 is a strict lower bound on the POP ⇔
∃𝑟∈ℕ s.t. 𝒇 𝜸 𝒗 𝟐 − 𝒘 𝟐 − 𝟏 𝒓 𝒊 (𝒗 𝒊 𝟐 − 𝒘 𝒊 𝟐 ) 𝟐 𝒅 + 𝟏 𝟐𝒓 𝒊 𝒗 𝒊 𝟒 + 𝒘 𝒊 𝟒 𝒅 ⋅ 𝑖 𝑣 𝑖 2 + 𝑖 𝑤 𝑖 𝑟 2 has ≥0 coefficients
Proof sketch of the theorem:
𝛾 is a strict lower bound on the POP ⇔
∃𝑟∈ℕ s.t. 𝒇 𝜸 𝒗 𝟐 − 𝒘 𝟐 − 1 𝑟 𝑖 (𝑣 𝑖 2 − 𝑤 𝑖 2 ) 2 𝑑 + 𝟏 𝟐𝒓 𝒊 𝒗 𝒊 𝟒 + 𝒘 𝒊 𝟒 𝒅 ⋅ 𝑖 𝑣 𝑖 2 + 𝑖 𝑤 𝑖 𝑟 2 has ≥0 coefficients
𝛾 is a strict lower bound on the POP ⇔
∃𝑟∈ℕ s.t. 𝒇 𝜸 𝒗 𝟐 − 𝒘 𝟐 − 1 𝑟 𝑖 (𝑣 𝑖 2 − 𝑤 𝑖 2 ) 2 𝑑 + 1 2𝑟 𝑖 𝑣 𝑖 4 + 𝑤 𝑖 𝑑 ⋅ 𝑖 𝑣 𝑖 2 + 𝑖 𝑤 𝑖 𝑟 2 has ≥0 coefficients
𝛾 is a strict lower bound on the POP ⇔
∃𝑟∈ℕ s.t. 𝑓 𝛾 𝑣 2 − 𝑤 2 − 1 𝑟 𝑖 (𝑣 𝑖 2 − 𝑤 𝑖 2 ) 2 𝑑 + 1 2𝑟 𝑖 𝑣 𝑖 4 + 𝑤 𝑖 𝑑 ⋅ 𝑖 𝑣 𝑖 2 + 𝑖 𝑤 𝑖 𝑟 2 has ≥0 coefficients
𝛾 is a lower bound on the POP ⇒ 𝑓 𝛾 is pd
Result by Polya:
𝑓 even and pd ⇒∃ 𝑟∈ℕ such that 𝑓 𝑧 ⋅ 𝑖 𝑧 𝑖 2 𝑟 has nonnegative coefficients.
Make 𝑓 𝛾 (𝑧) even by considering 𝒇 𝜸 𝒗 𝟐 − 𝒘 𝟐 . We lose positive definiteness of 𝑓 𝛾 with this transformation.
Add the positive definite term 𝟏 𝟐𝒓 𝒊 𝒗 𝒊 𝟒 + 𝒘 𝒊 𝟒 ) to 𝑓 𝛾 ( 𝑣 2 − 𝑤 2 ) to make it positive definite. Apply Polya’s result.
The term − 𝟏 𝒓 𝒊 (𝒗 𝒊 𝟐 − 𝒘 𝒊 𝟐 ) 𝟐 𝒅 ensures that the converse holds as well.

20. An optimization free method for lower bounds (3/3)
How to use this method in practice?
Given a polynomial optimization problem: min 𝑝(𝑥) , s.t. 𝑔 𝑖 𝑥 ≥0,
𝑅 such that 𝑥 𝑔 𝑖 𝑥 ≥0}⊆𝐵(0,𝑅),
an accuracy 𝝐 for the optimal value of the POP 𝑝 ∗
Expand and check nonnegativity of the coefficients of the polynomial
𝑓 𝛾 𝑣 2 − 𝑤 2 − 1 𝑟 𝑖 (𝑣 𝑖 2 − 𝑤 𝑖 2 ) 2 𝑑 + 1 2𝑟 𝑖 𝑣 𝑖 4 + 𝑤 𝑖 𝑑 ⋅ 𝑖 𝑣 𝑖 2 + 𝑖 𝑤 𝑖 𝑟 2
Have we reached accuracy 𝜖 in bisection?
Start bisection on 𝛾 by taking 𝛾= 𝑈 0 + 𝐿 0 2
𝑟=1
NB: 𝐿 0 ≤ 𝑝 ∗ ≤ 𝑈 0
No: keep doing bisection
Yes …
𝑟=2

21. Part II: Writing a polynomial as a difference of two convex polynomials

22. Difference of convex decomposition: Applications
Can be used for difference of convex programs, i.e., problems of the form: min 𝑓 0 (𝑥) 𝑠.𝑡. 𝑓 𝑖 𝑥 ≤0 where 𝑓 𝑖 𝑥 ≔ 𝑔 𝑖 𝑥 − ℎ 𝑖 𝑥 , 𝑔 𝑖 , ℎ 𝑖 convex.
Applications: Machine Learning (Sparse PCA, Kernel selection, feature selection in SVM)
Studied for quadratics, polynomials nice to study question computationally
Hiriart-Urruty, 1985
Tuy, 1995

23. Difference of Convex (dc) decomposition
Difference of convex (dc) decomposition: given a polynomial 𝑓, find 𝑔 and ℎ such that
𝒇=𝒈−𝒉,
where 𝑔,ℎ convex polynomials.
Questions:
Does such a decomposition always exist?
Can I obtain such a decomposition efficiently?
Is this decomposition unique?

24. Existence of dc decomposition (1/5)
Recall: Theorem: Any polynomial can be written as the difference of two dsos-convex polynomials. Corollary: Any polynomial can be written as the difference of two sdsos-convex/sos-convex/convex polynomials.
𝑦 𝑇 𝐻 𝑓 𝑥 𝑦 sos
SDP
SOS-convexity
𝑦 𝑇 𝐻 𝑝 𝑥 𝑦 sdsos
SOCP
SDSOS-convexity
𝑦 𝑇 𝐻 𝑝 𝑥 𝑦 dsos LP
DSOS-convexity
𝑓(𝑥) convex
𝑦 𝑇 𝐻 𝑓 𝑥 𝑦≥0,
∀𝑥,𝑦∈ ℝ 𝑛 ⇐ ⇔ ⇐

25. Existence of dc decomposition (2/5)
Lemma: Let 𝐾 be a full dimensional cone in a vector space 𝐸. Then any 𝑣∈ 𝐸 can be written as 𝑣= 𝑘 1 − 𝑘 2 , 𝑘 1 , 𝑘 2 ∈𝐾. Proof sketch:
=:𝑘′
∃ 𝛼<1 such that 1−𝛼 𝑣+𝛼𝑘∈𝐾 E K
⇔𝑣= 1 1−𝛼 𝑘 ′ − 𝛼 1−𝛼 𝑘
To change 𝒌 𝒌′ 𝒗
𝑘 1 ∈𝐾
𝑘 2 ∈𝐾

26. Existence of dc decomposition (3/5)
Here, 𝐸={polynomials of degree d, in n variables}, 𝐾={dsos-convex polynomials of degree d and in n variables }.
Remains to show that 𝐾 is full dimensional: the polynomial in the interior is constructed inductively on 𝑛 with appropriately scaled coefficients.
This shows that a decomposition can be obtained efficiently:
𝒇=𝒈−𝒉,
𝒈,𝒉 s/d/sos-convex
solving
is an LP/SOCP/SDP.

27. Uniqueness of dc decomposition
Dc decomposition: given a polynomial 𝑓, find convex polynomials 𝑔 and ℎ such that 𝒇=𝒈−𝒉.
Questions:
Does such a decomposition always exist?
Can I obtain such a decomposition efficiently?
Is this decomposition unique? 
Yes 
Through s/d/sos-convexity
Alternative decompositions
𝑓 𝑥 = 𝑔 𝑥 +𝑝 𝑥 − ℎ 𝑥 +𝑝 𝑥
𝑝(𝑥) convex
Initial decomposition
x𝑓 𝑥 =𝑔 𝑥 −ℎ(𝑥)
“Best decomposition?”

28. Convex-Concave Procedure (CCP)
Heuristic for minimizing DC programming problems.
Idea:
Input
𝑘≔0
x 𝑥 0 , initial point
𝑓 𝑖 = 𝑔 𝑖 − ℎ 𝑖 , 𝑖=0,…,𝑚
Convexify by linearizing 𝒉
x 𝒇 𝒊 𝒌 𝒙 = 𝑔 𝑖 𝑥 −( ℎ 𝑖 𝑥 𝑘 +𝛻 ℎ 𝑖 𝑥 𝑘 𝑇 𝑥− 𝑥 𝑘 )
Solve convex subproblem
Take 𝑥 𝑘+1 to be the solution of
min 𝑓 0 𝑘 𝑥
𝑠.𝑡. 𝑓 𝑖 𝑘 𝑥 ≤0, 𝑖=1,…,𝑚
convex
convex
affine
𝑘≔𝑘+1
𝒇 𝒊 𝒌 𝒙
𝒇 𝒊 (𝒙)

29. Convex-Concave Procedure (CCP)
Toy example: min 𝑥 𝑓 𝑥 , where 𝑓 𝑥 ≔𝑔 𝑥 −ℎ(𝑥)
Convexify 𝑓 𝑥 to obtain 𝑓 0 (𝑥)
Initial point: 𝑥 0 =2
Minimize 𝑓 0 (𝑥) and obtain 𝑥 1
Reiterate
𝑥 ∞
𝑥 3
𝑥 4
𝑥 2
𝑥 1
𝑥 0
𝑥 0

30. Picking the “best” decomposition for CCP
Algorithm
Linearize 𝒉 𝒙 around a point 𝑥 𝑘 to obtain convexified version of 𝒇(𝒙)
Algorithm
Linearize 𝒉 𝒙 around a point 𝑥 𝑘 to obtain convexified version of 𝒇(𝒙)
Algorithm
Linearize 𝒉 𝒙 around a point 𝑥 𝑘 to obtain convexified version of 𝒇(𝒙)
Idea
Pick ℎ 𝑥 such that it is as close as possible to affine around 𝑥 𝑘
Idea
Pick ℎ 𝑥 such that it is as close as possible to affine around 𝑥 𝑘
Mathematical translation
Minimize curvature of ℎ

31. Undominated decompositions (1/2)
Definition: g ,ℎ≔𝑔−f is an undominated decomposition of 𝑓 if no other decomposition of 𝑓 can be obtained by subtracting a (nonaffine) convex function from 𝑔.
𝒈 𝒙 = 𝒙 𝟒 + 𝒙 𝟐 ,
𝒉 𝒙 =𝟒 𝒙 𝟐 +𝟐𝒙−𝟐
Convexify around 𝑥 0 =2 to get 𝒇 𝟎 𝒙
𝒇 𝒙 = 𝒙 𝟒 −𝟑 𝒙 𝟐 +𝟐𝒙−𝟐
Cannot substract something convex from g and get something convex again.
DOMINATED BY
𝒈 ′ 𝒙 = 𝒙 𝟒 ,
𝒉 ′ 𝒙 =𝟑 𝒙 𝟐 +𝟐𝒙−𝟐
Convexify around 𝑥 0 =2 to get 𝒇 𝟎′ 𝒙
If 𝒈′ dominates 𝒈 then the next iterate in CCP obtained using 𝒈 ′ always beats the one obtained using 𝒈.

32. Undominated decompositions (2/2)
Theorem: Given a polynomial 𝑓, consider min 1 𝐴 𝑛 𝑆 𝑛−1 𝑇𝑟 𝐻 𝑔 𝑑𝜎 , (where 𝐴 𝑛 = 2 𝜋 𝑛/2 Γ(𝑛/2) ) s.t. 𝑓=𝑔−ℎ, 𝑔 convex, ℎ convex Any optimal solution is an undominated dcd of 𝑓 (and an optimal solution always exists). Theorem: If 𝑓 has degree 4, it is strongly NP-hard to solve (⋆). Idea: Replace 𝑓=𝑔−ℎ, 𝑔, ℎ convex by 𝑓=𝑔−ℎ, 𝑔,ℎ sos-convex.
(⋆)
𝑔,ℎ

33. Comparing different decompositions (1/2)
Solving the problem: min 𝐵= 𝑥 𝑥 ≤𝑅} 𝑓 0 , where 𝑓 0 has 𝑛=8 and 𝑑=4.
Decompose 𝑓 0 , run CCP for 4 minutes and compare objective value.
Feasibility
Undominated
min 𝑔,ℎ 1 𝐴 𝑛 𝑆 𝑛−1 𝑇𝑟 𝐻 𝑔 𝑑𝜎
𝑠.𝑡. 𝑓 0 =𝑔−ℎ
𝑔,ℎ sos-convex
min g,h 0
s.t. 𝑓 0 =𝑔−ℎ
𝑔,ℎ sos-convex

34. Comparing different decompositions (2/2)
Average over 30 instances
Solver: Mosek
Computer: 8Gb RAM, 2.40GHz processor
Feasibility
Undominated
Conclusion: Performance of CCP strongly affected by initial decomposition.

35. Main messages (1/2)
Computing lower bounds on POPs has many applications.
Traditionally, one can obtain lower bounds on POPs using sos polynomials.
Here, we presented two different methods to not use sos polynomials.
Replace sos polynomials by dsos/sdsos polynomials
Improve on these via sequences of LPs and SOCPs
Use a new optimization-free certificate to certify lower bounds on POPs
(only uses multiplication and coefficient checking)

36. Main messages (2/2)
Decomposing a polynomial into the difference of two convex polynomials has long been a problem of interest.
A decomposition can be obtained efficiently using sos-convexity and SDP
We can also obtain decompositions using LP and SOCP.
We show that the decomposition we pick impacts performance of the CCP algorithm, a heuristic to solve DC programs.

37. Thank you for listening
Questions?
Want to learn more?

38. Method 1: Cholesky change of basis (1/3)
dd in the “right basis”
psd but not dd
Goal: iteratively improve on basis

39. Method 1: Cholesky change of basis (2/3)
Sos problem
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾 SOS
Initialize
max 𝛾 𝛾
s.t. 𝑝 𝑥 −𝛾=𝒛 𝒙 𝑻 𝑸𝒛 𝒙 ,
𝑸 𝒅𝒅/𝒔𝒅𝒅
Step 1
Replace:
𝑈 1 =𝑐ℎ𝑜𝑙( 𝑄 ∗ )
Step 1
Replace:
𝑈 𝑘 =𝑐ℎ𝑜𝑙( 𝑈 𝑘−1 𝑇 𝑄 ∗ 𝑈 𝑘−1 )
Step 2
max 𝛾 𝛾
s.t. 𝑝 𝑥 −𝛾=𝒛 𝒙 𝑻 𝑼 𝟏 𝑻 𝑸 𝑼 𝟏 𝒛 𝒙 ,
𝑸 𝒅𝒅/𝒔𝒅𝒅
Step 2
max 𝛾 𝛾
𝑠.𝑡. 𝑝 𝑥 −𝛾=𝒛 𝒙 𝑻 𝑼 𝒌 𝑻 𝑸 𝑼 𝒌 𝒛 𝒙 ,
𝑸 𝒅𝒅/𝒔𝒅𝒅
New basis
𝒌≔𝒌+𝟏
One iteration of this method on a parametric family of polynomials:
𝑝 𝑥 1 , 𝑥 2 =2 𝑥 𝑥 2 4 +𝑎 𝑥 1 3 𝑥 2 +(1−𝑎) 𝑥 1 2 𝑥 2 2 +𝑏 𝑥 1 𝑥 2 3

40. Method 1: Cholesky change of basis (3/3)
Example:
min 𝑥 𝑝(𝑥) , where 𝑝 is in 4 variables and has degree 4
Lower bound on optimal value

41. Method 2: Column generation (3/4)
PRIMAL
DUAL
A general SDP
max 𝑦∈ ℝ 𝑚 𝑏 𝑇 𝑦
𝑠.𝑡. 𝐶− 𝑖=1 𝑚 𝑦 𝑖 𝐴 𝑖 ≽0
𝑺𝑫𝑷
Dual of SDP
min 𝑋∈ 𝑆 𝑛 𝑡𝑟(𝐶𝑋)
𝑠.𝑡. 𝑡𝑟 𝐴 𝑖 𝑋 = 𝑏 𝑖
𝑋≽0 𝑳𝑷
𝑺𝑫 𝑷 ∗
𝑳 𝑷 ∗
LP obtained with inner approximation of PSD by DD
Dual of LP
min 𝑋∈ 𝑆 𝑛 𝑡𝑟(𝐶𝑋)
𝑠.𝑡. 𝑡𝑟 𝐴 𝑖 𝑋 = 𝑏 𝑖
𝑣 𝑖 𝑇 𝑋 𝑣 𝑖 ≥0
max 𝑦∈ ℝ 𝑚 𝑏 𝑇 𝑦
𝑠.𝑡. 𝐶− 𝑖=1 𝑚 𝑦 𝑖 𝐴 𝑖 = 𝛼 𝑖 𝑣 𝑖 𝑣 𝑖 𝑇
𝛼 𝑖 ≥0
Pick 𝒗 s.t. 𝒗 𝑻 𝑿𝒗<𝟎.