1. Polynomial DC decompositions
Georgina Hall
Princeton, ORFE
Joint work with
Amir Ali Ahmadi
11/18/2018
INFORMS 2016

2. Difference of convex (dc) programming
Problems of the form min 𝑓 0 (𝑥) 𝑠.𝑡. 𝑓 𝑖 𝑥 ≤0,𝑖=1,…,𝑚 where 𝑓 𝑖 𝑥 ≔ 𝑔 𝑖 𝑥 − ℎ 𝑖 𝑥 , 𝑔 𝑖 , ℎ 𝑖 convex for 𝑖=0,…,𝑚.
What if such a decomposition is not given?
Studied for quadratics, polynomials nice to study question computationally
Hiriart-Urruty, 1985
Tuy, 1995

3. Difference of convex (dc) decomposition
Difference of convex (dc) decomposition: given a polynomial 𝑓, find 𝑔 and ℎ such that
𝒇=𝒈−𝒉,
where 𝑔,ℎ convex polynomials.
Questions:
Does such a decomposition always exist?
Can I obtain such a decomposition efficiently?
Is this decomposition unique?

4. Existence of dc decomposition (1/4)
A polynomial 𝑝 is a sum of squares (sos) if ∃ 𝑞 𝑖 , polynomials, s.t.
𝑝 𝑥 =∑ 𝑞 𝑖 2 𝑥 .
A polynomial 𝑝(𝑥) of degree 2d is sos if and only if ∃𝑸≽𝟎 such that
where 𝑧= 1, x 1 ,…, 𝑥 𝑛 , 𝑥 1 𝑥 2 ,…, 𝑥 𝑛 𝑑 T is the vector of monomials up to degree 𝑑.
Testing whether a polynomial is sos is a semidefinite program.
Specify sufficient condition for nonnegativity (should I add it in front of sos? Mention that this is the Gram matrix

5. Existence of dc decomposition (2/4)
Theorem: Any polynomial can be written as the difference of two sos-convex polynomials. Corollary: Any polynomial can be written as the difference of two convex polynomials.
𝑦 𝑇 𝐻 𝑓 𝑥 𝑦 sos
SOS-convexity
𝐻 𝑓 𝑥 ≽0,
∀𝑥∈ ℝ 𝑛
𝑦 𝑇 𝐻 𝑓 𝑥 𝑦≥0,
∀𝑥,𝑦∈ ℝ 𝑛
𝑓(𝑥) convex ⇔ ⇔ ⇐

6. Existence of dc decomposition (3/4)
Lemma: Let 𝐾 be a full dimensional cone in a vector space 𝐸. Then any 𝑣∈𝐸 can be written as 𝑣= 𝑘 1 − 𝑘 2 , with 𝑘 1 , 𝑘 2 ∈𝐾. Proof sketch:
=:𝑘′
∃ 𝛼<1 such that 1−𝛼 𝑣+𝛼𝑘∈𝐾 E K
⇔𝑣= 1 1−𝛼 𝑘 ′ − 𝛼 1−𝛼 𝑘
To change 𝒌 𝒌′ 𝒗
𝑘 1 ∈𝐾
𝑘 2 ∈𝐾

7. Existence of dc decomposition (4/4)
Here, 𝐸={polynomials of degree 2d, in n variables},
𝐾={sos-convex polynomials of degree 2d and in n variables }.
Remains to show that 𝐾 is full dimensional:
Also shows that a decomposition can be obtained efficiently:
In fact, we show that a decomposition can be found via LP and SOCP (not covered here).
∑ 𝒙 𝒊 𝟐 𝒅 can be shown to be in the interior of 𝐾.
𝒇=𝒈−𝒉,
𝒈,𝒉 sos-convex
solving
is an SDP.

8. Uniqueness of dc decomposition
Dc decomposition: given a polynomial 𝑓, find 𝑔 and ℎ convex polynomials such that 𝒇=𝒈−𝒉.
Questions:
Does such a decomposition always exist?
Can I obtain such a decomposition efficiently?
Is this decomposition unique? 
Yes 
Through sos-convexity
Alternative decompositions
𝑓 𝑥 = 𝑔 𝑥 +𝑝 𝑥 − ℎ 𝑥 +𝑝 𝑥
𝑝(𝑥) convex
Initial decomposition
x𝑓 𝑥 =𝑔 𝑥 −ℎ(𝑥)
“Best decomposition?”

9. Convex-Concave Procedure (CCP)
Heuristic for minimizing DC programming problems.
Idea:
Input
𝑘≔0
x 𝑥 0 , initial point
𝑓 𝑖 = 𝑔 𝑖 − ℎ 𝑖 , 𝑖=0,…,𝑚
Convexify by linearizing 𝒉
x 𝒇 𝒊 𝒌 𝒙 = 𝑔 𝑖 𝑥 −( ℎ 𝑖 𝑥 𝑘 +𝛻 ℎ 𝑖 𝑥 𝑘 𝑇 𝑥− 𝑥 𝑘 )
Solve convex subproblem
Take 𝑥 𝑘+1 to be the solution of
min 𝑓 0 𝑘 𝑥
𝑠.𝑡. 𝑓 𝑖 𝑘 𝑥 ≤0, 𝑖=1,…,𝑚
convex
convex
affine
𝑘≔𝑘+1
𝒇 𝒊 𝒌 𝒙
𝒇 𝒊 (𝒙)

10. Convex-Concave Procedure (CCP)
Toy example: min 𝑥 𝑓 𝑥 , where 𝑓 𝑥 ≔𝑔 𝑥 −ℎ(𝑥)
Convexify 𝑓 𝑥 to obtain 𝑓 0 (𝑥)
Initial point: 𝑥 0 =2
Minimize 𝑓 0 (𝑥) and obtain 𝑥 1
Reiterate
𝑥 ∞
𝑥 3
𝑥 4
𝑥 2
𝑥 1
𝑥 0
𝑥 0

11. Picking the “best” decomposition for CCP
Algorithm
Linearize 𝒉 𝒙 around a point 𝑥 𝑘 to obtain convexified version of 𝒇(𝒙)
Idea
Pick ℎ 𝑥 such that it is as close as possible to affine around 𝑥 𝑘
Mathematical translation
Minimize curvature of ℎ at 𝑥 𝑘
Worst-case curvature*
min g,h 𝜆 𝑚𝑎𝑥 ( 𝐻 ℎ 𝑥 𝑘 )
s.t. 𝑓=𝑔−ℎ
𝑔,ℎ convex
Average curvature*
min 𝑔,ℎ 𝑇𝑟 𝐻 ℎ ( 𝑥 𝑘 )
s.t. 𝑓=𝑔−ℎ,
𝑔,ℎ convex
* 𝜆 𝑚𝑎𝑥 𝐻 ℎ 𝑥 𝑘 = max 𝑦∈ 𝑆 𝑛−1 𝑦 𝑇 𝐻 ℎ 𝑥 𝑘 𝑦
* 𝑇𝑟 𝐻 ℎ 𝑥 𝑘 = 𝑦∈ 𝑆 𝑛−1 𝑦 𝑇 𝐻 ℎ 𝑥 𝑘 𝑦 𝑑𝜎

12. Undominated decompositions (1/4)
Definition: g ,ℎ≔𝑔−f is an undominated decomposition of 𝑓 if no other decomposition of 𝑓 can be obtained by subtracting a (nonaffine) convex function from 𝑔.
𝒈 𝒙 = 𝒙 𝟒 + 𝒙 𝟐 ,
𝒉 𝒙 =𝟒 𝒙 𝟐 +𝟐𝒙−𝟐
Convexify around 𝑥 0 =2 to get 𝒇 𝟎 𝒙
𝒇 𝒙 = 𝒙 𝟒 −𝟑 𝒙 𝟐 +𝟐𝒙−𝟐
Cannot substract something convex from g and get something convex again.
DOMINATED BY
𝒈 ′ 𝒙 = 𝒙 𝟒 ,
𝒉 ′ 𝒙 =𝟑 𝒙 𝟐 +𝟐𝒙−𝟐
Convexify around 𝑥 0 =2 to get 𝒇 𝟎′ 𝒙
If 𝒈′ dominates 𝒈 then the next iterate in CCP obtained using 𝒈 ′ always beats the one obtained using 𝒈.

13. Undominated decompositions (2/4)
𝐷= 0 𝑑 12 ∗ 𝑑 14 𝑑 𝑑 23 ∗ ∗ 𝑑 𝑑 34 𝑑 14 ∗ 𝑑 34 0
𝒙 𝟏 = 𝑥 𝑥 𝑥 , 𝒙 𝟐 = 𝑥 𝑥 𝑥 ,…
Incomplete distance matrix
Recover location of the 𝑚 points in ℝ 𝑑
Solve:
min 𝑥 𝑖 , 𝑥 𝑗 ∈ ℝ 𝑑 𝑖<𝑗 𝑥 𝑖 − 𝑥 𝑗 − 𝑑 𝑖𝑗 2 2
There is a realization in ℝ 𝑑 iff opt value=0.

14. Undominated decompositions (3/4)
= 𝑥 𝑖 − 𝑥 𝑗 𝑑 𝑖𝑗 4 −2 𝑑 𝑖𝑗 2 ⋅ 𝑥 𝑖 − 𝑥 𝑗 2 2
𝑔 𝑖𝑗
ℎ 𝑖𝑗
𝑔= 𝑖<𝑗 𝑔 𝑖𝑗 ,ℎ= 𝑖<𝑗 ℎ 𝑖𝑗 is an undominated dcd of the objective function.

15. Undominated decompositions (4/4)
Theorem: Given a polynomial 𝑓, consider min 1 𝐴 𝑛 𝑆 𝑛−1 𝑇𝑟 𝐻 𝑔 𝑑𝜎 , (where 𝐴 𝑛 = 2 𝜋 𝑛/2 Γ(𝑛/2) ) s.t. 𝑓=𝑔−ℎ, 𝑔 convex, ℎ convex Any optimal solution is an undominated dcd of 𝑓 (and an optimal solution always exists). Theorem: If 𝑓 has degree 4, it is strongly NP-hard to solve (⋆). Idea: Replace 𝑓=𝑔−ℎ, 𝑔, ℎ convex by 𝑓=𝑔−ℎ, 𝑔,ℎ sos-convex.
(⋆)
𝑔,ℎ

16. Comparing different decompositions (1/2)
Solving the problem: min 𝐵= 𝑥 𝑥 ≤𝑅} 𝑓 0 , where 𝑓 0 has 𝑛=8 and 𝑑=4.
Decompose 𝑓 0 , run CCP for 4 minutes and compare objective value.
Feasibility
𝝀 𝒎𝒂𝒙 𝑯 𝒉 ( 𝒙 𝟎 )
Undominated
min g,h 𝑡
s.t. 𝑓 0 =𝑔−ℎ
𝑔,ℎ sos-convex
𝑡𝐼− 𝐻 ℎ 𝑥 0 ≽0
min 𝑔,ℎ 1 𝐴 𝑛 𝑆 𝑛−1 𝑇𝑟 𝐻 𝑔 𝑑𝜎
𝑠.𝑡. 𝑓 0 =𝑔−ℎ
𝑔,ℎ sos-convex
min g,h 0
s.t. 𝑓 0 =𝑔−ℎ
𝑔,ℎ sos-convex

17. Comparing different decompositions (2/2)
Average over 30 iterations
Solver: Mosek
Computer: 8Gb RAM, 2.40GHz processor
Feasibility
𝝀 𝒎𝒂𝒙 𝑯 𝒉 𝒙 𝟎
Undominated
Conclusion: Performance of CCP strongly affected by initial decomposition.

18. Main messages
We studied the question of decomposing a polynomial into the difference of two convex polynomials.
This decomposition always exists and is not unique.
Choice of decomposition can impact performance of the CCP algorithm.
Dc decompositions can be efficiently obtained using the notion of sos-convexity (SDP)
Also possible to use LP or SOCP-based relaxations to obtain dc decompositions (not covered here).

19. Thank you for listening
Questions?
Want to learn more?
11/18/2018