1. A Combinatorial, Primal-Dual Approach to Semidefinite Programs
Satyen Kale
Microsoft Research
Joint work with
Sanjeev Arora
Princeton University

2. The Ubiquity of Semidefinite Programming
Max Cut [GW’95]
Balanced Partitioning [ARV’04]
Control Theory
dx(t)/dt = Ax(t)
Graph Coloring
[KMS’98, ACC’06]
SDP
(a Ç :b) Æ (:a Ç c) Æ (a Ç b) Æ (:c Ç b)
Constraint Satisfaction [ACMM’05]

3. Primal-Dual algorithms
Algorithms for SDP
Combinatorial,
Primal-Dual algorithms
MISSING
Interior point methods [NN’90, A’95]:
O(√m(n3 + m)) time
Ellipsoid method [GLS’81]:
O(n8) iterations
A1 ² X ¸ b1 
Am ² X ¸ bm
i wi(Ai ² X – bi) ¸ 0
Lagrangian Relaxation [KL’95], [AHK’05]:
Reduction to eigenvectors
poly(1/) dependence on , limits
applicability (e.g. Sparsest Cut)

4. Primal-Dual algorithms for LP: Multicommodity Flows
Unit capacities
Minimize colors
Objective: Maximize total flow,
while respecting edge capacities

5. Primal-Dual algorithms for LP: Multicommodity Flows
1 +  1
Edge weights we = 1
Repeat until some e reaches capacity:
Find shortest path p
Route 2/log(m) flow on p
Update we for all e 2 p as
we à we ¢ (1 + )
Output flow. 1
f_e explanation, shortest path over all commodities
Thm [GK’98]: Stops in Õ(m) rounds with
(1 – 2)¢OPT flow. Total running time is Õ(m2).

6. Primal-Dual algorithms for LP: Multicommodity Flows
Combinatorial
Multiplicative Weights Update Rule
Edge weights we = 1
Repeat until some e reaches capacity:
Find shortest path p
Route 2/log(m) flow on p
Update we for all e 2 p as
we à we ¢ (1 + )
Output flow.
f_e explanation, shortest path over all commodities
Thm [GK’98]: Stops in Õ(m) rounds with
(1 – 2)¢OPT flow. Total running time is Õ(m2).

7. Starting point for our work: Analogous primal-dual algorithms for SDP?
Difficulties:
Positive semidefiniteness hard to maintain
Rounding algorithms exploit geometric structure (e.g. negative-type metrics)
Matrix operations inefficient to implement
Our work: a generic scheme that yields fast, combinatorial, primal-dual algorithms for various optimization problems using SDP

8. Our Results: Primal-Dual algorithms
Problem
Previous best:
O(√log n) apx
Our alg.:
O(log n) apx
Undirected
Sparsest Cut
Õ(n2)
Õ(m + n1.5)
Balanced Sep.
Directed
Õ(n4.5)
Õ(m1.5 + n2+)
Õ(m1.5)
Min UnCut
Õ(n3)
---
Min 2CNF Deletion
Õ(nm1.5 + n3)
[AHK’04]
[AHK’04]

9. Our Results: Primal-Dual algorithms
Problem
Previous best:
O(√log n) apx
Our alg.:
O(log n) apx
Undirected
Sparsest Cut
Õ(n2)
Õ(m + n1.5)
Balanced Sep.
Directed
Õ(n4.5)
Õ(m1.5 + n2+)
Õ(m1.5)
Min UnCut
Õ(n3)
---
Min 2CNF Deletion
Õ(nm1.5 + n3)
[AHK’04]
Cannot be achieved by LPs even!
[AHK’04]

10. Our Results: Near-linear time algorithm for Max Cut
Õ(n + m) time algorithm to approximate Max Cut SDP to (1 + o(1)) factor
Previous best: Õ(n2) time algorithm by Klein, Lu ‘95

11. Prototype MW for LPs: Winnow [L’88]
a1 ¢ x ¸
a2 ¢ x ¸ 
am ¢ x ¸
x ¸ 0
i xi = 1 -  
Thm: Finds x in O(2log(n)/2)
iterations.
 = maxij |aij|
MW for LP:
Init: x = (1/n, …, 1/n)
Update:
xj = xj ¢ exp(- aij)/
 = norm. factor x
Oracle
Find i s.t. ai ¢ x < - ai
Convex comb. of constraints allowed:
i yi ai ¢ x < -, where yi ¸ 0, i yi = 1. Hence Primal-Dual.
MW algorithm: boosting, hard-core sets, zero-sum games, flows, portfolio, …
Analysis uses  as potential fn.

12. Prototype Matrix MW for SDPs
A1 ² X ¸
A2 ² X ¸ 
Am ² X ¸
X º 0
Tr(X) = 1 -  
Thm: Finds X in O(2log(n)/2)
iterations.
 = maxi kAik
Matrix MW for SDP:
Init: X = I/n
Update:
X = exp(- ts=1 Ais)/
 = norm. factor X
Oracle
Find it s.t. Ait ² X < -
Ait
Convex comb. of constraints allowed:
i yi Ai²X < -, where yi ¸ 0, i yi = 1. Hence Primal-Dual.
Analysis uses  as potential fn.

13. The Matrix Exponential
exp(A) = I + A + A2/2! + A3/3! + …
Always PSD: exp(A) º 0
Tricky beast:
exp(A+B) = exp(A) exp(B)
Computation:
O(n3) time
Usually, can use J-L lemma
Only need exp(A)v products: Õ(m) time
Golden-Thompson inequality:
Tr(exp(A+B)) · Tr(exp(A)exp(B))
Matrix MW for SDP:
Init: X = I/n
Update:
X = exp(- ts=1 Ais)/
 = norm. factor

14. Approximation via SDP Relaxations
Input
Reduction
0-1
Quadratic
Program
Relaxation
Rounding
Algorithm
Primal-Dual SDP
SDP Opt v1 vn v2
SDP:
Vector
vars
SDP Solver

15. Primal-Dual SDP Framework
Input
Reduction
0-1
Quadratic
Program
Rounding
Algorithm
Relaxation
Primal-Dual SDP v1 vn v2
y1,…, ym
Matrix MW
SDP:
Vector
vars X

16. Approximating Balanced Separator
G = (V, E)
Cut (S, S’) is c-balanced if
|S|, |S’| ¸ cn
Min c-Balanced Separator: c-balanced cut of min capacity
Numerous applications:
Divide-and-conquer algorithms
Markov chains
Geometric embeddings
Clustering
Layout problems … S S’

17. SDP for Balanced Separator
G = (V, E)
¼kvi – vjk2 = 0 if i, j on same side,
= 1 otherwise
SDP:
min i,j 2 E ¼kvi – vjk2
8i: kvik2 = 1
8ijk: kvi–vjk2 + kvj–vkk2 ¸ kvi–vkk2
i, j ¼kvi – vjk2 ¸ c(1-c)n2
vi = -1
vi = 1

18. 8ijk: kvi–vjk2 + kvj–vkk2 ¸ kvi–vkk2
Implementing Oracle
Primal:
min i,j 2 E ¼kvi – vjk2
8i: kvik2 = 1
8ijk: kvi–vjk2 + kvj–vkk2 ¸ kvi–vkk2
i, j ¼kvi – vjk2 ¸ c(1-c)n2
Oracle: given X (thus, vi), :
check first and third constraints
 inequalities: dual vars , multicommodity flow s.t.
total flow from any node is · d
no capacity is exceeded
i,j fijkvi – vjk2 ¸ 
Bounds eigenvalues via demand graph Laplacian
Thm: Given vi, :
Max-flow ) desired flow or cut of
value O(log(n)¢).
Multicommodity flow ) desired flow
or cut of value O(√log(n)¢).
fij = total flow
between i, j

19. Conclusions and Future Work
Matrix MW algorithm: more applications in
Solving SDPs: e.g. Min Linear Arrangement
Quantum algorithms: density matrix is a central concept
Learning: e.g. online variance minimization [WK COLT’06], online PCA [WK NIPS’06]
Other applications?
Linear time algorithm for Sparsest Cut?
Our algorithm runs in Õ(m + n1.5) time for an O(log n) approximation

20. Thank you!