Transcript Download Here

Iterative LP and SOCP-based
approximations to semidefinite and
sum of squares programs
Georgina Hall
Princeton University
Joint work with:
Amir Ali Ahmadi (Princeton University)
Sanjeeb Dash (IBM)
IOS 2016
Semidefinite programming: definition
• A semidefinite program is an optimization problem of the form:
min
𝑇𝑟(𝐶𝑋)
𝑛×𝑛
X∈S
s.t. 𝑇𝑟 𝐴𝑖 𝑋 = 𝑏𝑖 , 𝑖 = 1, … , 𝑚
𝑋≽0
Problem data: 𝐶, 𝐴𝑖 ∈ 𝑆 𝑛×𝑛 , 𝑏𝑖 ∈ ℝ.
IOS 2016
Semidefinite programming: two applications
Nonnegativity of a polynomial 𝒑 of degree 𝟐𝒅
Copositivity of a square matrix 𝑴
𝑝(𝑥) nonnegative
𝑀 copositive ⇔ 𝑥 𝑇 𝑀𝑥 ≥ 0 ∀𝑥 ≥ 0
𝑝 𝑥1 , … , 𝑥𝑛 = 𝑧 𝑥 𝑇 𝑄𝑧 𝑥 , 𝑄 ≽ 0
(𝑧 𝑥 : monomial vector of degree 𝑑)
𝑀 =𝑃+𝑁
with 𝑃 ≽ 0, 𝑁 ≥ 0
min 𝑝(𝑥)
⇔
𝑥∈ℝ𝑛
max 𝛾
𝛾
⇐
s.t. 𝑝 𝑥 − 𝛾 ≥ 0, ∀𝑥
max 𝛾
𝛾
s.t. 𝑝 𝑥 − 𝛾 = 𝑧 𝑥 𝑇 𝑄𝑧 𝑥 , 𝑄 ≽ 0
Stability number of a graph
𝛼𝐺 = min 𝜆
s.t. 𝜆 𝐼 + 𝐴 − 𝐽 copositive
⇒
Polynomial optimization
min 𝜆
s.t. 𝜆 𝐼 + 𝐴 − 𝐽 − N ≽ 0, 𝑁 ≥ 0
(De Klerk, Pasechnik)
IOS 2016
Semidefinite programming: pros and cons
• +: high-quality bounds for non convex problems.
• - : can take too long to solve if psd constraint is too big.
Polynomial optimization
Stability number of a graph
𝑝 𝑥 = 𝑧 𝑥 𝑇 𝑄𝑧 𝑥 , 𝑄 ≽ 0,
(𝑝 degree 𝑑 and in 𝑛 vars)
𝑄 : size
𝑛+𝑑
𝑑
×
𝑛+𝑑
𝑑
𝑃: = 𝜆 𝐼 + 𝐴 − 𝐽 − 𝑁 ≽ 0
𝑃 : size 𝑛 × 𝑛
(𝑛: # of nodes in the graph)
• Typical laptop cannot minimize a degree-4 polynomial in more than a
couple dozen variables. (PC: 3.4 GHz, 16 Gb RAM)
IOS 2016
LP and SOCP-based inner approximations (1/2)
Problem
Idea
Candidate (Ahmadi, Majumdar)
SDPs do not scale well
Replace psd cone by LP or SOCP
representable cones
LP: A dd
SOCP: 𝐴 sdd
• A is diagonally dominant (dd) if
𝑎𝑖𝑖 ≥
𝑗≠𝑖 |𝑎𝑖𝑗 | , ∀𝑖.
• A is scaled diagonally dominant (sdd) if ∃ a diagonal matrix 𝐷 > 0
s. t. 𝐷𝐴𝐷 is dd.
𝐷𝐷𝑛 ⊆ 𝑆𝐷𝐷𝑛 ⊆ 𝑃𝑆𝐷𝑛
IOS 2016
LP and SOCP-based inner approximations (2/2)
Replacing psd cone by dd/sdd cones:
• +: fast bounds
• - : not always as good quality (compared to SDP)
Iteratively construct a sequence of improving LP/SOCP-based cones
Initialization: Start with the DD/SDD cone
Method 1: Cholesky change of basis
Method 2: Column Generation
IOS 2016
Method 1:Cholesky change of basis (1/4)
dd in the “right basis”
psd but not dd
Goal: iteratively improve on basis 𝑧 𝑥 .
IOS 2016
Method 1: Cholesky change of basis (2/4)
Initialize
Take 𝑈0 = 𝐼
One iteration of this method
on 𝐼 + 𝑥𝐴 + 𝑦𝐵,
𝐴, 𝐵 fixed 10 × 10,
generated randomly
Step 1
Step 2
Solve: min 𝑇𝑟(𝐶𝑋)
s.t. 𝑇𝑟 𝐴𝑖 𝑋 = 𝑏𝑖 ,
𝑋 = 𝑈𝑘𝑇 𝑄𝑈𝑘 , 𝑄 dd/sdd
Replace:
𝑈𝑘+1 = 𝑐ℎ𝑜𝑙(𝑋 ∗ )
IOS 2016
Method 1: Cholesky change of basis (3/4)
• Example 1: minimizing a degree-4 polynomial in 4 variables
IOS 2016
Method 1: Cholesky change of basis (4/4)
• Example 2: stability number of the Petersen graph
100 instances of an Erdös-Rényi graph with 𝑛, 𝑝 = (20,0.5)
IOS 2016
Method 2: Column generation
Polytope 𝑃
Facet description
𝑃 = 𝑥 𝐴𝑥 ≤ 𝑏},
𝑎1𝑇
where 𝐴 = ⋮
𝑇
𝑎𝑚
,𝑏 ∈ℝ
𝑦𝑗
𝑎𝑖
𝑃
𝑚
𝑃={
𝑎𝑗
𝑖 𝛼𝑖 𝑦𝑖
| 𝛼𝑖 ≥ 0 ,
DD cone
Facet
description
𝐷𝐷𝑛 = 𝑀 𝑀𝑖𝑖 ≥
𝑣𝑖 𝛼𝑖
+
𝐷𝐷𝑛 =
Extreme ray
description
𝑦𝑖
Corner description
𝑖 𝛼𝑖
𝑃
= 1}
SDD cone
𝑗≠𝑖 |𝑀𝑖𝑗 |, ∀𝑖
}
𝑆𝐷𝐷𝑛 = 𝑀 ∃ diagonal 𝐷 > 0 s.t. 𝐷𝑀𝐷 dd}
𝑣𝑖T
𝑉𝑖
| 𝛼𝑖 ≥ 0
𝑖
𝑣𝑖 has at most two nonzero components = ±1
𝑉𝐷𝐷 : = {𝑣𝑖 }
𝑆𝐷𝐷𝑛 =
𝑖
Λ𝑖
+ +
+ +
𝑉𝑖𝑇
| Λ𝑖 ≽ 0
𝑉𝑖 is all zeros except for one component in each row=1
𝑉𝑆𝐷𝐷 : = {𝑉𝑖 }
IOS 2016
Method 2: Column Generation
• Main idea: add new atoms to the 𝐷𝐷𝑛 /𝑆𝐷𝐷𝑛 cones.
Initial SDP
Innerapproximate
approximatewith
with𝐒𝑫𝑫
𝐷𝐷𝑛𝒏
Inner
max𝑚 𝑏𝑇 𝑦
max𝑚 𝑏𝑇 𝑦
𝑦∈ℝ
𝑠. 𝑡. 𝐶 −
𝑦𝑖 𝐴𝑖 ≽ 0
𝑠. 𝑡.
𝑡. 𝐶𝐶 −
−
𝑠.
𝑖=1
𝐴𝑖𝑖 =
=
𝑦𝑦𝑖𝑖𝐴
Find a new “atom”
min 𝑡𝑟(𝐶𝑋)
Pick 𝑉 ∈𝑇ℝ𝑛×2
Pick 𝑣 s.t.𝑻 𝑣 𝑋𝑣 < 0.
s.t. 𝑽 𝑿𝑽 𝟎.
𝑋∈𝑆 𝑛
𝑦∈ℝ
𝑚
𝑚
𝑚
Take the dual
𝛼𝒊𝑖 𝚲
𝑣𝑖𝒊𝑣𝑽𝑖𝑇𝑻𝒊
𝑽
𝑠. 𝑡. 𝑡𝑟 𝐴𝑖 𝑋 = 𝑏𝑖
𝑣𝑻𝒊𝑖𝑇𝑿𝑽
𝑋𝑣𝑖𝒊 ≥
𝑽
≽0
𝟎
𝑖=1
𝑖=1
𝚲𝛼𝒊 𝑖≽≥𝟎,0,𝑽𝑣𝒊𝑖 ∈∈ 𝑽𝑉𝑺𝑫𝑫
𝐷𝐷
𝑷𝑺𝑫 =
Add𝑉𝑣 to
toprimal
primalSDD
DD inner
Add
innerapprox.
approx.
𝑷𝑺𝑫∗
max𝑚 𝑏𝑇 𝑦
𝑫𝑫∗
𝑚
𝑚
𝑫𝑫+1CG
𝑫𝑫
−
𝑠.𝑠.𝑡.𝑡. 𝐶𝐶 −
𝑦∈ℝ
=
𝑦𝑦𝑖𝑖𝐴𝐴𝑖𝑖 =
𝑖=1
𝑖=1
𝑣𝑖𝒊𝑣
𝑽𝛼𝒊𝑖𝚲
𝑽𝑖𝑇𝑻𝒊 + 𝜶𝒗𝒗
𝑽𝚲𝐕𝑻𝐓
𝚲𝛼𝒊 𝑖≽≥𝟎,0,𝑽𝑣𝒊𝑖 ∈∈ 𝑽𝑉𝑺𝑫𝑫
𝐷𝐷 , ,𝜶𝚲≥≽𝟎𝟎
IOS 2016
Five iterations of column
generation for 𝐼 + 𝑥𝐴 + 𝑦𝐵,
𝐴, 𝐵 fixed 10 × 10, generated
randomly
𝑦
Method 2: Column Generation
𝑥
• Column generation method: adds atoms to the initial cones
• Cholesky change of basis method: “rotates” existing atoms
𝑣𝑖 → 𝑈 𝑇 𝑣𝑖 , where 𝑈 = 𝑐ℎ𝑜𝑙(𝑋 ∗ )
IOS 2016
Method 2: Column Generation (3/3)
• Example 1: stability number of the Petersen graph
Upper bound on the stability
number through SDP:
min 𝜆
s.t. 𝜆 𝐼 + 𝐴 − 𝐽 − N ≽ 0, 𝑁 ≥ 0
Apply Column Generation
technique to this SDP
Petersen Graph
IOS 2016
Method 2: Column Generation (3/3)
• Example 2: minimizing a degree-4 polynomial
𝒏 = 𝟏𝟓
𝒏 = 𝟐𝟎
𝒏 = 𝟐𝟓
𝒏 = 𝟑𝟎
𝒏 = 𝟒𝟎
bd
t(s)
bd
t(s)
bd
t(s)
bd
t(s)
bd
t(s)
DSOS
-10.96
0.38
-18.01
0.74
-26.45
15.51
-36.52
7.88
-62.30
10.68
SDSOS
-10.43
0.53
-17.33
1.06
-25.79
8.72
-36.04
5.65
-61.25
18.66
𝐷𝑆𝑂𝑆𝑘
-5.57
31.19
-9.02
471.39
-20.08
600
-32.28
600
-35.14
600
SOS
-3.26
5.60
-3.58
82.22
-3.71
1068.66
NA
NA
NA
NA
𝒏 = 𝟏𝟓
𝒏 = 𝟐𝟎
𝒏 = 𝟐𝟓
𝒏 = 𝟑𝟎
𝒏 = 𝟒𝟎
bd
t(s)
bd
t(s)
bd
t(s)
bd
t(s)
bd
t(s)
𝐷𝑆𝑂𝑆𝑘
-6.20
10.96
-12.38
70.70
-20.08
508.63
NA
NA
NA
NA
1SDSOS
-8.97
14.39
-15.29
82.30
-23.14
538.54
NA
NA
NA
NA
IOS 2016
Main messages
• Can improve on initial DD and SDD inner approximations to the PSD cone.
• This is done with LPs and SOCPs and iteratively.
• We presented two methods for this: cholesky change of basis and column
generation.
Thank you for listening
Questions?
Want to learn more?
http://scholar.princeton.edu/ghall/home
17