Transcript Submodular Set Function Maximization: A Mini

Submodular Set Function
Maximization
A Mini-Survey
Chandra Chekuri
Univ. of Illinois, Urbana-Champaign
Submodular Set Functions
A function f : 2N  R is submodular if
f(A) + f(B) ≥ f(AB) + f(AB) for all A,B  N
Equivalently,
f(A+j) – f(A) ≥ f(A+i+j) – f(A+i)
for all A  N and i, j  N\A
Submodular Functions
• Non-negative submodular set functions
f(A) ≥ 0 for all A
• Monotone submodular set functions
f(ϕ) = 0 and f(A) ≤ f(B) for all A  B
• Symmetric submodular set functions
f(A) = f(N\A) for all A
Well-known Examples
• Cut functions in undirected graphs and hypergraphs
(symmetric non-negative)
• Cut functions in directed graphs (non-negative)
• Rank functions of matroids (monotone)
• Coverage in set systems (monotone)
• many others ...
Maximizing Submodular Set
Functions
Given f on a ground set N via a value oracle
max S  N f(S)
S satisfies some constraints
Maximizing Submodular Set
Functions
Given f on a ground set N via a value oracle
max S  N f(S)
S satisfies some constraints
Motivation:
• Many non-trivial applications (easy to miss!)
• Generalize known results for modular functions
Unconstrained Problem
max S  N f(S)
• Uninteresting for monotone f
• NP-Hard for non-negative f (Max-Cut is a special
case)
• Very hard to approximate for arbitrary f (reduction
from Set Packing)
Unconstrained Problem
[Feige-Mirrokni-Vondrak’07]
First O(1) approximation for non-negative f !
Easy O(1) algorithms, ½ for symmetric case
Non-trivial 2/5 = 0.4 approximation (slight
improvement to 0.41 [Vondrak])
Unconstrained Problem
[Feige-Mirrokni-Vondrak’07]
First O(1) approximation for non-negative f !
Easy O(1) algorithms, ½ for symmetric case
Non-trivial 2/5 = 0.4 approximation (slight
improvement to 0.41 [Vondrak])
Better than ½ requires exponential # of value queries
Open Problem: Close gap between 0.41 and ½
Unconstrained Problem
[Feige-Mirrokni-Vondrak’07]
Random set algorithm:
• pick each i in N with prob ½, let R be random set
• E[f(R)] ≥ OPT/4
• E [f(R)] ≥ OPT/2 for symmetric f
Simple Local Search:
•
•
•
•
Initialize S to best singleton
S = local optimum for adding or deleting if improvement
Output better of S and N\S
1/3 approx for non-negative f and ½ for symmetric f
Local Search Analysis
Lemma: If S is a local optimum then for any I  S or
S  I, f(I) ≤ f(S).
Proof: Say S  I and f(I) > f(S) then by submodularity
there exists i in I\S s.t f(S+i) > f(S).
Corollary: Let S* be an optimum solution and S be a
local opt. f(S  S*) ≤ f(S) and f(S  S*) ≤ f(S)
Local Search Analysis
f(S  S*) ≤ f(S) and f(S  S*) ≤ f(S)
N\S
S
S*
f(S  S*) + f(N\S) ≥ f(S*\S) + f(N) ≥ f(S*\S)
2f(S) + f(N\S) ≥ f(S*\S) + f(S*  S) ≥ f(S*)
implies max (f(S), f(N\S)) ≥ f(S*)/3
Maximizing Submodular Set
Functions with Constraints
max S  N f(S)
S satisfies some constraints
Question: what constraints?
For maximization probs, packing constraints natural
SI
I is a downward-closed: A  I, B  A implies B  I
Matroid and Knapsack
Constraints
• Combinatorial packing constraints
• I is the intersection of some p matroids on N
• Lemma: every downward-close family I on N is the
intersection of p matroids on N (for some p)
• Knapsack or matrix packing constraints
• A is a m x n non-negative matrix, b is m x 1 vector
• I = { x  {0,1}n| A x ≤ b }
• Combination of matroid and knapsack constraints
Matroid Constraints
• Uniform matroid: I = { S : |S| ≤ k }
• Partition matroid: I = { S : |S Ni| ≤ ki, 1 ≤ i ≤ h }
where N1, ..., Nh partition N, and ki are integers
• Laminar matroid: I = { S : |S  U| ≤ k(U), U in F }
for a laminar family of sets F
• Graphic matroid
Matroid polytope is integral and hence one can hope to
capture constraints via relaxation in polytope
Monotone Functions
Cardinality Constraint
max f(S) such that |S| ≤ k
• Max k-Cover problem is special case
• Greedy gives (1-1/e) approximation [NemhauserWolsey-Fisher’78]
• Unless P=NP no better approximation [Feige’98]
• Many applications, routinely used
Greedy Algorithm
1. S = 
2. While |S| < k do
• i  argmaxj fS(j)
• S  S+i
3. Output S
Greedy Analysis
• Sj : first j elements picked by Greedy
• f(S) = δ1 + δ2... +δk
• δj ≥ (OPT – f(Sj-1))/k (monotonicity and submod)
• f(S) ≥ (1-1/k)k OPT
A Different Analysis
• S = {i1, i2, ..., ik } picked by Greedy in order
• S* = {i*1, i*2, ..., i*k } an optimum solution
• Form perfect matching between S and S* s.t
elements in S  S* are matched to themselves
S
S*
A Different Analysis
• Form perfect matching between S and S*
Renumber S* s.t ij is matched to i*j
• For each j, because Greedy chose i*j instead of i*j
δj = f(Sj-1+ ij) – f(Sj-1) ≥ f(Sj-1+ i*j) – f(Sj-1) ≥ fS(i*j)
• Summing up
f(S) ≥ Σj fS(i*j) ≥ fS(S*) = f(S  S*) – f(S)
hence f(S) ≥ f(S  S*)/2 ≥ f(S*)/2 if f is monotone
Why weaker analysis?
Let M=(N, I) be any matroid on N
Solve max f(S) such that S  I
[Nemhauser-Wolsey-Fisher’78]
Theorem: Greedy is ½ approximation, and analysis is
tight even for partition matroids
Many applications, unfortunately unknown, till
recently, to approximation algorithms community!
Base Exchange Theorem
• B and B’ are distinct bases in a matroid M=(N, I)
Strong Base Exchange Theorem: There are elements
i  B\B’ and i’  B’\B such that B-i+i’ and B’-i’+i
are both bases.
Corollary: There is a perfect matching between B\B’ and
B’\B such that for each matched pair (i,i’), B-i+i’ is
a base .
Proof for Greedy
• B is the solution that Greedy outputs
• B* is an optimum solution
• B and B* are bases of M by monotonicity of f
Same argument as before works by using perfect
matching between B and B*
Multiple Matroids
max f(S) such that S is in intersection of p matroids
[Fisher-Nemhauser-Wolsey’78]
Theorem: Greedy is 1/(p+1) approximation
Generalize matching argument to match one element
of Greedy to p elements of OPT
Also works for p-systems
Non-negative functions?
max f(S) such that S is in intersection of p matroids
[Lee-Mirrokni-Nagarajan-Sviridenko’08]
Theorem: For fixed p, local search based algorithm that
achieves Ω(1/p) approximation.
[Gupta-Roth-Schoenbeck-Talwar’09]
For all p and with simple proof combining Greedy and
unconstrained algorithm. Slightly worse constants.
Knapsack Constraints
Similar ideas can be used with standard guessing large
items etc
Can we do better?
• ½ for one matroid, hardness is (1-1/e)
• 1/(p+1) for p matroids, hardness is (p/log p)
Can we do better?
[Calinescu-C-Pal-Vondrak’07]+[Vondrak’08]=[CCPV’09]
Theorem: There is a randomized (1-1/e) approximation for
maximizing a monotone f subject to a matroid constraint.
Can we do better?
[Calinescu-C-Pal-Vondrak’07]+[Vondrak’08]=[CCPV’09]
Theorem: There is a randomized (1-1/e) approximation for
maximizing a monotone f subject to a matroid constraint.
[Lee-Sviridenko-Vondrak’09]
Theorem: For fixed p≥2, there is a local-search based
1/(p+ε) approximation for intersection of p matroids
New useful insight for (two) matroid intersection
Multilinear Extension of f
Question: Is there a useful continuous relaxation of f
such that it can be optimized? And can we round it
effectively?
Multilinear Extension of f
[CCPV’07] inspired by [Ageev-Sviridenko]
For f: 2N  R+ define F:[0,1]N  R+ as
x = (x1, x2, ..., xn)  [0,1]|N|
F(x) = Expect[ f(x) ] =  S  N f(S) px(S)
=  S  N f(S) i  S xi  i  N\S (1-xi)
Multilinear Extension of f
For f: 2N  R+ define F:[0,1]N  R+ as
F(x) =  S  N f(S) i  S xi  i  N\S (1-xi)
F is smooth submodular ([Vondrak’08])
• F/xi ≥ 0 for all i (monotonicity)
• 2F/xixj ≤ 0 for all i,j (submodularity)
Optimizing F(x)
[Vondrak’08]
Theorem: For any down-monotone polytope P [0,1]n
max F(x) s.t x  P can be optimized to within a (11/e) approximation if we can do linear optimization
over P
Algorithm: Continuous-Greedy
Generic Approach
Want to solve: max f(S) s.t S  I
Relaxation:
max F(x) s.t x  P(I )
• P(I) is a polytope that captures/relaxes S  I
• Can solve to within (1-1/e) with continuous greedy
• How to round? F is a non-linear function
Rounding in Matroids
Let M=(S,I) be a matroid
• P(M) is the independent set polytope of M
• B(M) is the base polytope of M
Algorithm:
• Run continuous greedy to obtain a point x  B(M)
such that F(x) ≥ (1-1/e) OPT
• Round x to a vertex of B(M) (a base)
Rounding in Matroids
[CCPV’07]
Theorem: Given any point x in B(M), there is a
polynomial time algorithm to round x to a vertex x*
(hence a base of M) such that F(x*) ≥ F(x).
“Pipage” rounding technique of [Ageev-Sviridenko]
adapted to matroids
New Rounding Method
[C-Vondrak-Zenklusen’09]
Randomized Swap-Rounding:
• Express x = mj=1 βi Bi (convex comb. of bases)
• B = B1 , β = β1
• For k = 2 to m do
• Randomly Merge β B and βk Bk into (β + βk) B’
• Set B = B’, β = (β + βk)
• Output B
Merging two Bases
Merge B’ and B’’ into a random B that looks “half ” like
B’ and “half ” like B’’
Merging two Bases
Base ExchangeTheorem:
B’
B’’
i
B’B’’
j
B’B’’
B’-i+j and B’’-j+i are both bases
Merging two Bases
B’
i
prob ½
B’
B’’
i
B’B’’
B’’
B’B’’
i
B’B’’
j
B’B’’
B’
prob ½
B’’
j
B’B’’
j
B’B’’
Swap Rounding for Matroids
Theorem: Swap-Merging with input x in B(M) outputs
a random base B such that
1. E[f(B)] ≥ F(x) and
2. Pr[ f(B) < (1-δ) F(x)] ≤ exp(- F(x) δ2/8)
(concentration for lower tail of submod functions) and
3. For any vector a  [0,1]n , let μ = ax then
•
•
Pr[a 1B < (1-δ) μ] ≤ exp(- μ δ2/2)
Pr[a 1B > (1+δ) μ] ≤ ( e δ / (1+δ) δ ) μ
for linear functions)
(concentration
Almost like independent random rounding of x
Applications
Can handle matroid constraint plus packing constraints
x  P(M) and Ax ≤ b
• (1-1/e) approximation for submodular functions
subject to a matroid plus O(1) knapsack/packing
constraints (or many “loose” packing constraints)
• Simpler rounding and proof for “thin” spanning
trees in ATSP application ([Asadpour etal’10])
•
...
Other Tools
On-going work [C-Vondrak-Zenklusen]
• Extension of continuous greedy to handle multiple
submodular functions simultaneously
• Depending rounding via swap method for polyhedra
• matroid intersection
• matchings and b-matchings in non-bipartite graphs
• ...
Many applications of dependent randomized
rounding[Arora-Frize-Kaplan, Srinivasan’01, ....,
Asadpour etal,..]
Illustrative Application
Maximum Bipartite Flow in Networks with Adaptive
Channel Width [Azar-Madry-MoscibrodaPanigrahy-Srinivasan’09]
• Problem motivated by capacity allocation in wireless
networks
• (1-1/e) approximation via a specialized LP and
complicated analysis
• An easy O(log n) approximation
Problem Definition
θ(B,C) : threshold capacity between B and C
α(C): max flow that C desires from base stns
β(B): total capacity of B to serve clients
For each base station B, decide an operating
point τ(B) ≤ β(B)
• If τ(B) > θ(B,C) then u(B,C) capacity of
link (B,C) is 0, otherwise u(B,C) = τ(B)
• Maximize flow from base stations to clients
Base
stations
Clients
u(B,C)
α(C)
β(B)
Reduction
Base
stations
B
Copies of B one
for each τ(B)
Clients
α(C)
t
u(τ(B),C)
β(B)
N : all copies on left
f : 2N  R+ where f(S) = flow from S to t
Partition Matroid constraint: can pick only one copy for each B
Summary
• Substantial progress on submodular function
maximization problems
• Increased awareness and more applications
• (New) tools and connections: continuous greedy,
dependent rounding, local search, ...
• Several open problems still remain
Open Problems
• Unconstrained for non-neg f. Close gap bet 0.41 and ½
• max f(S) s.t S in intersection of p matroids
•
•
•
•
•
p=1 we have 1-1/e approx which is tight unless P=NP
for fixed p ≥ 2, 1/(p+ε) approx, otherwise 1/(p+1)
hardness is Ω (p/log p) for large p when f is modular
close gaps, most interesting for small p
How to round F(x) for more than 1 matroid? Don’t know
integrality gap for p=2!
• max f(x) s.t x in P(M) and Ax ≤ b, A is k-column-sparse.
Want Ω(1/k) approx. Known without matroid constraint
[Bansal-Korula-Nagarajan-Srinivisan’10]