Transcript 1) What should we teach in approximation algorithms courses?

What should be taught in
approximation algorithms
courses?
Guy Kortsarz, Rutgers
Camden
Advanced issues presented in many
lecture notes and books:
• Coloring a 3-colorable graph using vectors.
• Paper by Karger, Motwani and Sudan.
• Things a student needs to know:
Separation oracle for: A is PSD.
Getting a random vector in Rn.
This is done by choosing the Normal
distribution at every entry.
Given unit vector v, v . r is normal
distribution.
Things a student needs to know:
• There is a choice of vectors vi for every i V so
that so that for every (i, j) E, vi · vj  -1/2.
A student needs to know:
• S={i | r · vi }, threshold method, by now
standard.
• Sum of two normal distributions also normal.
• Two inequalities (non trivial) about the normal
distribution.
• The above can be used to find a large
independent set.
• Combined with the greedy algorithm gives about
n1/4 ratio approximation algorithm.
Advanced methods are also required
in the following topics often taught:
• The seminal result of Jain. With the
simplification of Nagarajan et. al. 2-ratio for
Steiner Network.
• The beautiful 3/2 ratio by Calinesco, Karloff
and Rabani, for Multiway Cuts: geometric
embeddings.
• Facharoenphol, Rao and Talwar, optimal
random tree embedding. With this can get
O(log n) for undirected multicut.
How to teach sparsest cut?
• Many still teach the embedding of a metric into L1 ,
with O(log n) distortion. By Lineal, London, Rabinovich.
• Advantage: relatively simple.
• The huge challenge posed by the Arora, Rao and
Vazirani result. Unweighted sparsest cut sqrt{log n}
• Teach the difficult lemma? Very advance. Very difficult.
• A proof appears in the book of Shmoys and
Williamson.
Simpler topics?
• I can not complain if it is TAUGHT! Of
course not. Let me give a list of basic topics
that always taught
• Ratio3/2 for TSP, the simple approximation of
2 for min cost Steiner tree.
• Set-Cover , simple approximation ratio.
• Knapsack, PTAS. Bin packing, constant ratio.
• Set-Coverage. BUT: only costs 1.
Knapsack Set-Coverage
• The Set-Coverage problem is given a set system
and a number k select k sets that cover as many
elemnts as possible.
• Knapsack version, not that known:
• Each set has cost c(s) and there is a bound B on
the maximum sum of costs, of sets we can
choose.
• Maximize number of elements covered.
Result due to Khuller , Moss
and, Naor, 1997, IPL
• The (1-1/e) ratio is possible.
• In the usual algorithm & analysis (1-1/e) only
follows if we can add the last set in the greedy
choice. Thus, fails.
• Because most times, adding the last set will give
cost larger than B.
• Trick: guess the 3 sets in OPT of least cost.
Then apply greedy (don’t go over budget B).
Why do I know this paper?
• I became aware of this result only several years after
published. And only because I worked on Min Power
Problems. No conference version!
• This result seems absolutely basic to me. Why is it no
taught?
• Remark: Choosing one (least cost) element of OPT
gives unbounded ratio. Choosing two sets of smallest
cost gives ratio ½. Guessing the three sets of least cost
and then greedy gives (1-1/e).
First general neglected topic
• Important and not taught: Maximizing a
submodular non-decreasing function under
Matroid Constrains, ratio 1/2, Fischer,
Nemhauser, Wolsey, 1977.
• Improved in 2008(!) to best possible (1-1/e) by
Vondrak in a brilliant paper.
First story: a submission I refereed
• I got a paper to referee, and it was obvious that it is
maximize Submodular function under Matroid
constrains
• If memory serves, the capacity 1, of the following
Matroid: G(V,E), edge capacities, fix S  V. T reaches
S if every vertex in T can send one unit of flow to S.
• The set of all T that reach S a special Matroid called
Gammoid. Everything in this paper, known!
• Asked Chekuri (everybody must have an oracle) what is
the Matroid, and Chekuri answered. Paper erased.
Story 2: a worse outcome.
• Problem. Input like Set-Cover but S= Si.
• Required: choose at most one set of every Si
and maximize the number of elements covered.
• Paper gave ratio ½. This is maximizing
submodular cover subject to partition Matroid.
PLEASE!!! Do not try to check who the authors
are. Not ethical. Unfair to authors, as well.
• Nice applications, but was accepted and ratio
not new.
Related to pipage rounding
• Due to Ageev, Sviridenko.
• Dependent rounding, is a generalization of
Pipage rounding by Gandhi, Khuller, Parthasarathy,
Srinivasan.
• Say that we have an LP and a constraint
xi=k. RR can not derive exact equality.
• Pipage Rounding : instead of going to a larger set of
solutions like IP to LP, we replace the objective
function.
The principals of pipage rounding
• We start with LP maximization with function
L(X).
• Define a non linear function F.
• Show that the maximum of F is integral.
• Show that integral points of F belong to the
Polyhedra of L. Namely feasible for L as long
as it is integral, and feasible for F.
The principals of pipage rounding
• Then, show that F(Xint ) ≥L(X* )/, for
> 1.
• Here Xint is the (integral) optimum of F and
X* the optimum fractional solution for L
• Because Xint is known to be feasible for L(x) due
to its integrality, it is feasible for L and thus 
approximation.
Example: Max Coverage
• Max j wi zi
S.T
 element j belongs to set i xi≥zj
 set i xi=p
xi and zj are integral
In Set Coverage we bound the number of sets.
The function F
F(x)= j wi (1-element j belongs to set i(1-xi) )
Define a function on a cycle.
As a function of .
The idea is to make plus  and then minus 
over the cycle.
• Make one entry on the cycle smaller by  and
another larger by .
•
•
•
•
The function F
• F(x)= j wi (1-element j belongs to set i(1-xj) )
• The idea is to make plus  and then minus  all
over the cycle.
• But to show convexity we make just one entry
on the cycle smaller by  and another larger by .
• The  appears as 2 in this term.
The function F
• As  appears as 2 in this term, the second
derivative is positive.
• Thus F is convex.
• Which means that the maximum is in the
borders.
• For example for x2 between -4 and 3.
• The maximum is in the border -4.
Changing the  two by two
• Putting plus and minus  alternating along a
cycle make at least one entry integral.
• Moreover, we can decompose a cycle into two
matching and there are two ways to increase and
decrease by .
• One direction of the two makes the function
not smaller.
• This implies that the optimum of F is integral.
Thus the optimum of F is
integral
• Its not hard to see that on integral vectors F and
L have the same value.
• Another inequality that is quite hard to prove is
that: 1-i=1 to k (1-xi)≥(1-(1-1/k)k)L(X)
• This gives a slightly better than 1-1/e ratio if k is
small.
Submodularity: related to very basic
technique.
• f is submodular if f(A)+f(B)f(AB)+f(AB)
• Makes a lot of difference if non-decreasing or
not. If not, in my opinion represent concave.
• If non-decreasing, brings us to the next lost
simple subject: Submodular cover problems.
• Input: U and submodular non-decreasing
function f and cost c(u) per item u.
• Required: a set S of minimum cost so that
f(S)=f(U).
Wolsey , 1982, did much better
• Each iteration pick item u so that helpu(S)/c(u) is
maximum.
• The ratio is max{u U}ln f(u)+1.
• Example: For Set-Cover ln|s|+1, s largest set.
• Example: Same for Set-Cover with hard capacities. A
paper in 1991, and one in 2002, did this result again
(second was 20 years after Wolsey). Special case after 20
years! But its worse, yet.
• Wolsey did better than that. Natural LP unbounded
ratio even for Set-Cover with hard capacities.
• Wolsey found a fabulous LP of gap max{u U}ln f(u) +1.
More general: density
•
•
•
•
•
•
•
•
Not taught at all but just cited. Why?
Here is a formal way:
Universe U and a function f: 2UR+
Each element in U has a cost c(u).
The function f not decreasing.
We want to find a minimum cost W so that f(W)=f(U).
We usually say, S U, c(S)=uS c(u)
But it works for an subadditive cost function
The density claim
• Say that we already created a set S via a greedy
algorithm.
• Now say that at any iteration we are able to find some Z
so that:
(f(Z+S)-f(S))/c(Z)≥(f(U)-f(S))/(δ·opt)
• Then the final set S has cost bounded by
(δ ln(U)+1) opt
What does it mean?
• Think for the moment of δ=1.
• Say that the current set S has no intersection with the
optimum.
• Then if we add all of OPT to S we certainly get a
feasible solution.
• Then clearly f(S+OPT)=f(U)
• And
• (f(S+OPT)-f(S))/c(Z)≥ (f(OPT)-f(S))/c(OPT)
• =(f(U)-f(S))/f(OPT)
• It means that we found a solution to add that has the
same density as adding OPT.
Proof continued
• f(U) -  j≤ i-1 f(Sj)≥ 1.
• We may assume that the cost of every set
added is at most
opt, therefore c(Sj ) ≤ opt
• Therefore it remains to bound:
 j≤ i-1 c(Zi)
Let us concentrate on what happens before
Si is added.
By the previous claims
• 1 ≤ f(U)-f(Z1+Z2 +……Zi-1)≤
Πj≤ i-1(1-c(Zi)/δ·opt)· f(U)
• 1/f(U) ≤ Πj≤ i-1(1-c(Zi)/δ·opt)·
• Take ln and use ln(1+x) ≤ x:
-ln( f(U))≤  i≤ j -c(Zi) )/δ·opt
 i≤ j c(Zi) ≤ opt δ ln( f(U))
and so the ratio of (δ ln( f(U))+1) follows.
A paper of mine
• Min c  x subject to ABx b, with A positive entries
and B flow matrix. Ratio logarithmic.
• We got much more general results. The above I was
sure then and sure now, KNOWN and presented as
known.
• Referees: Cite, or prove submodularity! We had to
prove (referees did not agree its known!).
• Example: gives log n for directed Source Location.
Maybe first time stated but I considered it known.
• This log n was proved at least 4 times since then.
Remarks
• The bad thing about these 4 papers is not that did not
know our paper (to be expected) but that they would
think such a simple result is NOT KNOWN.
• It is good to know the result of Wolsey: for example,
used it recently (Hajiaghayi ,Khandekar,K , Nutov) to
give a lower bound of about log 2 n for a problem in
fashion: Capacitated Network Design (Steiner network
with capacities). First lower bound for hard capacities.
Dual fitting and a mistake we all
make
• 1992. GK to Noga Alon:
• This (spanner) result bares similarities to the proof
done by Lovats for set-cover.
• Noga Alon (seems very unhappy, maybe angry): Give
me a break! That is folklore. Lovats told me he wrote it
so he would have something to cite.......
• Everybody cites Lovats here. Its simply not true.
• We don’t know the basics. Result known many years
before 1975.
• Should we cite folklore? Yes!
HOW to teach dual fitting for set
cover, unweighted?
• Let S be the collection of sets and T the
elements.
• The dual, costs 1: Maximize tT yt
• Subject to: ts yt  c(s)=1
• We define a dual: if the greedy chose a star of
length i, each element in the set gets 1/i
.2 .2 .2
.2
.2
The bound on the sum of elements
of a given set
1/7 1/5
1/7 1/4 1/4
1/3
1/12 1/11
1/12 1/12
1/10 1/9
1/12 1/8
1/12 1/7 1/6
1/2
1/2
1
Primal Dual of GW
• Goemans and Williamson gave a rather well
known Primal-Dual algorithm. Always taught,
and should be.
• A question I asked quite several researchers and
I don’t remember a correct response: Why
reverse delete?
• Why not Michael Jackson?
• GW primal dual imitates recursion.
• In LR reverse delete follows from recusrsion.
Local Ratio for covering problems
• Give weights to items so that every minimal.
solution is a  approximation. Reduce items
costs by weights chosen.
• Elements of cost 0 enter the solution.
• Make minimal.
• Recurse.
• No need for reverse delete. Recursion implies it.
• Simpler for Steiner Network in my opinion.
Local Ratio
• Without it I don’t think we could find a ratio 2 for
Vertex feedback set.
• A recent result of K, Langberg, Nutov. Minor result
(main results are different) but solves an open problem
of a very smart person: Krivelevich.
• Covering triangles, gap 2 for LP (polynomial).
• Open problem: tight?
• Not only we showed tight family but showed as hard as
approximating VC. Used LR in proof.
Group Steiner problem on trees
• Group Steiner problem on trees.
• Input : An undirected weighted rooted by r tree
T = (V; E) and subsets S1,……,Sp V.
• Goal: Find a tree in G that connects at least one vertex
from each Si to r.
• The Garg, Konjevod and Ravi proof while quite simple
can be much much further simplified. In both proofs:
O(log n· log p) ratio.
• The easier (unpublished) proof is by Khandekar and
Garg.
The theorem of Garg Konjevod and
Ravi
• There is an O(h log p)-approximation algorithm
for Group Steiner on trees.
T= (V; E) rooted at r has depth h.
• Simple observation: we may assume that the
groups only contain leaves by adding zero cost
edges.
• The GKR result uses an LP methods.
The fractional LP
• Minimize e cost(e)· xe
frg=1 For every g.
feg≤ xe
fvg ≤ v’ child of v fvv’(g)
fvg = fpar(v) v(g)
The xe are capacities. Under that, the sum of flows from r to the leaves that
belong to g is 1. If we set xe =1 for the edges of the optimum we get an
optimum solution.
Thus the above (fractional) LP is a relaxation.
The rounding method of GKR
• Consider xe and say that its parent edge is
(par(v),v)
• Independently for every e, add it to the solution
with probability xe/xpar(v)v
• We show that the expected cost is bounded by
the LP cost.
• The probability that an edge gets to the root
is a telescopic multiplication.
The probability that an edge is
chosen
• All terms cancel but the first and the last. The
First is xe. The last is the flow from ‘ The parent
of r to r ’ which we may assume is 1.
• Since this is the case, xe contributes
xe· cost(e) to the expected cost.
• Therefore, the expected cost is the LP value
which is at most the integral optimum.
However: what is the probability that a group is
covered?
The probability a group is
covered
• Let v be a vertex at level I in the tree,
then the probability that after rounding
there is a path from v to a vertex in g
is at least:
fvg /((h-i+1)· xpar(v)v)
Let P(v) be this probability that the
group is not covered
• Let P(v) be the probability that there is no path
from v to a leaf in group g. In the next
inequalities a vertex v’ is always a child of v and
the corresponding edge is e=(v,v’).
• P(v)=Πv’ (1-(xe· (1-p(v’))/xpar(v)v )
• Explanation: The probability for a group to get
connected to v’ for some child v’ of v is (1-P(v’)).
Given that, the probability that the edge (v,v’)
gets selected is xe·/xpar(v)v . The multiplication is
because the events are independent for different
children.
Proof continued
• (1-P(v’)) is the probability that v’ can reach a leaf
of g by a path after the randomized process.
• By the induction assumption:
(1-P(v’)) ≥ fgv’ /((h-i+1)· xpar(v)v)
Therefore:
P(v)≤Π(1-xe· fgv’ /(xpar(v)v(h-i)·xe)=
Π (1-fgv’ /(xpar(v)v(h-i))
Proof continued
• We use the inequality 1-x≤exp(-x) to get the
inequality:
P(v) ≤ exp(- fgv’
/(xpar(v)v(h-i))
• From the constrains of the LP we get:
• P(v) ≤exp( -fgv/(xpar(v)v(h-i)))
Ending the proof
• Use the inequality exp(1/(1-x))≤1-1/x
to get:
• P(v) ≤ 1- fvg/((h-i-1) · xpar(v)v)
• This ends the proof.
• We now only have to consider v=r
Proof continued
For the root we may think of xpar(r)r=1
• For the root frg=1 and thus the
probability that a group is covered is at
least 1/(h+1). The probability that a
group is not covered in (h+1)· ln p
iterations is at most
• (1-1/(h+1))(h+1)·ln p exp(-ln p)=1/p
•
End of proof.
• Since a group is not covered with probability
1/p we can take every uncovered group and join
it by a shortest path to r. A shortest path from
any group member to r is at most opt.
• Thus the expected cost of this final stage is:
1/p· p · opt=opt
• Thus the expected cost is (h+1)ln p· opt+opt
Making the h=log n
• Question: If the input for Group Steiner is a very tall
tree to begin with. How do we get O(log2 n) ratio?
• Use FRT? Looses a log n and complicated.
• Basic but probably not widely known: Chekuri Even
and Kortsarz show how to reduce the height of any
tree to log n with a penalty 8 on the cost. Combinatorial!
• In summary, we get an elementary analysis of
O(log n· log p) approximation ratio for the Group
Steiner on trees.
Recursive greedy
• Never taught. Directed Steiner, basic problem.
• A gem by Charikar et al. Say that the number of
terminals to be covered is z. There is a child u in T*
whose density is at most opt/z.
• Let z’ be the number of terminals in T*u
• The analysis stops once we cover at least z’/(h-1)
terminals. Details omitted but gives telescopic
multiplication that means density returned at most
hopt/z.
• Can make h O(1/) with ratio penalty n1/
(Zelikovsky). Time: larger but in the ball park of nh =
nO(1/).
Alternative approximation algorithm
for Directed Steiner
• This was known (Chekuri told me) apparently in more
complex form, since 1999.
• The simpler way (as far as I know) Mendel and Nutov.
• Create a graph H in which each path from the root r to
some terminal u of length at most 1/ , is a node.
• There is a directed edge between p’ and p if p extends p’
by one edge.
• By the theorem of Zelikovsky, a solution of cost at
most O(n 1/ )opt is embedded in H.
A non recursive greedy
approximation for Directed Steiner
• For every terminal t, make a group Ht of all paths of
length at most 1/ that start at r and end at t.
• This reduces the problem to Group Steiner on trees:
Connect at least one terminal of Ht by a path from r ,
for every t . Our analysis works and it’s a page and a
half.
• This gives a non Recursive Greedy algorithm of two
pages for Directed Steiner with same ratio: n. Only
black box is the (very complex) height reduction CEK
and the Zelikovsky theorem.
Certificate of failure
• Many papers say that: 'The value opt of OPT is
KNOWN'.
• Knowing opt?? How can we know opt? Absurd. Means
P=NP.
• I first saw this in a paper of Hochbaum and Shmoys
from J.ACM 1984. The paper is called: Powers of
graphs: A powerful approximation technique for
bottleneck problems.
• Certificate of failure. Take . If  < opt the algorithm
may return a set of size  opt.
• Alternatively, may return failure. In this case  < opt.
and then this hold true (this is why its certificate).
Certificate of failure
• In case  > opt algorithm returns a solution of
cost at most  opt.
• Binary search: fails for /2 but succeeds with 
. As /2<opt, and return a solution of cost at most 
opt, the ratio is 2
• Referees of my papers failed to understand that,
many many times. Convention does not seem to
be known to all. Should be!
Density LP: useful and basic
• Say that you have an LP for a covering problem
that has some good ratio.
• But now you only want to cover k of the
elements. For every element x, there will be a
variable yx that says how much x is taken.
• We write  yx=k but then divide the sum by k
which means that the objective value is also
divided by k. Thus we try to solve a density LP.
Density LP
• You can get the original ratio with penalty in the
ratio of O(log 2 n)
• Number of items inside the solution may be
much more than k therefore if we can get
exactly k may depend on the possibility of
density decomposition.
• I first was shown this (by Chekuri) about 6 years
ago. What do I not know about LP now? I fear
that a lot.
Application of the basics, example 1
• Broadcast problem, directed graph, Steiner set S.
• A vertex r knows a message and the goal is to
transmit it to all of S. Let K be the set that know
the message and N those who don’t. At every
round a directed matching from K to N.
• The endpoint in N of the matching join K.
• Minimize number of rounds.
• Let k=|S|. Remark: Result obtained with Elkin.
Algorithm
• Find u that has at least sqrt{k} terminals at distance at
most opt from u.
• Remove Tu with exactly sqrt{k} terminals from G and
height at most opt. Let N remaining vertices.
• Iterate untill no such u.
• Let K’ be the union of trees, R be the roots. Clearly
number of roots at most sqrt{k}.
• Can not employ recursion but can inform all K’ in
2sqrt{k}+2opt rounds.
To finish enough to inform distance
opt dominating set DN
• Cover NS by trees rooted at D. No vertex in
those trees has more than sqrt{k} terminals at
distance opt. So informing the rest of N given
D K, requires opt+sqrt {k} rounds.
• How do we inform a distance opt dominating
set?
• Reduce to the minimization version of
maximazing a non-decreasing submodular
function under partition Matroid.
Define a new graph
(k,n) (k,n1) (k,n2)
z
p
q
(k,np)
z p q
opt opt opt
n
Finding k<|S|Arborescence from r
with minimum maximum outdegree
s’
sqrt{k}
sqrt{k}
W
sqrt{k}

t’ k
1
1
t
Solution
• Solution obtained with Khandekar and Nutov.
• Edges that carry flow and an arborescence from
r to W. Flow(W) non-decreasing submodular
• We prove there exists a size sqrt{k/} feasible
W. Non-trivial proof, omitted.
• The capacity of vertices and edges, divided by
 is also sqrt{k/}.
• By the Wolsey theorem about sqrt{k/} ratio
approximation. The LP gap is sqrt{k}!
Summary
• It goes without saying that my opinions bound me only.
• My intention is not to change courses for real. Will be
presumptuous.
• Will I follow my own advice? Yes.
• Can not only use the wonderful existing slides.
• The little man always had to struggle in very difficult
circumstances.
• Thank you