Transcript Randomized Algorithms

Permutation routing in n-cube
1
n-cube
1-cube
2-cube
3-cube
4-cube
2
Labeling
00
000
01
100
0
101
010
1
110
10
1000
1100
1010
1110
1111
011
111
11
1001
1101
001
0000
0100
1011
0101
0010
0110
0001
0011
0111
3
Definition of n-cube
• An n-cube:
– Number of nodes: N = 2n, each node has a
distinct n-bit label.
– The node with label (i1, i2, …, in)  {0, 1}n and
the node with label (j1, j2, …, jn)  {0, 1}n are
connected if the “hamming distance” between
(i1, i2, …, in) and (j1, j2, …, jn) is 1.
• Each node has degree n, so there are
nN/2 edges in an n-cube.
4
Permutation Routing
• Each node i contains one packet m(i) that
should be sent to the destination node d(i).
• Each destination node d(i) for each node i,
for 0  i  N – 1, should form a
permutation of {0, …, N – 1}, i.e., every
node is the destination of exactly one
packet
5
Example
000
100
001
101
010
110
011
111
d(000)=010
d(001)=000
d(010)=110
d(011)=001
d(100)=111
d(101)=100
d(110)=101
d(111)=011
6
The rule
• All interconnections between nodes occur in
synchronous steps
• Each direction of each edge can carry at most
one message in one step
• During a step, a node can send at most one
packet to each of its neighbors
– That is, at most n outgoing messages for each node
in each time step.
7
Two steps suffice
000
100
001
101
010
110
011
111
d(000)=010
d(001)=000
d(010)=110
d(011)=001
d(100)=111
d(101)=100
d(110)=101
d(111)=011
8
Things can be complicated…
000
100
001
101
010
110
011
111
d(000)=111
d(001)=110
d(010)=101
d(011)=100
d(100)=011
d(101)=010
d(110)=001
d(111)=000
9
two time
Traffic control Need
steps for this
edge.
000
100
001
101
010
110
011
111
d(000)=111
d(001)=110
d(010)=101
d(011)=100
d(100)=011
d(101)=010
d(110)=001
d(111)=000
10
Oblivious Routing
• For any nodes i and j in {0, 1, 2, …, N–1},
the route between nodes i and j depends
on the labels of i and j only.
– The “memoryless” feature is attractive for
hardware design.
– For example, the bit-fixing algorithm is an
oblivious routing algorithm.
11
A Question
• Let P and P’ be two oblivious paths in ncube. If they overlap at some edge e and
then separate at the endpoint x of e, can P
and P’ rejoin again later at some node y?
y
x
12
Negative!
• The segments of P and P’ from x to y
depend only on the labels of x and y.
y
x
13
Bit-Fixing Routing Algorithm:
For each node i, we send the message mi from node i
to node d(i) by the following iterative procedure, where
(k1 ; k2 ; : : : ; kn ) is the label of node d(i):
Let node j be the current node that receives mi . If j =
d(i), the procedure stops. Otherwise, let (j1 ; j2 ; : : : ; jn )
be the label of node j. Suppose the ¯rst ` bits of nodes
j and d(i) are identical and they di®er at the (` + 1)-st
bit. Then, we send mi from node j to the node with
label (j1 ; j2 ; : : : ; j` ; k`+1 ; j`+2 ; : : : ; jn ). Repeat this procedure.
14
Example
• (0, 1, 0, 0, 1, 0) 
(1, 1, 0, 0, 1, 0) 
(1, 0, 0, 0, 1, 0) 
(1, 0, 1, 0, 1, 0) 
(1, 0, 1, 0, 1, 1).
15
The weakness
• Bit-fixing routing is a deterministic
oblivious routing algorithm.
• Deterministic oblivious routing algorithms
have a quite bad lower bound on the
number of time steps required for
permutation routing.
16
A SPAA’90 result
• [Kaklamanis, Krizanc, Tsantilas, SPAA’90]
– For any deterministic oblivious
permutation routing algorithm on a network
of N nodes each of degree n, there is an
instance of permutation routing requiring
((N/n)0.5) time steps.
• Comment: The network does not have to be
an n-cube.
17
Bit-fixing on n-cube
• By [Kaklamanis, Krizanc, Tsantilas, SPAA’90]
– The bit-fixing routing algorithm on an N-
node n-cube has an instance of
permutation routing requiring
((N/n)0.5)=((N/log N)0.5) time steps.
• We show a stronger lower bound
– There is an instance such that the bit-fixing
algorithm requires (N0.5) time steps.
18
A bad instance for bit-fixing
• Consider the case that n is an even
number, say, n = 2t for some positive
integer t.
• For each node i in {0,1,2,…,N-1}, let d(i)
be defined as follows.
– If (i1, i2, …, in) is the label of i, then let d(i) be
the node with label (it+1,it+2,…,in, i1, i2, …, it).
19
Example, n = 4.
1000
1100
1101
1010
1110
1001
1111
0000
0100
1011
0101
0010
0110
0001
0111
0011
d(0000)=0000
d(0001)=0100
d(0010)=1000
d ( 0 0 11 ) = 11 0 0
d(0100)=0001
d(0101)=0101
d ( 0 11 0 ) = 1 0 0 1
d ( 0 111 ) = 11 0 1
d(1000)=0010
d ( 1 0 0 1 ) = 0 11 0
d(1010)=1010
d ( 1 0 11 ) = 111 0
d ( 11 0 0 ) = 0 0 11
d ( 11 0 1 ) = 0 111
d ( 111 0 ) = 1 0 11
d ( 1111 ) = 1111
20
Traffic jam at an edge e
• Consider the directed edge e from
node (000…01 000…00) to
node (000…00 000…00)
• The message sent from each node with
label (???…?1 000…00) has to pass
edge e before reaching its destination
(000…00 ???…?1).
• There are 2n/2-1 = (N0.5) such messages.
21
Breaking the lower bound by
randomization…
Q: Any proposals for how to
randomize a permutation routing
in an n-cube?
22
Valiant and Brebner’s
algorithm [STOC’81]
23
Valiant-Brebner
• Phase 0:
– For each i = 0 to N – 1, choose an
intermediate node t(i) uniformly at random
from {0, 1, …, N – 1}.
• Comment:
– Note that t is not necessarily a permutation
on {0, 1, …, N – 1}.
24
Valiant-Brebner
• Phase 1:
– Route each message m(i) from node i to its
temporary destination t(i) using bit-fixing algorithm.
• Phase 2:
– Route each message m(i) from node t(i) to its real
destination d(i) using bit-fixing algorithm.
• Comments:
– No randomization in these two steps.
– Phase 2 starts only after Phase 1 is completely
finished.
25
The waiting list
• Each node keeps a waiting list for each of its n
neighbors.
• The waiting list operates in a First-In-First-Out
manner.
• For multiple messages joining a particular
waiting list at the same time, the tie can be
broken arbitrarily.
26
How good is Valiant-Brebner?
• Theorem
– With probability at least 1-O(N -1) the number of time
steps required by Valiant-Brebner’s randomized
oblivious routing algorithm is only O(log N).
• Comments:
– O(log N) is much better than the lower bound for
deterministic oblivious algorithms.
– The bound is the same even for the special case that
d(i) = i for each index i = 0, 1, …, N – 1.
27
Math tool
Prof. Herman Chernoff’s bound,
Annal of Mathematical Statistics 1952
28
Bernoulli trial
Let X be a binary random variable of value 1
(respectively, 0) with probability 𝑝
(respectively, 1 − 𝑝)
----------------------------------------------𝐸 𝑋 = 𝜇𝑋 = Pr 𝑋 = 1 = 𝑝
𝑉𝑎𝑟 𝑋 = 𝜎𝑋2 = 𝑝 1 − 𝑝 .
29
Bernoulli trials
Let 𝑋 = 𝑛𝑖=1 𝑋𝑖 , where each 𝑋𝑖 with 1 ≤ 𝑖 ≤ 𝑛 is an
independent binary r.v. of values 1 with probability 𝑝.
----------------------------------------------𝑛
𝐸 𝑋 =𝐸
𝑉𝑎𝑟 𝑋 = 𝑉𝑎𝑟
𝑛
𝑖=1
𝑋𝑖 =
𝑋𝑖 =
𝑖=1
𝑛
𝑛
𝑖=1
𝐸[𝑋𝑖 ] = 𝑛𝑝
𝑉𝑎𝑟[𝑋𝑖 ] = 𝑛𝑝 1 − 𝑝 .
𝑖=1
30
Poisson trials
Let 𝑋 = 𝑛𝑖=1 𝑋𝑖 , where each 𝑋𝑖 with 1 ≤ 𝑖 ≤ 𝑛 is an
independent binary r.v. of values 1 with probability 𝑝𝑖 .
----------------------------------------------𝐸 𝑋 =𝐸
𝑉𝑎𝑟 𝑋 = 𝑉𝑎𝑟
𝑛
𝑖=1
𝑛
𝑖=1
𝑛
𝑋𝑖 =
𝑋𝑖 =
𝑛
𝑖=1
𝐸[𝑋𝑖 ] =
𝑉𝑎𝑟[𝑋𝑖 ] =
𝑖=1
𝑛
𝑛
𝑖=1
𝑝𝑖
𝑝𝑖 (1 − 𝑝𝑖 ) .
𝑖=1
31
Chernoff bound
Theorem: Let 𝑋 = 𝑛𝑖=1 𝑋𝑖 , where 𝑋1 , … , 𝑋𝑛 are n
independent Poisson trails such that 0<Pr(𝑋𝑖 =1)<1
for each i=1,…n. Then for any 𝛿 > 0,
Pr 𝑋 ≥ 1 + 𝛿 𝜇 <
𝑒𝛿
(1+𝛿) 1+𝛿
where 𝜇 = 𝐸 𝑋 .
Therefore, if 1 + 𝛿 > 2𝑒,
Pr 𝑋 ≥ 1 + 𝛿 𝜇 ≤ 2−
𝜇
,
1+𝛿 𝜇 .
32
Analyzing Valiant-Brebner
33
• If a packet is routed to a randomly chosen
node t= (t1,...,tn), we can think each ti is 0
(or 1) with probability ½.
• For a given packet M, let T(M) be the
number of steps for M to finish Phase 1.
• For a given edge e, let X(e) denote the
total number of packets traversing edge e
during Phase 1.
• In each step of Phase 1, M is either
traversing an edge or waiting in a queue
when some other packet traverses an
edge on M’s route.
34
• Lemma : Let e1,...,em be the m <= n edges
traversed by a packet M in Phase 1.
Then T(M)  X(e1) + X(e2) +...+ X(em).
• Call a path P=(e1,...,em) of m  n edges that
follows the bit-fixing algorithm a possible packet
path, with ei=(vi-1, vi).
There are at most 22n such paths. Why?
• For any possible packet path P =(e1,...,em),
define T(P) = X(e1) + X(e2) +...+ X(em).
• Note that the probability that Phase 1 takes
more than K steps is bounded by the probability
that for some possible packet path P, T(P) >= K.
35
• Since T(P) = X(e1) + X(e2) +...+ X(em), it
is natural to use Chernoff bound, but
unfortunately X(ei) are not independent.
• To overcome this, we first use Chernoff
bound to prove that with high probability
(w.h.p.) no more than 6n different packets
cross any edge of P.
• Then condition on this event to derive a
high-probability bound on the total
number of transitions these packets make
through edges of P, again by Chernoff
bound.
36
• Fix a possible packet path P=(e1,...,em) with m
edges and ei=(vi-1, vi).
• Call a packet active at a node vi-1 on P if it
reaches vi-1 and has the possibility of crossing
edge ei to vi.
• Call a packet active if it is active at some vertex
of P.
• Suppose vi-1 and vi differ in the j-th bit then for
packet to be active at vi-1 its j-th bit cannot have
been fixed by the bit-fixing algorithm when
reaching vi-1 .
37
• Let Hk be a 0-1 random variable such that
Hk=1 if the packet starting at node k is
active, 0 otherwise. Hk are independent.
Why?
• Let H= H0 +...+ HN-1. What is E[H]?
• Consider all the active packets at
vi-1 = (b1 ,...,bj-1, aj, aj+1,..., an)
and assume
vi = (b1 ,...,bj-1, bj, aj+1,..., an) .
38
• Before fixing the j-th bit, what are the
starting addresses of packets that can
reach vi-1 = (b1 ,...,bj-1, aj, aj+1,..., an)?
(* ,.....,*, aj, aj+1,..., an).
• How many such packets?
• Each of such packet can reach vi-1 only its
random destination is of the form:
(b1 ,...,bj-1, *, *,..., *).
• There are 2n-j+1 such random destinations.
39
• There are at most 2j-1 packets with
starting addresses of the form:
(* ,.....,*, aj, aj+1,..., an) that can reach
vi-1 = (b1 ,...,bj-1, aj, aj+1,..., an).
• Each of such packet can reach vi-1 with
random destination of the form:
(b1 ,...,bj-1, *, *,..., *).
• Thus, for such a packet, the probability of
visiting vi-1 is 2-(j-1). Why?
• Hence, the average active packet number
per vertex on P is 1. Why?
40
• Since there are m vertices on P,
E[H]  m  n.
• By Chernoff bound, we have
Pr( H  6n  6E[H] )  2-6n.
• Use Pr(A)= Pr(A|B)Pr(B)+Pr(A| B)Pr( B)
 Pr(B) + Pr(A| B).
• Pr( T(P) 30n)
 Pr(H 6n) + Pr( T(P) 30n | H < 6n)
 2-6n + Pr( T(P) 30n | H < 6n).
41
• Pr( T(P) 30n)
 2-6n + Pr( T(P) 30n | H < 6n).
• Claim: Pr( T(P) 30n | H < 6n)  2-3n-1 .
With the claim we have:
Pr( T(P) 30n)  2-3n
• Observe that if a packet leaves the path P,
it cannot return to the path in Phase 1.
Why?
42
• Consider an active packet on the path P at
vertex vi-1 .
• Suppose the vertices of ei differ in the j-th
bit. What is the probability that the
packet crosses ei = (vi-1, vi)?
• The probability of crossing ei is at most ½.
vi-1
vi
43
vi-1
vi
• View an active packet at a node vi -1 on the path
P that might cross edge ei as a trial.
• The trial is successful if the packet leaves the
path, but a failure if the packet stays on the
path. The successful probability is at least ½.
• Observe that if there are at most 6n active
packets, then there are at most 6n successes.
44
vi-1
vi
• Note that each trial is independent.
• Claim : with at most 6n active packets.
Pr( the active packets cross edges of
P more than 30n times)

Pr( flip fair coin 36n times with successes
fewer than 6n times).
45
• Let Z be the number of heads (successes)
in 36n fair coin flips.
• Pr( T(P)  30n | H  6n)
 Pr( Z  6n)
 exp(-18n (2/3)2/2), by Chernoff bound
= e-4n  2-3n-1
• Pr( T(P) 30n)
 Pr(H 6n) + Pr( T(P) 30n | H < 6n)
 2-3n.
46
• There are at most 22n possible packet
paths P in hypercube.
• Pr(  P such that T(P) 30n)
 22n 2-3n
= 2-n
= O(N-1).
• This completes the analysis of Phase 1.
• Phase 2 can be viewed as running Phase 1
backwards.
• Hence no packet spends more than 30n
steps in Phase 2 with probability 1- O(N-1).
47
• Finally, the total number of packet
traversals across the edges of any packet
path during Phase 1 and Phase 2 is
bounded by 60n steps with probability 1O(N-1).
48
Conclusion
• Deterministic oblivious routing algorithms
may have poor performance for some
specific cases
• Valiant-Brebner’s randomized routing
algorithm in n-cube has asymptotically
optimal performance with with high
probability.
49