Transcript Mars Router - Stanford University

EE382C
Lecture 6
Adaptive Routing
4/14/11
1
EE 382C - S11 - Lecture 6
Question of the day
What is the worst-case traffic pattern for CQR routing on a
torus?
2
EE 382C - S11 - Lecture 6
Review of Routing Basics
• Deterministic, Oblivious, or Adaptive Routing
• Deterministic – simple, maintains ordering, only choice if no
path diversity
• Oblivious
– VAL – achieves optimum throughput on worst-case traffic
• But destroys locality, same throughput on all traffic
– Linearity lets us find worst-case traffic pattern for arbitrary oblivious
algorithm
• Bipartite maximal-weight matching problem
3
EE 382C - S11 - Lecture 6
Report Card from Last Time
Algorithm
Locality
Uniform traffic
Tornado traffic
Greedy
A
1 (A)
0.33 (F)
Random
F
0.5 (F)
0.4 (C)
RLB
C
0.76 (C)
0.53 (A)
VAL
F
0.5 (F)
0.5 (A-)
4
EE 382C - S11 - Lecture 6
Topology & Routing
5
EE 382C - S11 - Lecture 6
Adaptive routing
• Path from source A to destination B based on network
conditions
• Advantages
– Can provide low latency at low loads and good load balancing at
high loads
• Disadvantages
– Making good routing decisions with approximation information can
be difficult
– Packet reordering
6
EE 382C - S11 - Lecture 6
Minimal adaptive routing
• Two phases
– Route “up” adaptively
towards a random top level
node
– Route “down”
deterministically to
destination
• “Turn around” at first
common ancestor
– Keeps route minimal
7
EE 382C - S11 - Lecture 6
How is minimal adaptive on a Clos
different than minimal Oblivious?
8
EE 382C - S11 - Lecture 6
Transient Imbalance
16
Maximum buffer size
14
12
10
8
6
4
2
0
Routers in the
middle
stage
of the network
EE 382C
- S11 - Lecture
6
9
With Adaptive Routing
16
Maximum buffer size
14
12
10
8
6
4
2
0
Routers in the middle stage of the network
EE 382C - S11 - Lecture 6
10
Minimal adaptive routing
• Minimal Adaptive (MIN AD) [Gravano ’93]
– Always route in minimal quadrant
– Adaptively routing in each hop
• Optimal Qben; Poor Qadv
EE 382C - S11 - Lecture 6
11
Minimal adaptive drawback
03
13
23
33
02
12
22
32
01
11
21
31
00
10
20
30
12
EE 382C - S11 - Lecture 6
Fully adaptive routing
03
13
23
33
02
12
22
32
01
11
21
31
00
10
20
30
• Any issues with misrouting?
13
EE 382C - S11 - Lecture 6
Livelock issues
03
13
23
33
02
12
22
32
01
11
21
31
00
10
20
30
14
EE 382C - S11 - Lecture 6
Global Adaptive Routing
• Global - Make overall routing decision (e.g., which quadrant
to traverse in a torus).
• Adaptive - Based on network state
• Need to estimate network state at packet source.
• Need a method to partition routes - so we can choose
between them based on network state.
15
EE 382C - S11 - Lecture 6
Reconsider an 8-node Ring
Congested clockwise channels
Receiver
Sender
0
1
2
3
4
5
6
7
Sender cannot sense congestion locally
Two partitions: clockwise and counter-clockwise
16
EE 382C - S11 - Lecture 6
Channel Queue Routing CQR
1. Check queue lengths for
outgoing channels at source
2. Compute quadrant congestion
for each quadrant - q1, q2, q3, q4
3. There are 4 quadrants with hop
counts H1, H2, H3, H4
4. Choose quadrant i
corresponding to the smallest Hiqi
5. Route minimally adaptively
within chosen quadrant
17
EE 382C - S11 - Lecture 6
CQR: TOR Throughput
Accepted Thr oughput
0.6
Minimal
Non-minimal
Overall
0.5
0.4
0.3
0.2
0.1
0
0
0.1
0.2
0.3
0.4
Switches to nonOffered Load (fraction of capacity)
minimal at 0.12
0.5
0.6
18
EE 382C - S11 - Lecture 6
CQR: TOR Latency
50
Minimal
Non-Minimal
Overall
Average Latency ( Cycles)
45
40
35
30
25
20
15
10
5
0
0
0.1
0.2
0.3
0.4
0.5
0.6
Offered Load
19
EE 382C - S11 - Lecture 6
Report card
Throughput (frac of capacity)
Latency (Cycles)
Algo
Θ benign
UR traffic
Tlow (0.2
UR)
VAL
0.5 (F)
0.5 (A)
0.5 (C)
10.5 (F)
20.5 (B)
DOR
1.0 (A)
0.25 (F)
0.31 (F)
4.5 (A)
 (F)
RLB
0.76 (C)
0.31 (C)
0.51 (C)
6.5 (C)
5.5 (A)
1.0 (A)
0.33 (C)
0.63 (B)
4.5 (A)
 (F)
GOAL
0.76 (C)
0.5 (A)
0.68 (A)
5.5 (C)
5.5 (A)
CQR
1.0 (A)
0.5 (A)
0.73 (A)
4.5 (A)
5.5 (A)
MIN AD
Θ adv
traffic
Θ avg
Tint (0.4
TOR)
20
EE 382C - S11 - Lecture 6
Routing Mechanics
21
EE 382C - S11 - Lecture 6
Routing Mechanics
• The mechanism used to implement any routing algorithm:
deterministic, oblivious, or adaptive
• Table-based routing
– Source routing
– Node table routing
• Algorithmic routing
22
EE 382C - S11 - Lecture 6
Source table routing
Advantages:
10
11
12
13
00
01
02
03
•Very low routing delay
•Routers can be used in any
topology.
Disadvantages:
•Can’t be used for adaptive
routing.
•Packet overhead
•Any others?
23
EE 382C - S11 - Lecture 6
Building routing tables
• There may be several routes for each destination
– These may be selected using a pseudo random generator as a
selector
– For ordering, hash using a flow identifier
• Having multiple routes help in
– Fault tolerance
• If routes are edge disjoint
– Load balance
• Distribute load over several network paths
– Distribution
• When destination is a logical destination to a “service” rather than a
physical destination node. In this case, the multiple routes may in fact
lead to multiple destinations
24
EE 382C - S11 - Lecture 6
Encoding variable length header
N
E
N
N
E
N
N
W
P
E
N
N
N
N
E
W
E
-
N
-
N
-
W
F
N
-
W
-
N
X
N
N
N
-
W
-
N
X
N
N
N
-
W
-
N
X
N
N
(a) At start of route
-
-
P
-
E
F
(c) After four hops
(b) After first hop
N
-
W
-
N
X
N
N
(e) After eight hops
N
-
W
-
N
X
-
-
X
-
N
F
(f) After twelve hops
N
N
(d) After seven hops
Each line is a phit. Each rectangle is a flit. Read from right to left
25
EE 382C - S11 - Lecture 6
Impl variable length src table
Dest
Route 1
Route 2
E X E X WW X N E N E X N N X
• Avoids the storage waste associated with rounding
up all routes to a maximum length
26
EE 382C - S11 - Lecture 6
Node table routing
- Livelock avoidance (00 to 11 takes S
(10) then N)
- Latency at each hop
+ Local adaptivity (bolded letters are
misroutes)
+ Space saving
10
11
12
13
00
01
02
03
27
EE 382C - S11 - Lecture 6
Hierarchical Node Table
Table for node --- of an 8 ary 3 cube
28
EE 382C - S11 - Lecture 6
Algorithmic routing in a torus
routing header
sx x sy y
=0
•?
Choose first productive
dimension from left to right
•?
=0
Randomly select a productive
dimension
ydone
xdone
queue lengths
-y
+y
-x
•?
Allow selection of
unproductive dimensions if
productive dimension queue
lengths are above threshold
-y
+y
-x
+x
route selection
exit
selected
direction
vector
+x
Select a productive dimension
based on queue lengths
exit
productive
direction
vector
•?
• CHOICES:
Min OB, Min Ad, Full Ad, DOR
29
EE 382C - S11 - Lecture 6
Question of the day
What is the worst-case traffic pattern for CQR routing on a
torus?
30
EE 382C - S11 - Lecture 6
Summary
• Adaptive routing
–
–
–
–
–
Use network state
Is non-linear
Minimal adaptive - make local decisions
Non-minimal
Global adaptive (example: CQR)
• Sensing global state
• Partitioning routes
• Routing mechanics
– Tables
• Source, Node, Associative
– Algorithmic
31
EE 382C - S11 - Lecture 6