Transcript The Concurrent Matching Switch Architecture

The Concurrent Matching
Switch Architecture
Bill Lin (University of California, San Diego)
Isaac Keslassy (Technion, Israel)
Motivation

Traffic demands expected to grow, driven in part by
increasing broadband adoption
 10x increase in broadband subscription in just last 3
years, already over 100 million subscribers
 1.25-2.4 Gbps fiber to homes emerging (GPON,
GEPON, EPON, BPON …)

Larger routers needed for consolidation

Operators need scalable routers that provide good
performance
IEEE INFOCOM, Barcelona, April 23-29, 2006
2
Limitations of Previous Routers

Output-Queueing (OQ) Switch
 Well-known to provide good performance, but
scalability hampered by need for internal N speedup

Crossbar Switches, using Input-Queueing (IQ) or Combined
Input-Output Queueing (CIOQ)

Huge body of literature, but scalability hampered by
need for centralized scheduling and arbitrary perpacket switch configurations
IEEE INFOCOM, Barcelona, April 23-29, 2006
3
Limitations of Previous Routers

Load-Balanced Routers
 No centralized scheduler
 Scalable fixed configuration switch fabric in optics
 Guarantees 100% throughput
 100 Tb/s design with 160 Gb/s linecards shown

But packets may be delivered “out-of-order”
IEEE INFOCOM, Barcelona, April 23-29, 2006
4
Basic Load-Balanced Router
A3 A2 A1
R
Linecards
In
Linecards
R/N
R/N
R/N
R/N
R/N
B2 B1 B1
R
In
R
In
R/N
R/N
R/N
R/N
Out
R
Out
R
R/N
R/N
R/N
R/N
R/N
IEEE INFOCOM, Barcelona, April 23-29, 2006
Out
R
R/N
R/N
R/N
R/N
C1 C2 C1
Linecards
5
Basic Load-Balanced Router
Linecards
R
In
Linecards
R/N
R/N
Linecards
A1
C1 B1
R/N
Out
R
Out
R
R/N
Many Fabric Options (any spreading
device)
R/N
R/N
Space: Full uniform mesh
R/N
 Wavelength:
Static WDMA2
R
 Time:
Round-robin switches
C2
In R/N

R/N
B1
R/N
R/N
R/N
Just need fixed uniform rate channels at R/N
R/N
R/N
R
R/N
No dynamic
R/N switch reconfigurations
A3
In
R/N
IEEE INFOCOM, Barcelona, April 23-29, 2006
B2
C1
R/N
Out
R
6
Basic Load-Balanced Router
Linecards
R
In
Linecards
R/N
R/N
A1
C1 B1
R
In
R/N
R
In
R/N
IEEE INFOCOM, Barcelona, April 23-29, 2006
Out
R
Out of
Order !
R/N
R/N
A2
C2
B1
R/N
Out
R
Out
R
R/N
R/N
R/N
R/N
R/N
R/N
R/N
R/N
R/N
Linecards
A3
B2
C1
R/N
R/N
7
Packet Ordering Problem

Out-of-order packet delivery is undesirable
(e.g. bad for TCP)

Previous techniques (e.g. EDF, UFS, FOFF)
 Accumulate and delay packets at input/middle ports
 And/or delay and re-order packets at middle/output ports

However, these techniques are unsatisfactory because they
add substantial delays
IEEE INFOCOM, Barcelona, April 23-29, 2006
8
Impact on Avg. Delay
(N = 128, uniform traffic)
UFS
FOFF
Basic Load-Balanced
Significant
Delay
IEEE INFOCOM, Barcelona, April 23-29, 2006
9
Concurrent Matching Switch (CMS)

Basic idea
 Retain load-balanced router structure and scalability of a
fixed optical mesh, no dynamic reconfiguration
 Instead of packets, load-balance “request tokens” to N
parallel “schedulers”
 Each scheduler independently solves its own matching
 Packets delivered in order based on matching results
Goal is to provide much lower average delay than
accumulation-based methods for ensuring packet
order while retaining 100% throughput and scalability
IEEE INFOCOM, Barcelona, April 23-29, 2006
10
Architecture
Linecards
R
Linecards
A4 A3 A2 A1
Linecards
Out
R
Out
R
Out
R
Retain Fixed
Configuration
Meshes
R
B2 B1
R
C2 C1
C2 C1
C1
IEEE INFOCOM, Barcelona, April 23-29, 2006
BUT move
packet buffers
to INPUT
11
Architecture
Linecards
R
A4 A3 A2 A1
R
R
Linecards
2 0 0
0 0 1
1 1 1
B2 B1
1 0 0
0 0 1
1 1 0
C2 C1
C2 C1
C1
1 0 0
0 0 0
0 0 0
IEEE INFOCOM, Barcelona, April 23-29, 2006
Linecards
Out
Add N2
Token
Out
Counters
Out
R
R
R
12
Arrival Phase
Linecards
Linecards
Linecards
A2 A1 A1
R
A4 A3 A2 A1
2 0 0
0 0 1
1 1 1
Out
R
Out
R
Out
R
B2 B1 B1
R
B2 B1
1 0 0
0 0 1
1 1 0
C2 C1
C2 C1
C1
1 0 0
0 0 0
0 0 0
C4 C3 C2
R
IEEE INFOCOM, Barcelona, April 23-29, 2006
13
Arrival Phase
Linecards
R
Linecards
A4 A3 A2 A1
A1
A2 A1
2 1 0
0 0 1
1 1 1
B2 B1
1 0 1
0 0 1
1 1 0
C2 C1
C2 C1
C1
1 0 1
0 0 0
0 0 0
Linecards
Out
R
Out
R
Out
R
B2 B1 B1
R
C4 C3 C2
R
IEEE INFOCOM, Barcelona, April 23-29, 2006
14
Arrival Phase
Linecards
Linecards
R
A4 A3 A2 A1
A1
A2 A1
2 1 0
1 0 1
1 1 1
R
B1
B2 B1
B2 B1
1 0 1
0 1 1
1 1 0
C2 C1
C2 C1
C1
1 0 1
0 1 0
0 0 0
Linecards
Out
R
Out
R
Out
R
C4 C3 C2
R
IEEE INFOCOM, Barcelona, April 23-29, 2006
15
Arrival Phase
Linecards
Linecards
R
A4 A3 A2 A1
A1
A2 A1
2 1 0
1 0 1
1 1 2
R
B1
B2 B1
B2 B1
1 0 1
0 1 1
1 1 1
R
C2 C1
C2 C1
C4 C3 C2 C1
1 0 1
0 1 0
0 0 1
IEEE INFOCOM, Barcelona, April 23-29, 2006
Linecards
Out
R
Out
R
Out
R
16
Matching Phase
Linecards
Linecards
R
A4 A3 A2 A1
A1
A2 A1
2 1 0
1 0 1
1 1 2
R
B1
B2 B1
B2 B1
1 0 1
0 1 1
1 1 1
R
C2 C1
C2 C1
C4 C3 C2 C1
1 0 1
0 1 0
0 0 1
IEEE INFOCOM, Barcelona, April 23-29, 2006
Linecards
Out
R
Out
R
Out
R
17
Matching Phase
Linecards
Linecards
R
A4 A3 A2
A1
A2 A1
2 1 0
1 0 1
1 1 2
R
B1
B2 B1
B2
1 0 1
0 1 1
1 1 1
R
C2 C1
C2
C4 C3 C2 C1
1 0 1
0 1 0
0 0 1
IEEE INFOCOM, Barcelona, April 23-29, 2006
A1
B1
C1
Linecards
Out
R
Out
R
Out
R
18
Matching Phase
Linecards
Linecards
R
A4
A1
A2 A1
1 1 0
1 0 0
1 0 2
A1
B1
C1
R
B1
B2
0 0 1
0 0 1
1 1 0
A2
B1
C1
C2 C1
C2
C4 C3
0 0 1
0 0 0
0 0 0
A3
B2
C2
R
IEEE INFOCOM, Barcelona, April 23-29, 2006
Linecards
Out
R
Out
R
Out
R
19
Departure Phase
Linecards
Linecards
R
A4
A1
A2 A1
1 1 0
1 0 0
1 0 2
A1
B1
C1
R
B1
B2
0 0 1
0 0 1
1 1 0
A2
B1
C1
C2 C1
C2
C4 C3
0 0 1
0 0 0
0 0 0
A3
B2
C2
R
IEEE INFOCOM, Barcelona, April 23-29, 2006
Linecards
Out
R
Out
R
Out
R
20
Distributed Operation

All linecards operate in parallel in a fully distributed
manner

Arrival, matching, and departure phases overlap in a
pipeline manner
IEEE INFOCOM, Barcelona, April 23-29, 2006
21
Main Ideas




Each middle linecard acts as a “micro-router” with 1/Nth
of the arrival traffic
Therefore, it gets N time slots to think about the
schedule, time complexity amortized by a factor of N
If each micro-router can guarantee 100% throughput, so
can the overall switch
Each micro-router can work the way that it wants,
leveraging huge body of existing work on scheduling
CMS provides a new way of aggregating routers
together. Therefore, provides a new way of thinking
about scaling routers.
IEEE INFOCOM, Barcelona, April 23-29, 2006
22
Practicality

Well-studied randomized approximations to Maximum
Weighted Matching have been shown to achieve very
good results [Tassiulas 1998] [Giaccone, Prabhakar & Shah, 2003]

These algorithms only require O(N) complexity using
sequential hardware, but can provide 100% throughput
guarantees with no speedup and good delay results

Amortized over N time slots, CMS with these scheduling
algorithms can achieve
 O(1) time complexity (independent of switch size)
 100% throughput
 Good delay results
 Packet ordering
IEEE INFOCOM, Barcelona, April 23-29, 2006
23
Experimental Results
(N = 128, uniform traffic)
UFS
FOFF
CMS
Basic Load-Balanced
Difference of
N time slots
for matching
phase
IEEE INFOCOM, Barcelona, April 23-29, 2006
24
Conclusions

CMS is scalable
 Leverages scalability of fixed optical meshes
 Fully distributed
 Can achieve O(1) time complexity

CMS achieves good performance
 Guarantees 100% throughput
 Guarantees packet ordering
 Experimentally achieves low packet delays

CMS provides new way of thinking about scaling
routers and connects huge body of existing literature
on scheduling to load-balanced routers
IEEE INFOCOM, Barcelona, April 23-29, 2006
25
Thank You