Transcript Document

Stage-Distributed Time-Division Permutation Routing in a Multistage Optically Interconnected Switching Fabric
Alvaro Cassinelli(1), Makoto Naruse (2), Masatoshi Ishikawa(1)
ECOC 2003 /
We4.P.137
1: University of Tokyo, Dept. Information Physics and Computing, 7-3-1 Hongo Bunkyo-ku, Tokyo 113-0033, Japan. ([email protected])
2: Communications Research Laboratory, 4-2-1 Nukui-kita, Koganei, Tokyo 184-8795, Japan.
(A)
Introduction
Exchange switch
+
Multistage Interconnection Networks (MINs)
are capable of supporting arbitrary input-output
interconnections, and have been long proposed as an
interesting alternative to the full crossbar and the timeshared bus (in terms of performance, cost and scalability)
for use as permutation networks for multiprocessors but
also as point-to-point switching fabrics for telecom
applications.
parallel computers
Input
Multistage architecture
switching networks
0000
0001
0010
0011
0100
0101
0110
0111
1000
1001
1010
1011
1100
1101
1110
1111
(4)
E(1)
(4)
E(1)
(4)
E(1)
(4)
Fiber-based Modules vs. Free-Space
(2)
Shuffle inte rconnection
E(1)
0000
0001
0010
0011
0100
0101
0110
0111
1000
1001
1010
1011
1100
1101
1110
1111
Output
Bandwidth, cross-talk and volume consumption
considerations, all advocate for plane-to-plane optical
interconnects within massively interconnected electronic
system as a promising substitute to conventional electronic
circuitry in the near future.
Plane-to-plane guide-wave based optical interconnections
• better efficiency (than holograms) for long-range interconnections.
• No space-invariance required.
• Theoretically more volume efficient than free-space
interconnection
modules
Dense optical interconnect:
• Precise and robust alignment possible…
…an optical “3D optical
wiring” module between
2D VLSI arrays.
interconnection folded in 2D
Column-controlled buffered MIN architecture
Routing Strategy
Conflicts are not resolved individually at the switch level, but globally at the stage level by a “tournament” between all the incoming
requests to that particular stage. Provided that these request are uniformly directed to any possible output, "votes" leading to the adoption of one of
the two possible states of the global-switch will be evenly distributed. Such behaviour takes place for all stages of the network, so that at each
stage, half of the requests will be dropped and half will be able to pass to the next stage. This means there is an enormous number of discarded
packets, certainly much bigger than that occurring by internal blocking in a standard SEMIN. However, if one considers a buffered architecture, then
presumably there will be no need to provide it with a large buffer memory, because the packets that have been retained in the buffers are very likely
to go forward in the following tournament.
• no cross-talk in 3D, just like free-space optics.
 network
(C)
• multiple interleaved permutations possible.
• Maybe “hard” to build? Boring, but not a fundamentally difficult
(can be automated, can be done by “layers”, see below).
module control
• Alignment of both output and input needed.
Optical Multistage
Architecture
Paradigm
As a consequence, multistage architectures
using optical plane-to-plane regular interconnections
may well represent a theoretically optimal architecture
for use within a massively interconnected systems.
Processor
arrays
• Power dissipation may be a fundamental limitation, but we are far
from these limits.
The work presented here develops on some fundamental
aspects of such optical multistage architecture:
Stacking layers
0
4
8
1
5
2
(A) Guide-wave based interconnection modules
6
3
While many demonstrator have been built to illustrate the advantages of free-space optics over electronics for dense plane-to-plane
interconnections, there has been relatively little research on the use of two-dimensional wave-guide-based interconnections, (or guidewave based interconnection modules). Yet, these can easily achieve better transmission efficiency than holographic-based interconnections
while almost completely cancelling cross-talk, and contrary to the common belief they may be more volume efficient than free-space optics for
both space-invariant and space-variant interconnects.
(b)
(a)
Stacking planar lightwave circuits to produce 2D
guided-wave interconnection modules
“Brute” folding and row/column decomposed folding of the
perfect shuffle inter-stage permutation
In the case of “column/row-decomposable”
permutations (see left), which happen to be
the ones required in most regular
interconnected MINs, 2D interconnection
modules can be easily implemented by
stacking layers of printed lightwave circuits
containing 1D permutations (see right).
processing
(analysis/buffer)
bi-permutation
module
Routing parameters
Performance analysis of a buffered GSMIN
(B) Transparent column-controlled Optical Multistage Network
The use of “modular” inter-stage interconnections leads naturally to consider a new paradigm for the optical MIN architecture: interconnection
modules containing several interleaved global interconnections (i.e. permutations). Addressing of the required permutation can be done by
mechanical displacement of the whole multi-permutation module.
Cascading such modules without intermediate optoelectronic arrays gives a transparent ”globally-stage switched” multistage
interconnection network (GSMIN) that can be used as a circuit-switched permutation network for multiprocessor communications (this
architecture represents an original optical implementation of a column-controlled shuffle-exchange MIN).
Experimental results
Input pattern (exit VCSEL array)
Recently, we tested this approach by implementing
several 4x4 fiber-based modules each integrating a
different fixed topology.
Figures represent performance (normalized amount of satisfied requests) as a function of the input load (computed as the probability of a
request being issued at any input per unit time) for the MIN with stage-distributed switching, and for a standard Shuffle Exchange MIN with
independent control of switches (both 128x128 large networks). The input traffic follows the uniform request model.
Alignments
hardware
Output pattern (CCD image)
“Fair” selection mode (the state of the global-switch is chosen randomly if the poll is a draw)
Probability of packet acceptance
Probability of packet acceptance
Output
(CCD)
0.7
0.6
0.5
0.4
0.3
Depth Analysis = 1
0.2
Depth Transfer = 1
0.1
0
0
Mechanically Reconfigurable wave-guide-based interconnections
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
5
0.8
The performance of a "fair"
operated GSMIN is superior to
that of a standard ("fair"
operated) MIN for buffer sizes
larger than one.
0.7
3
0.6
3
0.5
2
1
0
0.4
0.3
2
1
Depth Analysis = Buffer Size
0.2
Also, for a buffer size equal to
three, the GSMIN network
gives better performance than
the crossbar, even at full-load.
Depth Transfer = Buffer Size
0.1
0
1
4
4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0
1
Input request probability per unit time ()
“Alternate” selection mode (the state of the global-switch changes if the poll is even)
Probability of packet acceptance
This performance may exceed that of the
GSMIN, but it seems to increase slowly
(with buffer size), so that when the buffer
size is equal to five (or four in the case of
a 64×64 network), both networks show
roughly the same performance.
inputs
4
4
0.8
3
0.7
3
0.6
0.5
2
0.4
1
0.3
0
2
0.2
1
Depth Analysis = Buffer Size
Depth Transfer = Buffer Size
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0
1
Input request probability per unit time ()
6
5
Prototype using interleaved fibers
Spanned hypercube using Bi-permutation modules
Non-integrated bi-permutation module:
=
{c2, id}
input
output
c3
Channels are multi
mode fibers:
MFD = 9.5  m
Grad diam. 125 m
NA: 0.1
c2
c1
0.9
4
0.8
0.7
3
0.6
0.5
2
0.4
Size: 128 x 128
0.3
1
GSMIN (alternate)
GSMIN (forced alternate)
0.2
Buffer size
c4
Probability of packet acceptance
1
There is no significant change in
performance between a buffered
GSMIN with "alternate" selection
and a buffered GSMIN with "forced
alternate" selection. A continual
"blind" alternation of switching
states performs just as well.
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
4
0.8
3
0.7
3
0.6
2
0.5
2
0.4
0.3
1
Size: 128 x 128
0
1
SEMIN (alternate)
SEMIN/GSMIN (forced alternate)
0.2
0.1
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0
1
Input request probability per unit time ()
Individual control of switches as well as arbitration
may be unnecessary on a standard SEMIN if the
buffer size is larger than four. Also, when the buffer
size is equal to three, the 128x128 GSMIN already
outperforms a 128x128 full-crossbar.
0.1
0
0
0.9
Buffer size
“Forced-alternate” selection mode (the state of the global-switch changes periodically)
Bi-permutation module
(integrates switching and
permutation stage)
6
6
5 5
4
1
0.1
0
0
Mechanically reconfigurable multi-stage architecture based on cascading
multi-permutation modules
Guide-wave based multi-permutation
interconnection module
0.9
Buffer size
outputs
actuators
A small mechanical (or optical )
perturbation produces a drastic
change of the interconnection
pattern from input to output.
A clear improvement in performance is
seen in the standard SEMIN for buffer
sizes larger than one.
When a standard SEMIN and a column-control
SEMIN (GSMIN) are operated in forced alternate
mode, they become strictly equivalent architectures.
Therefore, when a standard SEMIN is operated in
forced alternate mode, its performance roughly
degrades to that of an alternate operated GSMIN; on
the other hand, when a GSMIN is operated in forced
alternate mode, the performance remains
substantially the same.
6
6
5 5
1
Probability of packet acceptance
Multi-permutation interconnection module paradigm
Can have a “direct” optical implementation: the switching stage and its adjacent inter-stage permutation can be physically
merged into a unique “permutation module" providing two different interconnection patterns (bi-permutation switching module).
Column-controlled
Shuffle-Exchange stage
0.8
0.9
Input request probability per unit time ()
Simplicity and Scalability: A whole switching stage needs a unique control signal.
Two implementation of a column-controlled Multistage Interconnection Network
0.9
Buffer size
Why common control of switches per switching stage (column-control)?
Only the older
packets waiting on
the queues are
given attention in
the selection
phase, and that
only these packets
are candidates to
be transferred
during the transfer
phase.
Optimal situation: both “depth
of analysis” and “depth of
transfer” are equal to the
actual buffer size.
66
5
1
Buffer size
(B) Transparent column-controlled Multistage Network
1
Input
(VCSELs)
Column-controlled unbuffered MINs have been extensively studied for circuit-switched applications. We found that an inter-stage buffered
column-controlled MIN architecture may also represent an interesting alternative to the well-known Shuffle-Exchange MINs (SEMINs) for pointto-point packet-routed communications, both from the point of view of its implementation complexity and simplicity of routing protocol.
Automatic alignment of modules, both dynamically
and statically (pre-aligned ”plug-and-play”
exchangeable blocks), is a critical issue now being
studied.
input
(C) Buffered column-controlled Optical Multistage Network
0
1
Input request probability per unit time ()
Experience with two cascaded modules:
a
b
c
A more “direct” Implementation of a column-controlled
Multistage Network using multi-permutation switching
modules (the GSMIN network)
…topology mapped on a plane (optical
interconnects, VLSI integration)
Output
(to CCD)
Displacement
stages
“indirect” implementation obtained by electrically joining
individual switches (common control lines a, b and c) of a
Shuffle-Exchange network (here the Inverse Baseline)
four-dimensional hypercube
connected multiprocessor…
Four-dimensional “spanned”
hypercube network (16 nodes)
using four bi-permutation
modules, each providing a cube
permutation and the identity.
Input
(from VCSEL array)
Exit first
module
{c1, id}
{c2, id}
{c3, id}
{c4, id}
A total of 24=16 global
permutations for the whole
network.
A column-controlled MIN can be an efficient (circuit-switched) permutation network.
Input second
module
Conclusions and further research
Column-control certainly reduces overall interconnection capacity, but if things are properly designed, the MIN architecture can still
accommodate communication primitives of most static interconnection networks. The interest of such an arrangement lies in the ease of control
and straightforward implementation. We presented here preliminary experimental results demonstrating a simple optical architecture using
cascaded fiber-based bi-permutation modules. An electro-mechanical system has been developed providing stage-switching times on the order
of milliseconds, making this architecture suitable for reconfigurable, high-bandwidth inter-processor communications. Further research on this
topic include the feasibility study of dense guided-wave multi-permutation modules by stacking layers of planar lightwave circuits, as well as the
introduction of electro-optical switching.
Displacement pitch for commutation: 125 m
Interconnect N
Interconnect 3
Interconnect 2
Interconnect 1
Interconnect N
Interconnect 3
Interconnect 2
Interconnect 1
Interconnect N
Interconnect 3
Interconnect 2
Interconnect 1
A multistage ”spanned” version
of most direct network
topologies (hypercube, cubeconnected-cycles, etc.) can be
implemented as an unbuffered
column-controlled MIN
architecture.
A time-division multiplexing
(TDM) technique can be used to
select the interconnections at
each stage.
time
Permutation
appearance period
Alignment tolerance: 5 m (half peak power).
Inter-module Coupling Efficiency: 1.7dB (no additional
optics, matching oil or antireflection coating).
switching
region
 Validation of simple cascaded architecture
Electro-mechanical switching device
Possible implementation of an electro-optical bi-permutation
module using stacked layers and electro-optically controlled
normal coupling between the layers.
This electro-magnetic actuated
device can position the module
in both X and Y directions.
Time slot
A column-controlled MIN can emulate a full-crossbar by time-multiplexing
Despite the reduced permutation capacity of a column-controlled MIN, a permutation network can emulate (by time-multiplexing) a fullcrossbar provided that the reconfiguration latency remains small with respect to the interconnection request rate, and that the bandwidth is
sufficiently high to accommodate in a single time slot all the data to be transferred. An unbuffered column-controlled MIN may provide enough
optical bandwidth - but the reconfiguration time may be still too slow (using thermo-optical switches for instance).
The switching speed seems to be limited to the millisecond
range (resonant frequency).
Micro electro-mechanical actuators (MEMS) may also be an
interesting alternative when switching latency in the millisecond
range is tolerable.
First two modules of a spanned version of a 4-dimensional hypercube were
fabricated using interleaved optical fibers, and the resulting four possible
interconnections observed.
From the routing point of view, simulations showed that a buffered column-controlled MIN would not require excessive buffer size to
achieve respectable performances (under uniform traffic). The path-selection mechanism can be further reduced to the simple alternation of the
available stage permutations, without degrading the performance. Under such stage-distributed time-division permutation multiplexing, the SEMIN
and GSMIN fabrics become strictly equivalent routing architectures; hence (provided that buffer size is chosen to be larger than three) this
analysis-free strategy will provide a very simple arbitration mechanism for standard SEMIN networks. This is an interesting result on its own.
This arbitration free mechanism may be very appealing for all-optical networks, if optical buffering functions (e.g. delay lines) could be
integrated on the cascaded modules themselves, an issue worth exploring.