Transcript Control Update - Applied Research Lab

Supercharged PlanetLab Platform,
Control Overview
Fred Kuhns
[email protected]
Applied Research Laboratory
Washington University in St. Louis
[email protected]
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
Fast Path/Slow Path Architecture
Remote Login Interface
Fred Kuhns - 7/7/2015
exception packets
& in-band control
Application
Manager
shared
server
Control Interface
Parse
Lookup
Fast Path
Hdr
Format
Queue
Manager
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
...
output
interfaces
Filters
...
input
interfaces
...
exception packets
& in-band control
out-of-band
control
2
Prototype Organization
Switch
RTM
GPE
LC
NPE
DRAM
external interface
Key
Extract
(2 ME)
ingress side
egress side
ExtTx
(2 ME)
Queue
Manager
(2 ME)
Lookup
(2 ME)
Hdr
Format
(1 ME)
Queue
Manager
(2 ME)
IntTx
(2 ME)
TCAM
Hdr
Format
(1 ME)
SRAM
Lookup
(2 ME)
Rate
Monitor
(1 ME)
SRAM
SRAM
Key
Extract
(1 ME)
IntRx
(2 ME)
switch interface
SRAM
SRAM
ExtRx
(2 ME)
GPE
DRAM
SRAM
SRAM
Rx
(1 ME)
Key
Extract
(1 ME)
Lookup
(1 ME)
Hdr
Format
(1 ME)
Queue
Manager
(2 ME)
Tx
(1 ME)
TCAM
DRAM
• One NP blade (with RTM) implements Line Card
– separate ingress/egress pipelines
• Second NP hosts multiple slice fast-paths
– multiple static code options for diverse slices
– configurable filters and queues
• GPEs run standard Planetlab OS with vServers
Fred Kuhns - 7/7/2015
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
3
Development System
• Network Processor blades
– dual IXP 2850 NPs
• 3xRDRAM, 3xSRAM, TCAM
– dual 10GE interfaces
– 10x1GE IO interfaces
• General purpose blades
– dual Xeons, 4xGigE, disk
• 10 Gb/s Ethernet switch
– VLANs for traffic isolation
Fred Kuhns - 7/7/2015
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
4
System Block Diagram
External Interfaces
SPP Node
GPE
GPE
PCI
…
LC
xscale
xscale
…
interface
X
NPU-B
NPU-B
TCAM
2x1GE
NPU-A
xscale xscale
PLab
OS
LC
NPU-A
NPE
RTM
2x1GE
NPE
RTM 10 x 1GbE
…
X
Fabric Ethernet Switch (10Gbps, data path)
Base Ethernet Switch (1Gbps, control)
Control Processor (CP)
Global Node manager (GNM)
Resource DB
myPLC*
“Extern” GPEs
user login info
I2C
(IPMI)
Shelf manager
Global Resource Manager (GRM)
Fred Kuhns - 7/7/2015
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
5
Default Traffic Configurations
external interface PE
NPE
Control messages sent over
an
to fabric and base
isolated base Ethernet switch.
(additional GPEs)
For
isolation
andNAT
security
Line
card
performs
like function for traffic
from vservers.
…
3
4
x
GPE
LNM
LRM
MP
root context
planetlab OS
2
x
1
x
5
x
x
10GbE (fabric, data)
1GbE (base, control)
6
x
Substrate
LC
mux
CP
user login info
GNM
Default: traffic forwarded to
CP over 10Gbps Ethernet
switch (aka fabric)
PLC
Fred Kuhns - 7/7/2015
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
GRM
Resource DB
myPLC*
6
Logging Into a Slice
PE
NPE
GPE
LNM
Host
…
(located within node)
LRM
MP
root context
planetlab OS
4
x
Once authenticated, session
forwarded
to appropriate
3
2
x
GPE and vserver.
5
x
1
x
x
10GbE (fabric, data)
1GbE (base, control)
6
x
Substrate
LC
mux
CP
fwder
user loginsshinfo
GNM
ssh connection directed to
CP for user authentication
PLC
Fred Kuhns - 7/7/2015
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
GRM
Resource DB
myPLC*
7
Update Local Slice Definitions
PE
NPE
GPE
LNM
Host
…
(located within node)
LRM
MP
root context
planetlab OS
3
4
x
2
x
1
x
5
x
x
10GbE (fabric, data)
1GbE (base, control)
6
x
Substrate
LC
mux
CP
user login info
Resource DB
GNM
retrieve/update slice
descriptions
PLC
Fred Kuhns - 7/7/2015
update local database,
allocate slice instances
GRM
(slivers) to GPE nodes
myPLC*
slices
...
slices
slices
...
slices...
...
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
8
Creating Local Slice Instance
create new slice
retrieve/update slice
descriptions
NPE
PE
GPE
slices ...
LNM
Host
…
(located within node)
LRM
MP
root context
planetlab OS
3
4
x
2
x
1
x
5
x
x
10GbE (fabric, data)
1GbE (base, control)
6
x
Substrate
LC
mux
CP
user login info
GRM
Resource DB
GNM
myPLC*
PLC
slices
...
slices
slices......
Fred Kuhns - 7/7/2015
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
9
Allocating NPE (Creating Allocate
Meta-Router)
OpenNPE
localsliver
socket{code
for
FP - fast path
NPE
SRAM
Control Interface
TCAM
PE
GPE
LNM
...
tbl
Host
exception
and local
option, SRAM,
delivery
traffic; return
Interfaces/Ports,
etc} to
client vserver
lkup
(located within node)
…
Fast Path
FPk
LRM
MP
root context
planetlab OS
3
4
2
x k
VLAN
x
5
x
Forward request to
Returns
and
globalstatus
resource
assigned
global Port
x
manager
number
1
x
10GbE (fabric, data)
1GbE (base, control)
6
x
Substrate
LC
mux
MI1
CP
user login info
GNM
resources,
associate
global
UDP with
port new
PLC Allocate shared NPE Allocate
slice fast path {SRAM
block;
# filter
table
entries; #
Allocate
and
Enable
VLAN
for
requested
interface(s);
of queues queues; # of
buffers;
code
to packet
isolate
internal
configure
Line slice
card.option;
traffic, VLAN
amount of SRAM required;
total kreserved
bandwidth}
Washington
Fred Kuhns - 7/7/2015
WASHINGTON UNIVERSITY IN ST LOUIS
GRM
Resource DB
myPLC*
10
Managing the Data Path
• Allocate or Delete NPE
Slice instance
• Add, remove or alter
filters
– each slice is allocated a
portion of the NPE’s
TCAM
• Read or write to per slice
memory blocks in
SRAM
– each slice is allocated a
block of SRAM
• Read counters
– one time or periodic
• Set Queue rate or
threshold.
• Get queue lengths
Fred Kuhns - 7/7/2015
NPE
SRAM
TCAM
GPE
LNM
DP
DP
FPllk
LRM
MP
root context
planetlab OS
2
1
x
x
10GbE (fabric, data)
1GbE (base, control)
6
x
CP
user login info
GNM
GRM
Resource DB
myPLC*
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
FP - fast path
11