Transcript Innovations In Fabric Technology

March 25 – 27th, 2014 І Orlando, FL
Innovations In Fabric Technology
Srikanth Keesara & Dale Nash
©©2014
2014Avaya
AvayaInc.
Inc.Avaya
Avaya– –Confidential
Confidential& &Proprietary
Proprietary
DoDo
not
not
duplicate,
duplicate,
publish
publish
oror
distribute
distribute
further
further
without
without
the
the
express
express
written
written
permission
permission
of of
Avaya.
Avaya.
#AvayaATF
#AvayaATF
Srikanth Keesara
Avaya Distinguished
Engineer
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
#AvayaATF
Dale Nash
Director of VSP
Software
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
#AvayaATF
March 25 – 27th, 2014 І Orlando, FL
Virtual IST
©©2014
2014Avaya
AvayaInc.
Inc.Avaya
Avaya– –Confidential
Confidential& &Proprietary
Proprietary
DoDo
not
not
duplicate,
duplicate,
publish
publish
oror
distribute
distribute
further
further
without
without
the
the
express
express
written
written
permission
permission
of of
Avaya.
Avaya.
#AvayaATF
#AvayaATF
A Look at a Traditional IST
H1
10
S1
1
10
VSP-A1
VSP-B1
10
4
10
6
10
7
S6
10
H3
5
H2
10
S2
2
10
VSP-A2
101,10
100,10
3
10
VSP-B2
S7
10
H4
 VLAN 100 is the IST VLAN for Switch cluster A1/A2 and a member of the
IST MLT-4
 VLAN 101 is the IST VLAN for Switch cluster B1/B2 and a member of the
IST MLT-5
 VLAN 10 is a user VLAN and a member of
 SMLT-1, SMLT-2, SMLT-3, SMLT-6, SMLT-7, IST MLT-4 and IST MLT-5
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
5
Traditional IST Traffic Flows
H1
10
S1
1
10
VSP-A1
VSP-B1
10
4
10
6
10
7
S6
10
H3
5
H2
10
S2
2
10
VSP-A2
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
101,10
100,10
3
10
#AvayaATF
VSP-B2
S7
10
H4
6
Traditional IST: SMLT Failure Handling
H1
10
S1
1
X 10
VSP-A1
VSP-B1
10
4
10
6
10
7
S6
10
H3
5
H2
10
S2
2
10
VSP-A2
101,10
100,10
3
10
VSP-B2
S7
10
H4
 SMLT-1 fails at VSP-A1
 VSP-A1 redirect traffic to IST MLT
 VSP-A2 receives the traffic over the IST and sends onto SMLT-1
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
7
IST MLT Failure – NOT Supported
Traffic is dropped
H1
10
S1
1
X 10
VSP-A1
VSP-B1
10
10
6
10
7
S6
10
H3
X
4
5
H2
10
S2
2
10
VSP-A2
101,10
100,10
3
10
VSP-B2
S7
10
H4
 SMLT-1 fails at VSP-A1
 IST MLT fails between VSP-A1 and VSP-A2
 This causes traffic loss – since we do not currently handle an
IST failure
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
8
New Service : Transparent Port
4K vlans
H1
S1
1
X
VSP-A1
3
H3
S3
5
100
4K vlans
H2




S2
2
VSP-A2
4
S4
H4
I-SID-1: MLT-1 and MLT-3
I-SID-2: MLT-2 and MLT-4
Both services have 4K VLANs each.
But sending them on IST mixes up the traffic for both services.
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
9
New Service : Switched UNI
vlan-10
H1
S1
1
X
VSP-A1
3
H3
S3
5
100
vlan-10
H2




S2
2
VSP-A2
4
S4
H4
I-SID-1 = VLAN-10 on MLT-1 and MLT-3
I-SID-2 = VLAN-10 on MLT-2 and MLT-4
Both services have vlan-10
But sending them on IST mixes up the traffic for both services.
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
10
March 25 – 27th, 2014 І Orlando, FL
What is a Virtual IST?
It is a SPBM Tunnel between Cluster Switches
©©2014
2014Avaya
AvayaInc.
Inc.Avaya
Avaya– –Confidential
Confidential& &Proprietary
Proprietary
DoDo
not
not
duplicate,
duplicate,
publish
publish
oror
distribute
distribute
further
further
without
without
the
the
express
express
written
written
permission
permission
of of
Avaya.
Avaya.
#AvayaATF
#AvayaATF
Virtual IST
VSP-1
IST Tunnel
SPB
VSP-2
 If there is a path, then Virtual IST is UP
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
12
No More IST Failure
NO Traffic Loss
H1
10
S1
1
X 10
VSP-A1
VSP-B1
10
6
10
7
S6
10
H3
X
4
H2
10
S2
2
10
5
VSP-A2
VSP-B2
S7
10
H4
 SMLT-1 fails at VSP-A1
 Link(s) between VSP-A1 and VSP-A2 fails
 Virtual IST finds a new path and there is no traffic loss
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
13
Virtual IST – Topology Examples
VSP-4
VSP-4
VSP-1
VSP-1
IST
X
VSP-3
IST
VSP-3
VSP-2
VSP-2
VSP-5
VSP-5
VSP-4
VSP-1
X
X
IST
VSP-3
VSP-2
VSP-5
 If there is a path, then Virtual IST is UP
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
14
Provisioning – Virtual IST
VSP-1
1/1
IST Tunnel
SPB
1/1
VSP-2
router isis
….. Usual isis config parameters
system-id 00be.b001.0000
spbm 1 spbm 1 smlt-peer-system-id 00be.b002.0000
spbm 1 spbm 1 smlt-virtual-bmac 00be.b012.0000
vlan create 2 type port-mstprstp 0
vlan i-sid 2 2
interface vlan 2
ip address 2.2.2.1/30
virtual-ist peer-ip 2.2.2.2 vlan 2
router isis enable
router isis
….. Usual isis config parameters
system-id 00be.b002.0000
spbm 1 spbm 1 smlt-peer-system-id 00be.b001.0000
spbm 1 spbm 1 smlt-virtual-bmac 00be.b012.0000
vlan create 2 type port-mstprstp 0
vlan i-sid 2 2
interface vlan 2
ip address 2.2.2.2/30
virtual-ist peer-ip 2.2.2.1 vlan 2
router isis enable
 All other SMLT/RSMLT/SPBM configuration remains unchanged
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
15
Virtual IST Technology : Minimum Requirements
 Virtual IST should be up as long as there is a path in the core network.
 Support for traditional services



Traditional SMLT
Support RSMLT
L2/L3 Unicast and Multicast
 Support SMLT redundancy for Fabric-connect based services





L2VSN
L3VSN
IP Shortcuts
Inter VSN Routing
Multicast over SPB
 Ready to support SMLT redundancy for New Services




Switched UNI
Transparent UNI
ETREE
…
 Support mixing of product types in a switch cluster as long as they support
Virtual IST

Example : A cluster consisting VSP4000 + VSP8200
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
16
What’s in the Future for Virtual IST
Access
Device-4
BEB-C
Access
Device-2
.
.
.
Cluster2-1
Cluster1-1
Cluster2-2
Cluster1-2
SPB
Cluster2-15
Cluster1-15
Cluster2-16
Cluster1-16
BEB-A
IST
.
.
.
Access
Device-1
BEB-B
SMLT
Access
Device-3
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
17
March 25 – 27th, 2014 І Orlando, FL
VENA Packet Engine for the VSP 9000
2nd Generation Line Card
©©2014
2014Avaya
AvayaInc.
Inc.Avaya
Avaya– –Confidential
Confidential& &Proprietary
Proprietary
DoDo
not
not
duplicate,
duplicate,
publish
publish
oror
distribute
distribute
further
further
without
without
the
the
express
express
written
written
permission
permission
of of
Avaya.
Avaya.
#AvayaATF
#AvayaATF
What’s Covered
 2nd Generation Line Card Attributes
 2nd Generation I/O Module Hardware Architecture
 Avaya’s Custom Built VENA Packet Engine for the VSP
9000
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
19
2nd Generation Line Card Attributes
 Based on Avaya’s custom built data path




Implemented with 28nm FPGA technology
Significant processor offloading built into the design
Programmable Network Processor Unit (NPU)
System performance 200Gbps today, 400Gbps tomorrow
 Uses DDR3 memory technology for both data structures
and packet buffering
 Provides extreme scalability
 Ingress-buffered virtual output queues based architecture
 Optimized multicast traffic handling and packet
replication
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
20
March 25 – 27th, 2014 І Orlando, FL
nd
2
Generation I/O Module
Hardware Architecture
©©2014
2014Avaya
AvayaInc.
Inc.Avaya
Avaya– –Confidential
Confidential& &Proprietary
Proprietary
DoDo
not
not
duplicate,
duplicate,
publish
publish
oror
distribute
distribute
further
further
without
without
the
the
express
express
written
written
permission
permission
of of
Avaya.
Avaya.
#AvayaATF
#AvayaATF
Avaya’s Custom Data Path Evolution
• 7 Generations of NPU/Data Path architectures.
400G VENA
Packet
Engine
• Architecture evolved from programmable ASICs to
FPGA acceleration to full FPGA-based solution which
provides unprecedented performance, scalability and
cost containment
400G
28nm
Performance
• VENA Packet Engine provides 200Gbps performance
today and is 400Gbps ready for tomorrow.
RSP2.6 +
Zagros1 +
K2
• RSPng incorporates innovations including DDR3 DRAM
to drive costs down and provide exceptional scalability
90/110nm
RSP 2.6
(ERS)
RSP 2.5
(ERS)
48G
20G
110nm
RSP 2
(ERS)
10G
RSP 1.5
(ERS)
In Design
Available
CXP
‘96
‘97
‘98
‘99
‘00
‘01
‘02
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
‘03
‘04
‘05
#AvayaATF
‘06
‘07
‘08
‘09
5G
‘10
‘11
‘12
‘13
‘14
22
2nd Generation I/O Modules Overview
10GE
40GE
No. of Slices
3
3
No. of User Ports
48
12
240 Gbps
240 Gbps
2:1
2:1
SFP+
QSFP
No
Yes
Yes *
No
Board level
Board level
BW Throughput (Full Duplex)
Oversubscription Ratio
Optics
Port Striping
MAC Sec. support
Power control granularity
‘*’ – available in a future release via software upgrade
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
23
2nd Generation I/O Module 3-Slice Architecture
48-Port 10GE and 12-Port 40GE
User I/O
CTL
CTL
ZIP
SLICE 0
PCI-e
ZIP
PCI-e
CTL
CTL
SLICE 1
ZIP
SLICE 2
PCI-e
24x6.25G
SERDES
XBAR Fabric
User I/O
User I/O
24x6.25G
SERDES
XBAR Fabric
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
ZIP
PCI-e
COP
Block
24x6.25G
SERDES
XBAR Fabric
24
VENA Packet Forwarding Engine
Architecture
User I/O
8x10GE
2x40GE
CTL
User I/O
8x10GE
2x40GE
DDR-3
DDR-3
DDR-3
DDR-3
DDR-3
PHY/Optics
PHY/Optics
DDR-3
8x10.3G
SERDES
CTL
8x10.3G
SERDES
HAB
HAB
Zagros 2
CIF
ZIP
RSPng
CIF
Zagros 2
TCA
M
ZIP
TAM
HGX
HGX
PCI-e
HGX
HGX
Fabric Node
(QE 80K)
DDR-3
DDR-3
DDR-3
DDR-3
DDR-3
DDR-3
24x6.25G
SERDES
XBAR Fabric
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
25
March 25 – 27th, 2014 І Orlando, FL
Avaya’s Custom Built VENA
Packet Engine
©©2014
2014Avaya
AvayaInc.
Inc.Avaya
Avaya– –Confidential
Confidential& &Proprietary
Proprietary
DoDo
not
not
duplicate,
duplicate,
publish
publish
oror
distribute
distribute
further
further
without
without
the
the
express
express
written
written
permission
permission
of of
Avaya.
Avaya.
#AvayaATF
#AvayaATF
Key Technology Highlights
 400G Ready FPGA-based flexible and programmable data path,
implemented in 28nm technology, with a programmable network processor
 Uses low-cost DDR3 memory for both data structures and packet buffering
giving the system unprecedented scalability
 4 GByte of DDR3 memory available to the forwarding engine
 2 GByte of memory for the control plane processor
 Ingress-buffered virtual output queue
 Optimized multicast packet replication which copies the packets as late in
the processing cycle as possible
 Soft programmable packet classification engine enabling the system to
adapt to changing standards with only a software upgrade
 Flow aware deep packet inspection at line rate
 QOS aware oversubscription management at line rate
 Packet classification achieved with information from L2, L3 & L4 headers
 Hardware assisted data collection engine which enables the system to
harvest statistics without effecting the packet forwarding or control processor
performance
 Guaranteed delivery of critical in-band control packets in the presence of
congestion and oversubscription
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
27
The Resultant Product Capabilities
 4 GBytes of low-cost DDR3 memory provides unprecedented system
scalability:





256K L2 MAC Records
1M L3 FIB Entries
24K IPMC streams
32K IPFIX Flows per forwarding engine; up to 1M flows per chassis
Up to 1024 MLT groups
 Optimized multicast packet replication which is capable of duplicating up to
80 Mpps
 Packet Classification Engine capable of processing 165Mpps per
classification engine
 Industry leader in resiliency and fast failover technology




Multi-fault tolerant MLT/DMLT/SMLT/ECMP/VLACP/SPB sub 10ms
Rapid Failure Detection and Recovery of Data Path (RFDR < 16ms)
Hardware-based control plan overload and loop protection
Large scale switch clustering using Split MLT and Shortest Path Bridging
 Unique in the industry for Lossless Ethernet without any hardware upgrade
 Unique in the industry for predictive congestion management for storage
traffic
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
28
Improvements over 1st Generation Line Cards
 Provides 4x performance
 Reduces Watts/Mpps significantly
 Provides hardware support for
higher density 10GigE and 40GE
clear channel and striped
 Significantly increased system
scalability and performance
 Integrates I/O MAC's and packet
classification functions
 Provides 2x10GE port density: 24
to 48 ports;
 Provides 12x40GigE ports
 Improves Fast Failover Technology
 Adds dedicated hardware support
for SMLT/SPB packet processing
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
29
March 25 – 27th, 2014 І Orlando, FL
Component Overview
Preclassifier
©©2014
2014Avaya
AvayaInc.
Inc.Avaya
Avaya– –Confidential
Confidential& &Proprietary
Proprietary
DoDo
not
not
duplicate,
duplicate,
publish
publish
oror
distribute
distribute
further
further
without
without
the
the
express
express
written
written
permission
permission
of of
Avaya.
Avaya.
#AvayaATF
#AvayaATF
Packet Pre-classification Engine Overview
 Provides line rate packet classification functionality to offload the
NPU
 Rule based classification engine with TCAM hardware assist
 Programmable HASH based VLAN Policy hardware assist
 Line rate throughput with the ability to process a packet every clock
cycle; resulting in 165Mpps throughput capability per classification
engine
 Capable of looking deep into the packet to allow for first pass packet
classification for even the most difficult packet flows (MAC-in-MAC;
Q-in-Q; L2 & L3 VSN SPB applications; etc)
Soft programmable packet classification engine enabling the
system to adapt to changing standards with only a software
upgrade
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
31
Packet Preclassification Engine Capabilities















Policy based VLAN classification
Double VLAN tagged packets
STG filter
VLAN Port membership filter
Potential VLAN membership check
SPB RPF check
SPB NNI Termination check
Multiple B-VID Match
Local MAC SA Learn check
Router MAC DA check for UNI and
C-MAC DA
MAC SA/DA black list check
IST Filter
Data packet QoS classification
Port-based policers and rate limiters
Single rule-based packet capture
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
 IFP classification actions:
#AvayaATF








Increment rule counter
Set Packet Type
Set control packet QoS
Set RSP code entry point
Set packlet forward size to RSP
Set the Test Flag
Drop packet–enables Port-based
Counter Group ID
Set Port-based Counter Group ID
32
March 25 – 27th, 2014 І Orlando, FL
Component Overview
Zcore
©©2014
2014Avaya
AvayaInc.
Inc.Avaya
Avaya– –Confidential
Confidential& &Proprietary
Proprietary
DoDo
not
not
duplicate,
duplicate,
publish
publish
oror
distribute
distribute
further
further
without
without
the
the
express
express
written
written
permission
permission
of of
Avaya.
Avaya.
#AvayaATF
#AvayaATF
Zcore Component Objects
 Packet Pre-Classification
Engine
 All Packet Replication &
QOS Buffering &
Queuing Mechanisms
(WRR & Priority)
 Statistic Collection with
Hardware assisted
harvesting
 Command Process
Handling from the RSPng
IOML
Preclassifier
(IHP)
ZCore
ETP
ESB
TCAM
RSPng
PMM
TPU
ZFA
EHP
DDR3
FNIL
ZCore does all Pre & Post Packet Processing for the RSPnp
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
34
March 25 – 27th, 2014 І Orlando, FL
Component Overview
RSPng
©©2014
2014Avaya
AvayaInc.
Inc.Avaya
Avaya– –Confidential
Confidential& &Proprietary
Proprietary
DoDo
not
not
duplicate,
duplicate,
publish
publish
oror
distribute
distribute
further
further
without
without
the
the
express
express
written
written
permission
permission
of of
Avaya.
Avaya.
#AvayaATF
#AvayaATF
RSPng Component Objects
 Packets can be sent to
either the SAM or the AM
 One highly multi-threaded
pipelined 256-bit
Streamlined Analysis
Machine
 One 64-bit Programmable
Analysis Machine with 64
threads
 Foreground & Backgound
processing models
IOML
Preclassifier
RSPng
PPU
HAP
ZCore
SAM
TCAM
CIF
AM
FNIL
All packet forwarding decisions are made by the RSPng
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
36
March 25 – 27th, 2014 І Orlando, FL
Component Overview
Sierra Script Engine
©©2014
2014Avaya
AvayaInc.
Inc.Avaya
Avaya– –Confidential
Confidential& &Proprietary
Proprietary
DoDo
not
not
duplicate,
duplicate,
publish
publish
oror
distribute
distribute
further
further
without
without
the
the
express
express
written
written
permission
permission
of of
Avaya.
Avaya.
#AvayaATF
#AvayaATF
37
Hardware Assisted Statistics Data Collection:
Avaya’s Script Engine
 2nd Generation Line Cards provide enhanced hardware assistance for the
mundane (yet very important) job of collecting status/statistics and system
performance information
 MAC, Filter, KHI, Port, Application and IPFIX Flow information is harvested
through this engine
Avaya’s Script Engine provides unprecedented performance visibility,
with minimal system overhead
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
38
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
#AvayaATF
BEST OF ATF
SPEAKER AND TEAM AWARD
BE SURE TO
TWEET YOUR FEEDBACK
ON THIS PRESENTATION
#AvayaATF
Winners will be announced at closing of event
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
40
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF