Transcript Nextra in the Net Economy - Swiss Network Operators Group
Experiences with Implementing MPLS/VPN Services
Philip Bridge Nextra (Schweiz) AG 18.10.2000
Some Paraphrases
• ‘For the first time in the history of the Internet the transmission people are giving the network people more bandwidth than they know what to do with’ –
Peter Lothberg
• ‘If you aren’t scared, you don’t understand’ –
Mike O’Dell
4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
2
MPLS-based VPN
Layer-2 Switching IP Backbone ‘Private’ Internet IP Routing
4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
3
MPLS
Label Switch Router (LSR)
4/30/2020
Provider Edge Router (PE)
Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
4
MPLS
• IP Routing creates consistent, network-wide Routing Tables 4/30/2020 PE LSR Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
IP Routing Protocol (OSPF, IS-IS) 5
MPLS
• Label Distribution Protocol runs in parallel to IP routing • LSRs use LDP to swap IP Route-to-Label bindings • Creates Label Forwarding Tables Label Distribution Protocol 4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
6
MPLS
• LDP & IP Routing create Label Switch Paths between PE routers Local label 3 X Remote label 5 3 IP prefix 3.3.3.3
10.0/16 o/p interface C A Local label 5 Remote label X IP prefix 3.3.3.3
LSP
2) I reach 3.3.3.3
via int C, label 5 o/p interface A Local label X Remote label 3 X 3 IP prefix 3.3.3.3
10.0/16 o/p interface F A 4/30/2020 4) I reach 3.3.3.3
via int F, label 3
F
3) To reach 3.3.3.3
via me use label 3
C A
3.3.3.3
1) To reach 3.3.3.3
via me use label 5 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
7
MPLS
• PE routers ‘encapsulate’ IP packet with Label header • Works an any layer-2 (Ethernet, WAN link, ATM…) Local label
3
Remote label
5
X 3 IP prefix 3.3.3.3
10.0/16 o/p interface
C
A Local label 5 Remote label
X
IP prefix
3.3.3.3
o/p interface
A
Local label X Remote label
3
X 3 IP prefix
3.3.3.3
10.0/16 o/p interface
F
A 4/30/2020
5
3.3.3.3
Label
3
IP DA 3.3.3.3
IP Data IP Packet
F
3.3.3.3
IP Packet Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
C A
3.3.3.3
8
MPLS
• Normal IP ‘ connectionless ’ routing builds layer-2 LSP ‘ circuits ’!
• LSPs take same path that would be taken by IP-based forwarding 4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
9
MPLS & Network Scalability
• LDP & OSPF IP routing build LSPs between PE routers 4/30/2020 2) To reach 3.3.3.3
via me use label 3 3) I reach 3.3.3.3
via int F, label 3
F
2.2.2.2
1) To reach 3.3.3.3
via me use label 5
C
3.3.3.3
A
Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
10
MPLS & Network Scalability
• PE-PE LSPs used to build BGP session between PE routers • Core LSRs do not have any BGP routing 4/30/2020
LSP between BGP endpoints C F
2.2.2.2
BGP Session
3.3.3.3
A
Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
11
MPLS & Network Scalability
• BGP tells each PE which remote PE for each external route • Recursive lookup in the routing table to find a route to the remote PE • Route to the remote PE is actually a LSP
LSP between BGP endpoints
5) BGP neighbor 3.3.3.3 has a route to 10.0/16...
4/30/2020 6) …so to reach 10.0/16 I use int F, label 3
C A 10.0/16 F
2.2.2.2
BGP Session
3.3.3.3
4)
BGP:
I have a route to 10.0/16 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
12
MPLS & Network Scalability
• BGP route exchange between PEs, no BGP in Core • LSPs only established between BGP session endpoints • A few LSPs carry 1000’s of Edge routes between PEs • Complex routing can be ‘pushed out’ of the Core to the Edges 4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
13
MPLS & VPNs
• BGP Edge Routes are hidden from the Core • IP addresses of data packets are hidden from the Core • Overlapping ‘address families’ can share the Core…VPNs!
4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
14
Virtual Routers
• PE-PE BGP hides Customer routes from Core • MPLS hides IP addresses of packets from Core • But routes and and addresses are still visible in PE routing table!
4/30/2020
B 10.0/16 C F
2.2.2.2
BGP Session
3.3.3.3
A 10.0/16
Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
15
Virtual Routers
• Solution is to assign a ‘Virtual Router to each Customer port • Routing Tables in Virtual Routers are invisible to each other • Cisco name: Virtual Routing and Forwarding Instance (VRF) 4/30/2020
B 10.0/16 C F
2.2.2.2
BGP Session
3.3.3.3
A 10.0/16
Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
16
Layer 2 Labels
• Extend BGP to carry L2 labels and routes • PE uses 2nd Level Label (L2) to distinguish between VPNs 3) To reach
10.0/16
via 3.3.3.3 I use L2 10 1) BGP:To reach
10.0/16
via me use L2 10 2) BGP: To reach via me use L2 12
10.0/16
4) To reach Next Hop 3.3.3.3
I use int F, L1 label 3 4/30/2020
B 10.0/16 C F
2.2.2.2
BGP Session
3.3.3.3
A 10.0/16
5) …so to reach
10.0/16
I use int F, L1 label 3, L2 label 10 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
17
Layer 2 Labels
• 1st Level Label gets packets to correct destination PE • Destination PE strips 1st Level Label (L1) from incoming packets • Destination PE uses 2nd Level Label (L2) to forward incoming packets to correct VRF 1) Packets arriving with L2 10 are for
red VRF
2) Packets arriving with L2 12 are for
blue VRF B 10.0/16 C F
2.2.2.2
BGP Session
3.3.3.3
A 10.0/16
4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
18
VPN Forwarding
• Packet arrives on interface E = • L1 removed red VRF • PE looks up route to 10.0.0.1 in VRF Routing Table • BGP route from 3.3.3.3 gives L2 = 10 • Recursive lookup on Next Hop 3.3.3.3 gives L1 = 3 • Packet label switched through Core to 3.3.3.3
• L2 tells PE router to treat packet according to red VRF • L2 removed, packet forwarded out of interface A
3 10 10.0.0.1
IP Packet L1
3
4/30/2020 L2 IP DA
10 10.0.0.1
IP Data IP Packet
E F 10 10.0.0.1
IP Packet
10.0.0.1
IP Packet Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
C 10 10.0.0.1
IP Packet
A
3.3.3.3
10.0.0.1
IP Packet 19
MPLS & VPNs
• MPLS VPNs are like ‘Private Internets’ • Internet protocols work within the VPN • Easy to understand - similar to Frame-Relay • Compliments Tunnel-based (IPSec) VPNs 4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
20
IP Backbone Sharing
‘Private Internet’ ‘Private’ Internet
4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
21
Frame Relay Comparison
4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
22
Frame Relay Comparison
‘Private’ Internet
4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
23
MPLS & VPNs
• Customers A and B want their own VPNs, and an ExtraNet • Customer A wants to connect his VPN to the Internet • Customer B does not trust security of Customer A...
Internet Service
VPN-B Extra Net VPN-A 4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
24
Experience with MPLS VPNs
• Customer VPNs in service for >1 year.
• Several dozen VPNs • Sizes between 2 and dozens of sites • Average size of ca. 10 sites.
• 50:50 mixture of managed/unmanaged CPE • Many VPNs have fixed and Dial-Up Access 4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
25
Experience with MPLS VPNs
• Many VPNs are combined with Internet Access Service, and a large proportion of these with managed Firewall services.
• From the Customer perspective, an MPLS VPN ‘looks’ different from a classical IP network. This has to be explained.
– strange traceroutes – discontiguous routing domains 4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
26
Experience with MPLS VPNs
• MPLS labels increase the length of a packet. Can be a problem with Ethernet equipment from some vendors. Was a big problem for us at the beginning. Is still a problem for IPsec VPNs • Equipment vendor MPLS/VPN implementations are reliable. Still some small bugs and missing features, but nothing that can’t be worked around 4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
27
Experience with MPLS VPNs
• MPLS/VPN is a very powerful paradigm for building an infrastructure that can deliver rich set of Services very flexibly and very rapidly • It is a simple concept, but there are so many implementation possibilities that complexity can (very) easily can get out of control 4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
28
Experience with MPLS VPNs
• Tools to properly manage VPNs are still lagging way behind the network functionality • Available tools restrict the ability of a Service Provider to innovate and develop solutions that are unique • GUIs for provisioning are OK, but the real problem is fault resolution…about 5-10 times more difficult to resolve problems than normal IP-based ISP networks 4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
29
Experience with MPLS VPNs
• It is crucial to impose a structured, modular Service model onto the VPN architecture from the beginning – Helps Salesmen and Customers to understand what can and cannot be done – Helps implementation team to configure the solution – Helps NOC to trouble-shoot – Reduces load on 3rd level pre-sales & support 4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
30
Next Challenges
• Integration of MPLS/VPN and DiffServ QoS • Multi-provider/Multi-AS extension of MPLS/VPN based Services • Traffic Engineering 4/30/2020 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999
31
Thank You!
http://www.swinog.ch/swinog1/presentations.html