Transcript Use of BGP and MPLS VPNs: A Case Study

Use of BGP and MPLS VPNs:
A Case Study
Fred P. Baker
CCIE#3555
Contents
•
•
•
•
•
Current Network
The MPLS VPN project
Routing Objectives
What we did
How we tested
Current Network
Current Environment
• Hub and spoke to 4 data centers
– Sites do not in general connect to 2 data centers due
to cost and OSPF issues
• Generally place servers by geography
– You servers are in the data center your links are in
• Mostly Frame Relay to ATM interworking with
some private lines
– 70 of some 350 remote sites have 2 links
• ATM PVC dual mesh between the data centers
• 12000 agent location network done by MCI with
combination of DSL and Fractional T1
Address Space
• 10.0.0.0/8
– Mostly inside
– Some BP
• 192.168.0.0/16
– Used all over
• 172.16.0.0/12
– Extranet
• 167.127.0.0/16
– Public address space
– Used mostly by extranet
– Some legacy inside
Core
• ATM PVCs
• 2 10meg between each pair of data
centers
• 2 routers on the core
• So 2 meshes
Allstate Core
IPC@ H
LAN
HO LAN
rt6
rt5
rt1
rt6
rt1
rt7
IPX@D
LAN
rt1
rt7
ADC
LAN
10.0.0.0 address allocation
/11 for core 1 per data center
HO IP Address
Range
10.32.x.x - 10.63.x.x
IPC@ H
IP Address Range
10.64.x.x - 10.95.x.x
rt6
rt1
rt5
rt6
Core
IP Address Range
10.0.x.x - 10.31.x.x
rt1
rt7
IPX@D
IP Address Range
10.128.x.x - 10.159.x.x
rt1
rt7
ADC
IP Address Range
10.96.x.x - 10.127.x.x
Allstate Data Center
OSPF
Remote Site
Total Stub
Switching
Router
Data Center
Areas
Si
Switching
MSFC
Core
Switch
Access
MSFCs
AREA 0
Si
Si
Distribution
Switch
Distribution
MSFC
Core
Router
Core
Router
OS/390 VIPA
Total Stub Area
DLSW
Distribution
Router
WAN Core
Area 0
Core
Switch
Si
Agent BB
Total Stub Area
OSPF
Remote Site
Total Stub
Si
Switching
Router
OSPF
Remote Site
Total Stub
ATM/FR
EIGRP
Dual DC sites
OSPF
Remote
Site
Core Router: Communicates between Data Centers
Total Stub
Switching Router: Talks to other routers
Distribution Router: Talks to other networks/Routing domains Direct Connect
Access Router: First hop router
Routing Protocol
•
•
•
•
Single OSPF AS
Cisco and OS/390 based routers only
Firewalls now static routed
Peer authentication soon
Remote sites
•
•
•
•
AT&T frame relay at the site
ATM into the data center
Some ISDN backup
A remote site is connected to a single data
center (for now)
• Servers and applications tend to have
geographic affinity
Remote Site
Ethernet
Standard Access
IPC@D LAN
Standard
Access
with ISDN
swdc-allrt1
Ethernet
Premium
swdc-allrt7
Premium
Access
Dual Rtr
Dual WAN
Mult DC
Core
Premium Connection
Core
Connection
Enhanced Access
Ethernet
Premium
Ehanced
Access
Dual Rtr
Dual WAN
adc-all-rt7
ADC LAN
Enhanced Access
adc-all-rt1
Remote Site Switch Layer
Layer 2 Network with
Spanning Tree
Si
Cost 4
Si
RP
Gig Fiber
RP
Gig Fiber
Trunk with Vlan 1, 2
MDF #1 Cost 4
Spanning Tree Root
Bridge Priority 100
DP
Cost 3019
Trunk with Vlan 1, 2
DP
DP Cost 4
DP
VLAN 1
X
Cost 4 MDF #2
Backup Root
DP
Bridge Priority 200
Cost 19 DP
DP
Cost 4
Trunk with Vlan 1, 2
Cost
DP 19
RP
Cost 4
Cost 3019
RP
VLAN1
X
RP
RP
Cost 3019 X
VLAN2
Agent Broadband
•
•
•
•
•
10,000 locations
Connected via IPSEC VPN
WorldCom managed routers
NO split tunneling
IPSec Transport with GRE tunnel to Dallas
and Hudson
• Agent PCs are 10.*.*.*
• Agent access is via Allstate Internet Proxy
Overview
Hub VPN Router Redistribution
BGP AS
EIGRP AS 519
EIGRP AS 519
BGP AS
Hub site routers to not peer in EIGRP AS 519. VPN
routers only peer with DS routers at the hub site
through eBGP.
192.168.3.0/24
e2 .1
Downstream Router Redistribution
OSPF Process
BGP AS
BGP AS network
STATIC Routes
e0 .1
e1 .1
192.168.2.0/24
All redistribution is controlled by route-map
statements filtering only desired routes.
e1 .2
Lakewood
back-end
LB0 10.12.0.69/30
192.168.1.0/24
erieIntranet
e0 .2
Sandusky
back-end
OSPF
AREA 0
LB0 10.12.0.45/30
e1 .241
e0 .241
10.66.2.240/28
10.128.2.240/28
e0 .245
LB0 10.12.0.85/30
Lakewood DS1
OSPF NSSA TS
AREA 161
e2 .131
e2 .3
LB0 10.12.0.89/30
Lakewood DS2
e1 .130
e1 .2
10.151.212.0/25
f0/0 .4
e0 .246
eBGP
f0/1 .132
LB1 192.168.24.129/32
LB2 192.168.24.254/32
LINCOLN (DSL)
Nauticus
(Allstate Lab Mockup)
Current Solution
e2 .131
f0/0 .4
f0/1 .133
LB0 10.12.0.65/30
Sandusky DS2
e3 .130
eBGP
f0/1 .132
LB1 192.168.24.1/32
LB2 192.168.24.126/32
TR (DSL)
LB1 192.168.24.130/32
LB2 192.168.24.253/32
KITTY HAWK (T1)
e1 .246
e2 .3
e0 .2
10.88.212.0/25
10.151.212.128/25
f0/0 .5
OSPF NSSA TS
AREA 160
e1 .245
LB0 10.12.0.61/30
Sandusky DS1
f0/0 .5
10.88.212.128/25
f0/1 .133
LB1 192.168.24.2/32
LB2 192.168.24.125/32
IKE (T1)
WorldCom ITSO Internet LAB Connectivity
DSL Sites
T1 Sites
SPOKE
EIGRP
AS 519
AVENGER
10.160.1.1/26
DEFENDER
10.160.2.1/26
DEVESTATOR
10.160.3.1/26
BURKE
10.162.161.1/26
RAMAGE
10.162.162.1/26
STOUT
10.162.163.1/26
Agent Broadband in Data Center
Cisco Systems 7206VXR
VPN Routers
Loopback 0 /32 from
192.168.24.0/24 Network
U85515C35-NOT INSTALLED
FE2/0
2/35
2/33
U85515C33-NOT INSTALLED
FE2/0
2/33
2/31
U85515C31-NOT INSTALLED
FE2/0
2/31
2/29
FE2/0
U85515C29-NOT INSTALLED
U85515C25-NOT INSTALLED
FE2/0
2/25
2/23
U85515C23-NOT INSTALLED
FE2/0
2/23
U85515C21-NOT INSTALLED
2/21
U85515C19-NOT INSTALLED
2/19
POS1/1
.109
GE5/0
.1
U85515C17-CTALLSTAAFUS
FE0/1 .27
FE0/0 .155
2/49
U85515C13-CTALLSTAABUS
FE0/1 .25
2/13 .13 FE2/0
LB0 .50
FE0/0 .153
AS 65033
HSRP Standby Group 1 =.3 LB0 .48 AS 65031 FE0/0 .152
U85515C9-CTALLSTA09US
HSRP Standby Group 2 = .67
.110
Cisco Systems
12008
U85515TS1
2/48
2/47
2/50
FE0/0 .6
FE0/1 .70
Host Table Loopback
192.168.24.254
Access to console for
all U85515 equipment
and U85514TS2.
.11 FE2/0
LB0 .46
2/7
U85515C7-CTALLSTA07US
FE0/1 .22
.10 FE2/0
LB0 .44
FE0/0 .150
2/5
U85515C5-CTALLSTA05US
FE0/1 .21
.9 FE2/0
LB0 .425
FE0/0 .149
2/3
U85515C3-CTALLSTA03US
FE0/1 .20
.8 FE2/0
LB0 .40
FE0/0 .148
2/1
U85515C1-CTALLSTA01US
FE0/1 .19
.7 FE2/0
LB0 .38
FE0/0 .147
2/2
2/49
2/47
2/48
FE0/1 .5
FE0/0 .69
U85514TS2
AS 65027
AS 65025
AS 65023
C is c o 1 2 0 0 0 c s e rie s
GE5/0
.2
Primary
OC-3/155Mbps
POS2/0
AS 65003
U85514C4-CTALLSTA04US
FE0/1 .2
LB0 .4
FE0/0 .131
.72 FE2/0
2/6
U85514C6-CTALLSTA06US
FE0/1 .3
LB0 .6
.73 FE2/0
FE0/0 .132
2/8
LB0 .8
.74 FE2/0
AS 65005
AS 65007
U85514C8-CTALLSTA08US
FE0/1 .4
FE0/0 .133
AS 65011
HSRP Standby Group 1 =.3
HSRP Standby Group 2 = .67
GE6/0
.65
AS 65009
U85514C10-CTALLSTAAIUS
FE0/1 .5
2/10
LB0 .10
.75 FE2/0
FE0/0 .134
2/12
U85514C12-CTALLSTAAAUS
FE0/1 .6
LB0 .12
.76 FE2/0
FE0/0 .135
2/14
U85514C14-CTALLSTAACUS
FE0/1 .7
LB0 .14
.77 FE2/0
FE0/0 .136
2/16
LB0 .16
.78 FE2/0
.34
2/50
Cisco Systems
12008
2/18
2/15
2/10
2/12
2/13
2/14
2/16
2/11
2/18
2/49
2/20
2/9
2/22
2/24
.252
GE1/0
GE2/0
.125
U85515DS1
AS 65001
AS 65013
AS 65015
U85514C16-CTALLSTAAEUS
FE0/1 .8
FE0/0 .137
AS 65017
U85514C18-CTALLSTAAGUS
FE0/1 .9
LB0 .18
.79 FE2/0
FE0/0 .138
AS 65019
U85514C20-NOT INSTALLED
2/20
FE2/0
2/7
2/26
swdc-mdfrsw3
loopback 0
10.12.0.77
2/28
2/5
2/30
2/32
2/3
2/34
2/36
2/1
2/2
2/35
OSPF
NSSA
NO SUMMARY
AREA 160
2/50
2/9
2/16
2/7
2/5
2/18
2/3
2/1
2/20
2/22
2/24
2/26
2/26
U85514C28-NOT INSTALLED
2/28
FE2/0
2/28
U85514C30-NOT INSTALLED
2/30
2/30
U85514C32-NOT INSTALLED
2/32
2/32
U85514C34-NOT INSTALLED
2/34
FE2/0
2/34
U85514C36-NOT INSTALLED
2/36
FE2/0
2/50
GE1/0
.124
2/15
2/13
2/14
2/11
U85514C26-NOT INSTALLED
FE2/0
LB0 10.12.0.93/30
.251
GE2/0 LB1 65.218.69.30/32
2/21
2/10
2/19
2/17
2/12
U85514C24-NOT INSTALLED
FE2/0
swdc-mdf-rsw2
loopback 0
10.12.0.69
2/25
2/8
2/23
2/24
FE2/0
Network
10.128.2.240/29
2/49
2/29
2/6
2/27
U85514C22-NOT INSTALLED
FE2/0
OSPF excepts
BGP routes via
filtering.
2/33
2/4
2/31
2/22
FE2/0
.244
.241
GE0/0 GE4/1
LB0 10.12.0.97/30
LB1 65.218.69.29/32
eBGP peering
U85514C2-CTALLSTA02US
FE0/1 .1
LB0 .2
.71 FE2/0
FE0/0 .130
2/4
U85515CAT2
Cisco Systems 2948G
Network 65.221.230.64/26
SC0 65.222.45.68
157.130.148.32/30
2/4
2/17
2/6
2/8
Static routes anchored
to Allstate Loopback0
and advertized in BGP:
10.0.0.0/8
10.32.0.0/11
10.128.0.0/11
64.94.5.0/24
166.90.140.0/24
167.127.0.0/16
172.16.0.0/12
192.168.0.0/16
AS 65021
U85514E2
C is c o 1 2 0 0 0 c s e rie s
.33
AS 65029
Each VPN has its own EIGRP
AS 519 routing domain. EIGRP
AS 519 and BGP mutually
redistribute routes via filtering.
Access to console for
all U85514 equipment
and U85515TS1.
Host Table Loopback
192.168.24.252
POS4/1
FE0/1 .23
FE0/0 .151
2/9
GE6/0
.66
GW9.DFW9
AS 65037
U85515C11-CTALLSTAAHUS
FE0/1 .24
2/11 .12 FE2/0
POS2/0
157.130.148.108/30
2/19
2/2
FE2/0
AS 65035
C is c o 1 2 0 0 0 c s e rie s
Shadow
OC-3/155Mbps
2/21
FE2/0
U85515C15-CTALLSTAADUS
FE0/1 .26
2/15 .14 FE2/0
LB0 .52
FE0/0 .154
U85515E1
C is c o 1 2 0 0 0 c s e rie s
2/27
U85467CAT3
Cisco Systems 2948G
Network 10.88.210.128 /25
SC0 10.88.210.250
U85514CAT1
Cisco Systems 2948G
Network 65.221.230.0/26
SC0 65.221.230.4
2/27
2/25
2/17 .15 FE2/0
LB0 .54
GW7.DFW7
2/29
U85515C27-NOT INSTALLED
FE2/0
2/36
U90977CAT4
Cisco Systems 2948G
Network 10.151.210.0 /25
SC0 10.88.210.123
Allstate
Irving Texas
SouthWest Data Center
Super Hub
2/35
AS 65001
.242
.243
GE0/0 GE4/1
U85514DS2
Static routes anchored
to Allstate Loopback0
and advertized in BGP:
10.0.0.0/8
10.32.0.0/11
10.128.0.0/11
64.94.5.0/24
166.90.140.0/24
167.127.0.0/16
172.16.0.0/12
192.168.0.0/16
Agent office
IPC Hudson - Agency Broadband QOS Test
Allstate Northbrook - GGG
Allstate IPC Hudson
Production Edge
Production LAN switch
Test 1751 Spoke -u82977
MCI Inet
Test Crypto - 7200 VXR
Ethernet
Production LAN switch
glic-mdf-rsw2 - Cisco 6509
port - tba
192.168.25.0/25
Allstate LAN - IP 10.173.193.1
Mask 255.255.255.192
Test Downstream - 7200
Static routes
Allstate Data Network 10.0.0.0/8
Allstate agent LAN 10.173.185.1/26
VPN router to downstream
Agent router public interface
VPN router public interface
VLAN 66
Allstate FE - IP 10.66.2.245
Mask 255.255.255.248
Allstate FE - IP 10.66.2.243
Mask 255.255.255.248
Topology for MCI QOS Test
Last Updated:
July 7, 2003
Author: Network Engineering
FILENAME:
Mciqos.vsd
Page: 1 of 1
Internet/Extranet
• We do not use the default route
• There are 3 data center with ISP
connections
• We code static routes to the firewalls (we
don’t trust firewalls running dynamic
routing protocols) and redist to OSPF
The project
The project
• We use a single data network provider
• This is a single point of failure of that
providers ATM/Frame networks
• Add a second data provider
– Initially to use for the dual attached sites
– Then convert 1 of the core ATM meshes to the
second provider
Layer 2 vs Layer 3 provider
• Frame Relay is layer 2 connectivity
– The routers have a direct peering relationship
• Many providers are offering Layer 3
– Costs are the same or even less
– MPLS VPN is the data transport
• Many providers are using MPLS to move even layer 2
networks
– You have a routing relationships with the provider not
with yourself
• So More complex to configure and fix
• Not a simple OSPF network anymore
Which one we picked
• Layer 3…
– DR becomes free do not need to run more PVCs to a
DR data center
– The data center placement of servers assumption is
changing
• Apps are being put to 1 DC
–
–
–
–
–
Also there is more site to site traffic than we expect
So we can reduce traffic on the ATM core
And increase response time
Do dual homed sites first convert 1 link to L3
Single homed late
MPLS VPN
VPN A/Site 2
10.2/16
VPN B/Site 1
10.1/16
CE B1
P1
CE2B1
10.2/16
CEA2
1
PE2
VPN B/Site 2
P2
PE1
CEA1
CEB2
PE3
P3
CEA3
10.3/16
CEB3
10.1/16
VPN A/Site 1
VPN A/Site 3
10.4/16
VPN B/Site 3
Route types
• CE customer Edge
– your router
– run BGP to provider
– Knows nothing about other customers or provider
routes
• PE provider Edge
– Knows about all local customer VPNS
– Has multiple routing tables
• P providers
– Transport only
– No customer routes
Routing objectives
• Support load share from the home DC
• Remote site goes direct to non home DC
over L3
• Remote site directly to remote site
• Reduce transit of the core
• Support a L3 provider in the core replacing
1 ATM mesh
• Do not use remote sites to transit traffic
Technical Objectives
• Limit the number of bgp attributes used
• Keep the remote site configuration simple
• Do not inject the default route unless you
must
• How to inject the Internet routes
Routing protocol design
Don’t forget the 3 rules of routing
• Longest subnet mask
• Lowest distance
• Best metric
BGP features we used
•
•
•
•
•
As path
Path length filters
No export
Backdoor
If AS Paths are equal then router uses
eBGP route
How to route
• Must look at the routes going BOTH ways
– Routes to
– Routes from
• The routes you advertise drags traffic to you
• The routes you take in is how you route back
• We load share by having each router use a
different path, then send equal cost into IGP
• Use MPLS
VPN based
L3 provider
• Remote sites
2nd link to L3
• Each data
center
connects to
L3
• Will not use
L3 to route
between DCs
due to QoS
concerns
Result
Core ATM PVC
Mesh
BGP AS 65402
BGP AS 65401
Data Center #2
Data Center #1
Frame-Relay
Router
Frame-Relay
Router
L3 Router
Frame-Relay
BGP AS 650xx
L3
iBGP
Frame-Relay
Router
L3 Router
Remote-Site X
Si
Si
OSPF 500
L3 Router
Routing
• Use BGP at remote sites
– Can use OSPF with SOME providers but not all
– BGP works much better
– Each site is 1 AS
• EACH data center is 1 AS
– This allows us to put an L3 provider in later
– BGP routes BETWEEN ASes
• Address ASes from private space
• This is ok because provider is a VPN
Route injection to/from BGP
• Allstate Data Center
– Explicit network statements to BGP
– Redist BGP to OSPF
• Remote site routes
– Redist from OSPF
• Decided that using network statements to complex
– BGP routers send just default route to any switches
• We will accept the extra LAN transit
• Internet routes
– Redist static
Internet routes
• There will be non BGP L3 switches
between Inet and allstate core
• Redist static into OSPF already
• So just redist into BGP also
• Put internet router in same AS as
datacenter (have to as no direct path)
• Use sync
• Send to L3 provider and to sites over L3
BGP to L3 provider (and then
remote sites
• Data center side
–
–
–
–
Send data center /11s
Send internet routes
Take routes from L3 provider
Do not forward other eBGP learned routes
• Remote site side
–
–
–
–
Send all local routes
do not forward other learned eBGP routes
Remember the no export to kill transit
Receive all routes
• Want to take L3 when I can
DC to Remote site FR
• Send all bgp derived routes
• Do as prepend of the data center AS
• This makes AS path =2 for DC on FR and
L3 paths
• This makes AS Path=3 for DC to DC via
ATM core so site to remote DC traffic over
L3
Remote site to DC on FR
• Do as prepend of 1 AS at remote end
• Need this so FR and L3 paths have AS
Path=2 so we load share
• Filter routes with AS Path >1
– I only want to send the local site routes up the
FR link
– Do not want DC to send transit traffic to site
IBGP in the remote site
•
•
•
•
•
•
•
•
Set next hop self
Routers must have a shared Enet
No redist of BGP to OSPF
So cant use sync so cant transit a L3 switch
Do not forward routes I learn via FR
Do not want a transit from L3 up the FR link
Do not want a transit to L3 from FR link
Set no export attribute on routes from DC over the FR
link
• This prevents site from passing them to L3
• Cannot AS path filter on IBGP because I want to pass
the DC route via iBGP
– Why I use no export
Results
DC to DC
Core ATM PVC
Mesh
• Each site
learns over
ATM
network with
AS Path = 1
• Cannot route
over L3
provider
BGP AS 65402
BGP AS 65401
Data Center #2
Data Center #1
Frame-Relay
Router
Frame-Relay
Router
L3 Router
Frame-Relay
BGP AS 650xx
L3
iBGP
Frame-Relay
Router
L3 Router
Remote-Site X
Si
Si
OSPF 500
L3 Router
Remote site to non home dc
Core ATM PVC
Mesh
• Non home DC
sent via L3 AS
Path = 2
• Home data
sends via FR
AS Path = 3
due to prepend
BGP AS 65402
BGP AS 65401
– Use if L3
down
Data Center #2
Data Center #1
Frame-Relay
Router
Frame-Relay
Router
L3 Router
Frame-Relay
BGP AS 650xx
L3
iBGP
Frame-Relay
Router
L3 Router
Remote-Site X
Si
Si
OSPF 500
L3 Router
non home dc to remote site
• Non Home DC
learns remote
site routes from
L3
• Home data
center sends
only the /11
summary
• so longest match
says L3
Core ATM PVC
Mesh
BGP AS 65402
BGP AS 65401
Data Center #2
Data Center #1
Frame-Relay
Router
Frame-Relay
Router
L3 Router
Frame-Relay
BGP AS 650xx
L3
iBGP
Frame-Relay
Router
L3 Router
Remote-Site X
Si
Si
OSPF 500
L3 Router
home dc to remote site
Core ATM PVC
Mesh
• Load share
• Routes from L3
have AS Path =
2
• Routes from FR
have AS Path =
2 due to prepend
• So each router
uses eBGP route
BGP AS 65402
BGP AS 65401
Data Center #2
Data Center #1
Frame-Relay
Router
Frame-Relay
Router
L3 Router
Frame-Relay
BGP AS 650xx
L3
iBGP
Frame-Relay
Router
L3 Router
Remote-Site X
Si
Si
OSPF 500
L3 Router
remote site to home dc
Core ATM PVC
Mesh
• Don’t care as
much about load
share
• Routes from L3
have AS Path =
2
• Routes from FR
have AS Path =
2 due to prepend
• So each router
uses eBGP route
BGP AS 65402
BGP AS 65401
Data Center #2
Data Center #1
Frame-Relay
Router
Frame-Relay
Router
L3 Router
Frame-Relay
BGP AS 650xx
L3
iBGP
Frame-Relay
Router
L3 Router
Remote-Site X
Si
Si
OSPF 500
L3 Router
remote site to remote site
Core ATM PVC
Mesh
• Use L3
network
• Learn site
specific routes
directly from
site
• Learn /11
summaries
from DCs
BGP AS 65402
BGP AS 65401
Data Center #2
Data Center #1
Frame-Relay
Router
Frame-Relay
Router
L3 Router
L3 Router
L3
Frame-Relay
BGP AS 650xx
BGP AS 650xx
iBGP
iBGP
Frame-Relay
Router
L3 Router
Frame-Relay
Router
Remote-Site 1
Si
Si
OSPF 500
L3 Router
Remote-Site 2
Si
Si
OSPF 500
Agent routes
•
•
•
•
•
•
•
Only dual DC connected things that don’t use BGP
Many routes summarized as /19s
I get these from MCI as OSPF externals
Have not decided how to inject them
They go to two data centers for redundancy
So I need to send them via BGP
So a router will get an OSPF external from the local MCI
connection and the other data center via BGP
• eBGP < OSPF so BOOM
• Use backdoor on core routers to set distance on the
agent routes to > than OSPF
• So if local MCI connection up use it, else transit core
Testing
Local Testing
• Use 7 routers
• 1 remote site
OSPF route not
shown
• Paths
– iBGP at remote
– L3
– FR to home DC
– Inter DC
TNG1
10.60.2.249
10.60.2.253
AS65000
TNG5
10.60.2.33
10.60.2.35
TNG3
10.60.2.13
10.60.2.14
10.60.2.36
10.60.2.34
TNG2
AS100
10.60.2.9
10.60.2.10
AS65001
TNG7
TNG4
10.60.2.17
10.60.2.18
OSPF
CPOC
•
•
•
•
•
•
Cisco Proof Of Concept
In Raleigh and San Jose
Lab use is free (if you are big enough)
Send in specific test plan
Your SE goes in a week ahead of time
Lab is all setup when you arrive
Testing
• Test migrations
• Test routing
– based on our policies
– failovers
• Measure convergence
• Test a migration of a core ATM mesh to L3
• Get some data and experience on the MPLS
side
• Try multicast over MPLS/VPN
CPOC Network Diagram
Allstate Core Migration - Network Setup
Core ATM
2Meshes
10.10.x.y
S_LS
AS 65401
10.70.x.y
AS 65402
Data Center#1
AS 65404
Data Center#4
cake
10.100.x.y
10.40.x.y
R_EXT 4_A
R_EXT 3_A
iBGP
iBGP
baldhead
R_CPE_DC1_A
iBGP
borabora
iBGP
Si
muscat
iBGP
iBGP
R_CPE_DC2_A
semillon
feta
iBGP
chardonnay
R_CPE_DC3_A
iBGP
S_DC4_A
R_CPE_DC4_B
merlot
PVCs
R_PE_C
FR_2
FR_1
FrameRelay
R_PE_D
ricotta
romano
L3Provider
(MPLS)
parmesan
iBGP
chablis
pinot
R_Core_A
- riesling to ecu1 (DLCI 100 to DLCI 200)
- riesling to ecu2 (DLCI 120 to DLCI 220)
- muscat to navy1 (DLCI 101 to DLCI 201)
- muscat to navy4 (DLCI 121 to DLCI 221)
- chardonnay to ecu3 (DLCI 130 to DLCI 330)
- chardonnay to navy3 (DLCI 131 to DLCI 331)
- pinot to ecu4 (DLCI 140 to DLCI 440)
- merlot to navy5 (DLCI 150 to DLCI 550)
swi ss
R_Core_B
R_PE_G
R_PE_E
colby
R_CPE_DC4_A
R_CPE_DC3_B
cabernet
iBGP
Si
S_DC3_A
R_CPE_DC2_B
cheesewhiz
mozzarella
R_PE_A
brie
bahamas
iBGP
Si
S_DC2_A
R_CPE_DC1_C
R_PE_B
hawaii
iBGP
iBGP
Si
S_DC1_A
franzia
R_CPE_DC1_B
AS 100
AS 65403
cookie
custard
R_EXT 2_A
iBGP
reisling
Data Center#3
Data Center#2
10.130.x.y
cobbler
R_EXT 1_A
R_PE_F
cheedar
Remote Site #1
ecu1
Remote Site #2
navy1
R_CPE_RS1_A
iBGP
R_CPE_RS1_B
Si
Si
S_CPE_RS1_A
S_CPE_RS1_B
crete
AS 65001
10.80.x.y
easter
ecu2
navy2
R_CPE_RS2_A
Si
S_CPE_RS2_A
cayman
iBGP
R_CPE_RS2_B
Si
S_CPE_RS2_B
aruba
Remote Site #4
Remote Site #5
R_CPE_RS4_A
R_CPE_RS5_A
Remote Site #3
navy4
R_CPE_RS2_C
ecu3
navy3
R_CPE_RS3_A
iBGP
R_CPE_RS3_B
Si
Si
S_CPE_RS3_A
bermuda
S_CPE_RS3_B
barbados
AS 65002
AS 65003
10.81.x.y
10.140.x.y
ecu4
navy5
AS 65004
AS 65005
10.102.x.y
10.51.x.y
OSPFConnection
MPLS Connection
BGP Connection
CPOC Learnings
• Inject all links both ATM core and L3 into BGP as they
will source pings
• Turn sync off due to code defect
• You must explicitly code send community in iBGP
• If you reference a non-existent as-path statement NO
ROUTES
• OSPF LSAs stay in the data base up to 90 minutes due
to timer jitter
– This is a migration issue
• Do lots of clear routes/clear ip bgp in the migration
• Need to change the BGP timers as default convergence
is 3 minutes
• iBGP only sends the best route
Going forward
• Already run BGP to some remote sites
• Migrate the core to bgp first
– Do a dress rehearsal
– Will be a big scary change so plan well
• Examine tools
– May not be able to assume we will get traps
– May have to watch the BGP tables for
changes
• Get a test connection in place