Transcript Experimental Results of BGP Convergence

BGP Convergence
Measurement Issues
Susan Hares, NextHop
Padma Krishnaswamy, NextHop
Marianne Lepp, Juniper Networks
Alvaro Retana, Cisco
Howard Berkowitz, Gett Communications
Elwyn Davies, Nortel Networks
1
Convergence?
AS
He's Dead, Jim
AS
Tester
Flapping
AS
AS
AS
Tester
AS
Internet-wide
2
Convergence: Within an AS
Single AS
Tester
R
R
R
Tester
AS-wide
iBGP
3
Convergence: Within a Box
Tester
Single Box
Routing/Control
Forwarding
Tester
4
Convergence for BMWG
• Box wide
— eBGP initially
— Control Plane initially
— Black box
— Specify begin and end of convergence measurement
— Specify measurement point
5
Send a packet stream from TR
– 3 Measurements
• Convergence 1:
to 1st
1st packet sent from Test Generator
packet received by Test Collector
— Transmission in and out plus process time of 1st packet
• Convergence 2: Last Packet sent from Test Generator
to last packet received by Test Collector
— Transmission in, queuing, processing of preceding updates, tail
end processing, transmission out of last packet
• Convergence 3: 1st packet sent from Test Generator to
last packet received by Test Collector
— Transmission in and out (relative to DUT), plus back-up in BGP
update and processing of entire stream
6
Measurement 1-3: Factors
• Packing matters
— Influences number of packets in the train
— Attribute packing
– Classification speed
– Packetization triggers
• IBGP synchronization turned off
• Turn off Minimum Route Advertisement Interval
Timers
• Smoothing in BGP to avoid self-synchronization in the
Network
7
BGP Convergence Depends On …
• Route mixtures
• Packet packing
• Timers
• TCP implementations
• Peers types, number of peers, and connectivity
• BGP-specific functionality
— Eg. Confederations, use of route reflectors, etc.
• Topology
• Vantage point within the network
• Policy
8
Benchmarking Convergence
Approach
• Must be repeatable
• Must be consistent
• Must be specifiable
• Must take into account
— Route mixture (data)
— Peers types and connectivity
— BGP-specific functionality
— Topology
9
Goals
• Provide a baseline of expected performance in today’s
network.
• Test different vendor implementations fairly
• Design tests that can be replicated
• Good results require good data
— The amount, type and composition of the information advertised
to the DUT has an impact on the convergence.
10
Route Mixtures
11
Modeling Route Mixtures:
Why not just use a feed?
• The route mixture is highly dependent on the
vantage point – Tier 1 ISP, Enterprise, Access,
etc.
• Problems with Looking Glass
— Vantage point
• Need to test tables larger than current live
tables
• Needs to be repeatable, consistent, and
specifiable
12
Route Mixture
• Factors that describe the BGP Table:
composition and timing
—Prefix distribution
–Node distribution and levels on tree
—AS mixtures and path lengths
—Attribute distribution (nexthop, communities,MED,
localpref)
—Packet packing
—Update sequencing (timing)
–Packet trains
13
Prefix Distribution
• Example: A table of all /32s is not representative of
the real world
• Prefixes are distributed across dozens of prefix
lengths
— For IPv4, the distribution is spread out through the Class A, B,
and C address spaces.
— For IPv6, there is no data
• Need to describe prefix distribution per prefix length
— Better characterization for IPv4 if Class also taken into account
• Analyze current Internet table to determine prefix
distribution characteristics
14
Prefix Distribution
• Example percentages of prefix distribution:
•
Mask
Overall
Class A
Class B
Class C
•
16
17
18
19
20
21
22
23
24
0.08114
0.00912
0.01813
0.05910
0.03372
0.04128
0.05574
0.07878
0.53355
0.00076
0.00030
0.00093
0.00378
0.00152
0.00085
0.00171
0.00235
0.00892
0.06637
0.00142
0.00113
0.00196
0.00151
0.00127
0.00226
0.00450
0.02366
0.01401
0.00741
0.01607
0.05336
0.03070
0.03915
0.05176
0.07193
0.50097
Total prefix length
distribution.
IPv4 sample distribution
across classes.
15
IP v6 Prefix Distribution
• Example percentages of prefix distribution:
•
Mask
•
0-10
11-20
21-30
31-40
41-50
51-60
61-70
71-80
81-90
•
91-100
Overall
3FEE
2001
0.08114
0.00912
0.01813
0.05910
0.03372
0.04128
0.05574
0.07878
0.53355
0.00076
0.00030
0.00093
0.00378
0.00152
0.00085
0.00171
0.00235
0.00892
0.06637
0.00142
0.00113
0.00196
0.00151
0.00127
0.00226
0.00450
0.02366
0.01401
0.00741
0.01607
0.05336
0.03070
0.03915
0.05176
0.07193
0.50097
0.53355
0.00892
0.02366
0.50097
•
100-110 0.53355
0.00892
0.02366
0.50097
•
111-128 0.53355
0.00892
0.02366
0.50097
Total prefix length
distribution.
other
IPv6 sample distribution
across currently routed
Addres space
16
Node Distribution
• Is tree dependent
• Width and depth of table are important
• Route mixtures should exercise various choices of
trees
— A route mixture that minimizes the number of nodes is not
accurate
— A route mixture that maximizes the spread of prefixes creates
is not accurate
17
Node Distribution
ROOT
Levels
10.0.0.0
192.0.0.0
54.0.0.0
10.1.0.0
192.5.0.0
10.10.0.0
192.8.0.0
54.10.0.0
10.1.1.0
10.1.1.1
192.8.2.0
10.10.5.0
54.10.1.0
54.10.4.0
192.8.2.0
192.8.2.128
Nodes
18
IP v6 Node Distribution
Levels
ROOT
3FEE::
3FEE:0100
3FEE:0101
2001::
2001::01
3FEE:2000::
3FEE:0101
2001:02
2001:0201
2001:0201:01
3FEE:0101:01
2001:0201:02
Nodes
19
Node Distribution
• For example, the following tables both contain three Class A /32
prefixes
— Table A
1.1.1.1/32, 1.1.1.2/32, 1.1.1.3/32
— Table B
1.1.1.1/32, 2.1.1.1/32, 3.1.1.1/32
— Their distribution in a tree will be different.
– Table A represents a narrow distribution, while Table B represents a
wide distribution.
Table B
ROOT
ROOT
1.0.0.0
2.0.0.0
3.0.0.0
1.1.0.0
1.1.0.0
2.1.0.0
3.1.0.0
1.1.1.0
1.1.1.0
2.1.1.0
3.1.1.0
1.1.1.1
2.1.1.1
3.1.1.1
1.0.0.0
1.1.1.1
Table A
1.1.1.2
1.1.1.2
20
Node Distribution Summary
• The width of the table must be measured per prefix
distribution and length
• Need to determine how many nodes each
address/prefix length combination use in a real table
• Solution: Analyze current Internet table to determine
node distribution characteristics
21
Route Components
22
BGP Attribute Distribution
• A BGP table contains many “attribute combinations”
• Analysis shows:
—11.75% of the routes have a unique AS_PATH
—2.5% of the routes have some other unique attribute.
—0.25% of the table have both a unique AS_PATH and some other
unique attribute
23
BGP Attribute Distribution
• Prefixes that share an attribute are not necessarily grouped
together
• Analysis shows an average of two consecutive NLRI share the same
attribute combination
—
—
—
—
—
—
1.0.0.0/8
2.0.0.0/8
3.0.0.0/8
4.0.0.0/8
5.0.0.0/8
6.0.0.0/8
AS_PATH 100 200
AS_PATH 100 200
AS_PATH 200 300
AS_PATH 200 300
AS_PATH 200 300
AS_PATH 100 200
24
Planes (control),
Trains,
and no Automobiles
25
Packet Packing
• Each packet has attributes and NLRIs
• Attribute packing is the ability to detect and pack NRLIs
with the same attributes into a packet
• NLRI packing is:
— the number of NLRIs per packet
— MPBGP not considered for 1st draft
– IPv6 packing is not different than IPv4
– Multicast packing (IP v4 and IP v6) may impact packing
• Specifics are affected by implementation
26
Update Sequencing (Timing)
• Parameters are:
— Number of packets in a train
— Interval between packets in a train
— TCP parameters, traffic and implementations affect
this
Packet 1 Packet 2 Packet 3
Packet train
Packet 4
Packet train
27
Timers
• Key timers
— Min-Route Advertisement Interval, Min AS Originations Interval
best setting still in debate
--
— Route Flap damping mechanisms
– Implementations vary
– Shorter prefixes get less damping
– RIPE 229 suggest parameters
– 1st Bgp Conv draft mandates route flap damping off
— TCP settings
• Operators need to give feedback
28
Peers, not Beers
29
Peer type matters
• EBGP vs IBGP
• EBGP
— 3rd party versus 1st party nexthop
— promiscuous versus specific peering
• IBGP - Route Reflection client and Confederations
affect convergence patterns
— See ietf-idr-route-oscillations-01.txt
• Still single box but these affect work done by box
30
Multiple Peers in test Environment
• Peers can have staggered starts
— Most realistic
• Peers can all send simultaneously
— Most load on the router
• Peers can have staggered starts in groups
31
Sample topology with 4 Peers
tcpdump
TG1
TG2
DUT
tcpdump
TC
tcpdump
TG3
tcpdump
TG4
tcpdump
32
Peer Specifics
• Type of Peer
— Promiscuous/Specific
• Sequence
— Connection establishment
— Sending 1st data
— Spacing of updates
— Connection up/down
33
Timing & Synchronization
• Consistency among timestamps taken by different
devices is a requirement
• Should be at least 1 order of magnitude better than
measured quantity
— For BGP convergence, we are time-stamping packets
• NTP? GPS? Other?
• Synchronization between measurements can a
significant factor
34
Some Boxes work
Harder than Others
35
BGP Protocol functions will impact
convergence
• Route Reflections,
• Confederations
• Add/delete communities
• RFC 2547, Label switching
• Multi-protocol
• Route flap damping
• Min Route Advertisement
36
Parameters we suggest for Protocol
Functions for 1st Document
• No Route Reflectors (no IBGP this version)
• No confederations
• No Add/Delete communities
• No 2547 VPNS or multicast
• Route flap damping OFF
• Min Route Advertisement Interval specified
• Min AS Origination Interval specified
37
Topology
38
Topology matters
• Exchange point topology
• N star topologies meshed for Route Reflection
• Confederations with particular topologies
• IBGP/EBGP mesh overlay
• Building blocks
— single link, line, mesh, partial mesh, star, wheel
39
Single link: 1st Document
TR1
TRes
DUT
TR2
line
TRn
n >= 1
40
Line
DUT
TR1
DUT
DUT
Longer line
TRes
41
Mesh
TRes
DUT
DUT
TRes
mesh
DUT
TR1
DUT
TRes
42
Partial Mesh
TRes
DUT
DUT
TRes
DUT
TR1
DUT
TRes
43
References
• IETF51 BMWG talk:
http://www.ietf.org/proceedings/01aug/slides/bmwg-4/
• NextHop IETF51 talk:
http://www.ietf.org/proceedings/01aug/slides/bmwg5/index.html
• Howard’s IETF51 talk:
http://www.ietf.org/proceedings/01aug/slides/bmwg6/index.html
• Recommendations for flap damping, Ripe 229:
http://www.ripe.net/ripe/docs/ripe-229.html
• BGP Convergence Terminology ID:
http://www.ietf.org/internet-drafts/draft-ietf-bmwgconterm-00.txt
• BGP Convergence Methodology: http://www.ietf.org/internetdrafts/draft-ietf-bmwg-bgpbas-00.txt
44
Thank You
Questions?
45
Route Mixtures Matter!
• The amount, type and composition of the information
advertised to the DUT has an impact on the
convergence.
• Goal is to provide a baseline of expected performance
in today’s network.
• Test different vendor implementations fairly
• Design tests that can be replicated
• Good results require good data
46