Transcript The Stable Paths Problem As A Model Of BGP Routing Timothy G.

The Stable Paths Problem As A
Model Of BGP Routing
Timothy G. Griffin
AT&T Research
[email protected]
http://www.research.att.com/~griffin
NJIT
April 24, 2002
Outline
Part I: The glue that holds the Internet
together : interdomain routing with
The Border Gateway Protocol (BGP)
Part II: A formal model of BGP routing
policies
Joint work with Bruce Shepherd and
Gordon Wilfong (Bell Labs)
Architecture of Dynamic Routing
OSPF
BGP
AS 1
IGP = Interior Gateway Protocol
Metric based: OSPF, IS-IS, RIP,
EIGRP (cisco)
EGP = Exterior Gateway Protocol
EIGRP
AS 2
Policy based: BGP
The Routing Domain of BGP is the entire Internet
Technology of Distributed Routing
Link State
•
•
•
•
•
•
Topology information is
flooded within the routing
domain
Best end-to-end paths are
computed locally at each
router.
Best end-to-end paths
determine next-hops.
Based on minimizing
some notion of distance
Works only if policy is
shared and uniform
Examples: OSPF, IS-IS
Vectoring
•
•
•
•
•
•
Each router knows little
about network topology
Only best next-hops are
chosen by each router for
each destination network.
Best end-to-end paths
result from composition
of all next-hop choices
Does not require any
notion of distance
Does not require uniform
policies at all routers
Examples: RIP, BGP
The Gang of Four
Link State
IGP
EGP
OSPF
IS-IS
Vectoring
RIP
BGP
Many Routing Processes Can Run on a Single Router
BGP
RIP Process
BGP Process
RIP Routing tables
BGP Routing tables
OSPF Process
OSPF Routing tables
RIP
Domain
OS kernel
OSPF
Domain
Forwarding Table Manager
Forwarding Table
6
AS Numbers (ASNs)
ASNs are 16 bit values.
64512 through 65535 are “private”
•
•
•
•
•
•
•
•
Currently over 12,000 in use.
Yale: 29
MIT: 3
Harvard: 11
Genuity: 1
AT&T: 7018, 6341, 5074, …
UUNET: 701, 702, 284, 12199, …
Sprint: 1239, 1240, 6211, 6242, …
…
ASNs represent units of routing policy
Autonomous Routing Domains Don’t
Always Need BGP or an ASN
Qwest
Nail up routes 130.132.0.0/16
pointing to Yale
Nail up default routes 0.0.0.0/0
pointing to Qwest
Yale University
130.132.0.0/16
Static routing is the most common way of connecting an
autonomous routing domain to the Internet.
This helps explain why BGP is a mystery to many …
ASNs Can Be “Shared” (RFC 2270)
AS 701
UUNet
AS 7046
Crestar
Bank
AS 7046
NJIT
AS 7046
Hood
College
128.235.0.0/16
ASN 7046 is assigned to UUNet. It is used by
Customers single homed to UUNet, but needing
BGP for some reason (load balancing, etc..) [RFC 2270]
How Many ASNs are there?
Thanks to Geoff Huston. http://www.telstra.net/ops on June 23, 2001
AS Graphs Can Be Fun
The subgraph showing all ASes that have more than 100 neighbors in full
graph of 11,158 nodes. July 6, 2001. Point of view: AT&T route-server
BGP Table Growth
Thanks to Geoff Huston. http://www.telstra.net/ops/bgptable.html on August 8, 2001
Nontransit vs. Transit ASes
ISP 2
ISP 1
NET A
Traffic NEVER
flows from ISP 1
through NET A to ISP 2
(At least not intentionally!)
IP traffic
Internet Service
providers (often)
have transit
networks
Nontransit AS
might be a corporate
or campus network.
Could be a “content
provider”
13
Selective Transit
NET B
NET A DOES NOT
provide transit
Between NET D
and NET B
NET C
NET A
NET A provides transit
between NET B and NET C
and between NET D
and NET C
NET D
Most transit networks transit in a selective manner…
IP traffic
14
Customers and Providers
provider
provider
customer
IP traffic
customer
Customer pays provider for access to the Internet
The Peering Relationship
peer
provider
peer
customer
Peers provide transit between
their respective customers
Peers do not provide transit
between peers
traffic
allowed
traffic NOT
allowed
Peers (often) do not exchange $$$
Peering Provides Shortcuts
Peering also allows connectivity between
the customers of “Tier 1” providers.
peer
provider
peer
customer
BGP-4
• BGP = Border Gateway Protocol
• Is a Policy-Based routing protocol
• Is the de facto EGP of today’s global Internet
• Relatively simple protocol, but configuration is complex and the
entire world can see, and be impacted by, your mistakes.
•
1989 : BGP-1 [RFC 1105]
–
•
Replacement for EGP (1984, RFC 904)
1990 : BGP-2 [RFC 1163]
• 1991 : BGP-3 [RFC 1267]
•
1995 : BGP-4 [RFC 1771]
–
Support for Classless Interdomain Routing (CIDR)
18
BGP Operations (Simplified)
Establish session on
TCP port 179
AS1
BGP session
Exchange all
active routes
AS2
Exchange incremental
updates
While connection
is ALIVE exchange
route UPDATE messages
19
Four Types of BGP Messages
• Open : Establish a peering session.
• Keep Alive : Handshake at regular intervals.
• Notification : Shuts down a peering session.
• Update : Announcing new routes or withdrawing
previously announced routes.
announcement
=
prefix + attributes values
20
BGP Attributes
Value
----1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
...
255
Code
--------------------------------ORIGIN
AS_PATH
NEXT_HOP
MULTI_EXIT_DISC
LOCAL_PREF
ATOMIC_AGGREGATE
AGGREGATOR
COMMUNITY
ORIGINATOR_ID
CLUSTER_LIST
DPA
ADVERTISER
RCID_PATH / CLUSTER_ID
MP_REACH_NLRI
MP_UNREACH_NLRI
EXTENDED COMMUNITIES
Reference
--------[RFC1771]
[RFC1771]
[RFC1771]
[RFC1771]
[RFC1771]
[RFC1771]
[RFC1771]
[RFC1997]
[RFC2796]
[RFC2796]
[Chen]
[RFC1863]
[RFC1863]
[RFC2283]
[RFC2283]
[Rosen]
Most
important
attributes
reserved for development
From IANA: http://www.iana.org/assignments/bgp-parameters
Not all attributes
need to be present in
every announcement
Attributes are Used to Select
Best Routes
192.0.2.0/24
pick me!
192.0.2.0/24
pick me!
192.0.2.0/24
pick me!
192.0.2.0/24
pick me!
Given multiple
routes to the same
prefix, a BGP speaker
must pick at most
one best route
(Note: it could reject
them all!)
BGP Route Processing
Open ended programming.
Constrained only by vendor configuration language
Receive Apply Policy =
filter routes &
BGP
Updates tweak attributes
Apply Import
Policies
Based on
Attribute
Values
Best
Routes
Best Route
Selection
Best Route
Table
Apply Policy =
filter routes &
tweak attributes
Transmit
BGP
Updates
Apply Export
Policies
Install forwarding
Entries for best
Routes.
IP Forwarding Table
23
Route Selection Summary
Highest Local Preference
Enforce relationships
Shortest ASPATH
Lowest MED
i-BGP < e-BGP
traffic engineering
Lowest IGP cost
to BGP egress
Lowest router ID
Throw up hands and
break ties
Tweak Tweak Tweak
•
For inbound traffic
– Filter outbound routes
– Tweak attributes on
outbound routes in
the hope of
influencing your
neighbor’s best route
selection
•
inbound
traffic
For outbound traffic
– Filter inbound routes
– Tweak attributes on
inbound routes to
influence best route
selection
In general, an AS has more
control over outbound traffic
outbound
traffic
outbound
routes
inbound
routes
ASPATH Attribute
AS 1129
135.207.0.0/16
AS Path = 1755 1239 7018 6341
135.207.0.0/16
AS Path = 1239 7018 6341
AS 1239
Sprint
AS 1755
135.207.0.0/16
AS Path = 1129 1755 1239 7018 6341
Ebone
AS 12654
AS 6341
AT&T Research
RIPE NCC
RIS project
135.207.0.0/16
AS Path = 7018 6341
AS7018
135.207.0.0/16
AS Path = 6341
Global Access
135.207.0.0/16
AS Path = 3549 7018 6341
AT&T
135.207.0.0/16
AS Path = 7018 6341
AS 3549
Global Crossing
135.207.0.0/16
Prefix Originated
26
AS Graphs Do Not Show Topology!
BGP was designed to
throw away information!
The AS graph
may look like this.
Reality may be closer to this…
AS Graphs Depend on Point of View
peer
peer
provider
customer
1
3
2
4
1
3
5
1
2
4
5
6
3
1
6
4
2
2
6
4
5
3
5
6
This explains why there is no UUNET (701) Sprint (1239) link on previous slide!
Shorter Doesn’t Always Mean Shorter
In fairness:
could you do
this “right” and
still scale?
Mr. BGP says that
path 4 1 is better
than path 3 2 1
Duh!
AS 4
AS 3
Exporting internal
state would
dramatically
increase global
instability and
amount of routing
state
AS 2
AS 1
Shedding Inbound Traffic with
ASPATH Padding Hack
AS 1
provider
192.0.2.0/24
ASPATH = 2 2 2
192.0.2.0/24
ASPATH = 2
primary
backup
customer
AS 2
192.0.2.0/24
Padding will (usually)
force inbound
traffic from AS 1
to take primary link
30
Padding May Not Shut Off All Traffic
AS 1
AS 3
provider
provider
192.0.2.0/24
ASPATH = 2
192.0.2.0/24
ASPATH = 2 2 2 2 2 2 2 2 2 2 2 2 2 2
primary
backup
customer
AS 2
192.0.2.0/24
AS 3 will send
traffic on “backup”
link because it prefers
customer routes and local
preference is considered
before ASPATH length!
Padding in this way is often
used as a form of load
31
balancing
COMMUNITY Attribute to the Rescue!
AS 1
AS 3
provider
provider
AS 3: normal
customer local
pref is 100,
peer local pref is 90
192.0.2.0/24
ASPATH = 2
COMMUNITY = 3:70
192.0.2.0/24
ASPATH = 2
primary
backup
customer
AS 2
192.0.2.0/24
Customer import policy at AS 3:
If 3:90 in COMMUNITY then
set local preference to 90
If 3:80 in COMMUNITY then
set local preference to 80
If 3:70 in COMMUNITY then
set local preference to 70
32
Hot Potato Routing: Go for the Closest
Egress Point
192.44.78.0/24
egress 2
egress 1
15
56
IGP distances
This Router has two BGP routes to 192.44.78.0/24.
Hot potato: get traffic off of your network as
Soon as possible. Go for egress 1!
33
Getting Burned by the Hot Potato
2865
High bandwidth
Provider backbone
17
SFF
Low bandwidth
customer backbone
Heavy
Content
Web Farm
NYC
15
56
San Diego
Many customers want
their provider to
carry the bits!
tiny http request
huge http reply
34
Cold Potato Routing with MEDs
(Multi-Exit Discriminator Attribute)
Prefer lower
MED values
2865
17
Heavy
Content
Web Farm
192.44.78.0/24
MED = 56
192.44.78.0/24
MED = 15
15
56
192.44.78.0/24
This means that MEDs must be considered BEFORE
IGP distance!
Note1 : some providers will not listen to MEDs
Note2 : MEDs need not be tied to IGP distance
35
Policies Can Interact Strangely
(“Route Pinning” Example)
backup
customer
1
3
2
Disaster strikes primary link
and the backup takes over
4
Install backup link using community
Primary link is restored but some
traffic remains pinned to backup
News at 11:00h
• BGP is not guaranteed to converge on a
stable routing. Policy interactions could lead
to “livelock” protocol oscillations.
See “Persistent Route Oscillations in Inter-domain Routing” by K. Varadhan, R.
Govindan, and D. Estrin. ISI report, 1996
• Corollary: BGP is not guaranteed to recover
from network failures.
PART II
Can we model BGP?
Underlying problem
Distributed means of
computing a solution.
Shortest Paths
RIP, OSPF, IS-IS
X?
BGP
What Problem is BGP solving?
X could
•
•
•
•
aid in the design of policy analysis algorithms and heuristics
aid in the analysis and design of BGP and extensions
help explain some BGP routing anomalies
provide a fun way of thinking about the protocol
Separate dynamic and static semantics
static
semantics
BGP Policies
dynamic
semantics
BGP
Stable Paths
SPVP
Problem (SPP)
Booo Hooo,
Many, many
complications...
SPVP = Simple Path
Vector Protocol = a
distributed
algorithm for
solving SPP
An instance of the Stable Paths Problem (SPP)
•
•
•
•
A graph of nodes and edges,
Node 0, called the origin,
For each non-zero node, a
set or permitted paths to
the origin. This set always
contains the “null path”.
A ranking of permitted
paths at each node. Null
path is always least
preferred. (Not shown in
diagram)
1
When modeling BGP : nodes represent
BGP speaking routers, and 0 represents
a node originating some address block
210
2
20
5
5210
2
4
420
430
3
30
0
1
130
10
most preferred
…
least preferred
Yes, the translation
gets messy!
A Solution to a Stable Paths Problem
2
210
20
A solution is an assignment of
permitted paths to each node
such that
•
•
node u’s assigned path is either
the null path or is a path uwP,
where wP is assigned to node w
and {u,w} is an edge in the
graph,
each node1is assigned the
highest ranked path among
those consistent with the paths
assigned to its neighbors.
5
5210
2
4
420
430
3
30
0
1
130
10
A Solution need not represent
a shortest path tree, or
a spanning tree.
An SPP may have multiple solutions
120
10
120
10
1
120
10
1
0
0
2
210
20
DISAGREE
1
2
210
20
First solution
0
2
210
20
Second solution
Multiple solutions can result in
“Route Triggering”
10
1230
1
230
210
2
1
primary
link
0
2
0
1
10
1230
2
230
210
0
backup
link
3210
30
3
Remove primary link
3
3
Restore primary link
3210
30
BAD GADGET : No Solution
2
210
20
4
0
130
10
1
3
3
320
30
Persistent Route Oscillations in Inter-Domain Routing. Kannan Varadhan, Ramesh Govindan,
and Deborah Estrin. Computer Networks, Jan. 2000
SURPRISE : Beware of Backup Policies
210
20
BGP is not robust :
it is not guaranteed
to recover from
network failures.
1
130
10
2
Becomes a BAD GADGET if link
(4, 0) goes down.
4
40
420
430
0
3
3420
30
PRECARIOUS
Has a solution, but can get “trapped”
4
310
3120
5
5310
563120
53120
4310
453120
43120
1
3
120
10
0
6
2
6310
643120
63120
This part has a solution only
when node 1 is assigned the
direct path (1 0).
210
20
As with DISAGREE, this part
has two distinct solutions
Solving an SPP
Just enumerate all path assignments
And check stability of each….
Exponential complexity
But, in worst case you (probably)
can’t do any better…
Use 3-SAT…
Variables V = {X1, X2, …, Xn}
Clauses
C1 = X17 or ~X23 or ~X3,
C2 = ~X2 or X3 or ~X12
….
Cm = X6 or ~X7 or X18
Question Is there an variable assignment
A:V
{true, false} such that
each clause C1, … ,Cm is true?
3-SAT is NP-complete
Modeling assignment to variable X
XX0
X0
X
X
X
0
0
0
XX0
X0
X = false
X = true
SPP Solvability is NP-complete
C X7 0
C X5 0
C X3 0
X7
C
X7 or X5 or X3
X7
X5
X5
X3
X3
0
BAD GADGET
SPVP protocol
Pick the best path available at any given time…
process spvp[u]
{
receive P from w 
{ rib-in(uw) := u P
if rib(u) != best(u) {
rib(u) := best(u)
foreach v in peers(u) {
send rib(u) to v
}
}
}
}
SPVP wanders around assignment space
= assignment
= solution
Distributed algorithms to solve SPP?
• OSPF-like :
–
–
–
–
Distribute topology, path ranks
Solve SPP locally
Exponential worst case
How can loops be avoided when multiple solutions
exist?
• RIP-like:
This is BGP…
– Pick the best path from the set of your neighbor’s paths,
tell your neighbors when you change your mind
– Can diverge
– Not guaranteed to find a solution, even when one exists
– Even when converges, no bound on convergence time
A sufficient condition for sanity
If an instance of SPP has an
acyclic dispute digraph, then
Static (SPP)
Dynamic (SPVP)
solvable
safe (can’t diverge)
unique solution
predictable restoration
all sub-problems
uniquely solvable
robust with respect to
link/node failures
Dispute Digraph
u
v
…
(u v)P
…
(u v)Q
...
…
Q
…
P
...
P
Q
Gives the dispute arc
Q
(u v)P
0
Dispute Digraph (cont.)
u
P
v
…
(u,v)P
...
…
P
...
Gives the transmission arc
P
(u,v)P
0
Dispute Digraph Example
130
10
210
20
1
2
0
20
10
420
210
3420
3
4
3420
30
420
430
BAD GADGET II
CYCLE
430
130
30
What is to be done?
Static
Approach
Dynamic
Approach
Extend BGP with
a dynamic means of
detecting and suppressing
policy-based oscillations?
Automated Analysis
of Routing Policies
Inter-AS
coordination
These approaches are complementary
Some Applications SPP Theory
• A Safe Path Vector Protocol. Timothy G. Griffin,
Gordon Wilfong. INFOCOM 2001
– Dynamic solution for SPVP based on histories
(dynamically constructed dispute cycles).
• Inherently safe backup routing with BGP. Lixin Gao,
Timothy G. Griffin, Jennifer Rexford. INFOCOM 2001
• Show that if customer/provider peer/peer model is
followed, then all is well,
– Show that this can be exteded with complex
backup policies and remain safe.
– Analysis of “cold potato” routing problems (MED
oscillation). Griffin and Wilfong. Work in progress
– MED requires a modification to SPP model
– Analysis of Internal BGP (IBGP) configuration. Griffin
and Wilfong. Work in progress.
A Few Research Topics
• Dynamic Behavior of BGP
– Convergence time, message overhead
• BGP Security
– S-BGP defined, but not deployed. Is it a
good solution.
– Need an “interdomain trust model”
• Beyond BGP?
– When will it break? What will replace it?
Selected Papers on BGP Sanity
•
•
•
•
•
•
•
Persistent Route Oscillations in Inter-Domain Routing. Kannan Varadhan, Ramesh Govindan,
and Deborah Estrin. Computer Networks, Jan. 2000. (Also USC Tech Report, Feb. 1996)
– Shows that BGP is not guaranteed to converge
An Architecture for Stable, Analyzable Internet Routing. Ramesh Govindan, Cengiz
Alaettinoglu, George Eddy, David Kessens, Satish Kumar, and WeeSan Lee. IEEE Network
Magazine, Jan-Feb 1999.
– Use RPSL to specify policies. Store them in registries. Use registry for conguration
generation and analysis.
An Analysis of BGP Convergence Properties. Timothy G. Griffin, Gordon Wilfong. SIGCOMM
1999
– Model BGP, shows static analysis of divergence in policies is NP complete
Policy Disputes in Path Vector Protocols. Timothy G. Griffin, F. Bruce Shepherd, Gordon
Wilfong. ICNP 1999
– Define Stable Paths Problem and develop sufficient condition for “sanity”
A Safe Path Vector Protocol. Timothy G. Griffin, Gordon Wilfong. INFOCOM 2001
– Dynamic solution for SPVP based on histories
Stable Internet Routing without Global Coordination. Lixin Gao, Jennifer Rexford.
SIGMETRICS 2000
– Show that if certain guidelines are followed, then all is well.
Inherently safe backup routing with BGP. Lixin Gao, Timothy G. Griffin, Jennifer Rexford.
INFOCOM 2001
– Use SPP to study complex backup policies
Pointers
• SIGCOMM 2001 Tutorial on BGP:
• http://www.research.att.com/~griffin/sigcomm2001_bgp
_tutorial/abstract.html
• Links on Interdomain routing and BGP:
• http://www.research.att.com/~griffin/interdomain.html
• Papers on BGP theory:
• http://www.research.att.com/~griffin/bgpresearch.html
[email protected]
Thank You!