aarnet3, radb and rpsl APAN NOC Taipei - 25 August 2005 [email protected] AARNet3 Network Highlights • • • • STM-64c (10Gbps) Backbone Dual STM-1 to NT & Tasmania Replacing Procket 8812with.
Download ReportTranscript aarnet3, radb and rpsl APAN NOC Taipei - 25 August 2005 [email protected] AARNet3 Network Highlights • • • • STM-64c (10Gbps) Backbone Dual STM-1 to NT & Tasmania Replacing Procket 8812with.
aarnet3, radb and rpsl APAN NOC Taipei - 25 August 2005 [email protected] AARNet3 Network Highlights • • • • STM-64c (10Gbps) Backbone Dual STM-1 to NT & Tasmania Replacing Procket 8812with Juniper M320 Deploying DWDM from Adelaide to Brisbane – Providing multiple GigE to regional areas – Rolling our backbone onto our DWDM kit • Multiple trans Pacific circuits – 2 x STM-64c for research and education – 2 x STM-4c (2 x 622Mbps) for commodity – 2 x STM-1 • Looking to expand footprint to Asia 2 © 2005, AARNet Pty Ltd AARNet3 Network 3 © 2005, AARNet Pty Ltd AARNet3 International Connectivity 4 © 2005, AARNet Pty Ltd Commodity Transit Provision • International commodity transit from – Palo Alto – Los Angeles – Seattle etc etc • Domestic commodity transit in – – – – – – 5 © 2005, AARNet Pty Ltd Sydney Melbourne Adelaide Canberra Brisbane Perth etc etc AARNet PoPs • Domestic existing – – – – – – – – Sydney (3) Melbourne (2) Brisbane (2) Adelaide (2) Perth (3) Canberra (2) Hobart (1) Darwin (1) • Coming soon – Alice Springs (1) 6 © 2005, AARNet Pty Ltd • International existing – – – – – Seattle Palo Alto Los Angeles Hawai’i Suva • Coming soon – Singapore – Frankfurt The AARNet3 environment • Currently over 60 routers deployed – This will expand to over 80 by the end of 2005 • A mix of Juniper, Cisco and Procket routers – Currently Procket at the core – migrating to Juniper – Cisco routers at the customer edge – Link speeds varying from STM-64c to STM-4s and STM-1s for long haul – 10GbE intra PoPs and GbE connections from PoPs but still some managed services and legacy ATM 7 © 2005, AARNet Pty Ltd The BGP environment • • • • • • • • 8 17 commodity transit connections Over 163 peers both commodity and R&E Most peerings are bilateral, a few are multilateral Some 16 peerings with external international R&E networks Over 200 iBGP peerings Over 250 IPv4 prefixes advertised and growing… IPv6 enabled IPv4 multicast enabled © 2005, AARNet Pty Ltd BGP policy complexity • • • • • • • • • • • 9 7575:1 Export external to AARNet with "no-export" 7575:2 No export beyond AARNet 7575:3 Prepend AS7575 once 7575:4 Prepend AS7575 twice 7575:5 Prepend AS7575 thrice 7575:6 Blackhole traffic 7575:7 Regional only 7575:70 AARNet local preference 70 7575:80 AARNet local preference 80 7575:90 AARNet local preference 90 …and much more… © 2005, AARNet Pty Ltd How do we manage this complexity? • Very hard to manage on an ad-hoc basic with such diversity • Needs an overall policy that manages router BGP configurations • Needs cross vendor router support • Turn towards IRRs and RPSL to manage this 10 © 2005, AARNet Pty Ltd What is RPSL? • Object oriented language • Structured whois objects • Refinement of RIPE 181 (and it’s predecessors) based on operational experience • Describes things interesting to routing policy – – – – 11 Prefixes AS Numbers Relationships between BGP peers Management responsibility © 2005, AARNet Pty Ltd How we went about it • Need to identify which IRR to use – AARNet uses RADB. – Others run their own for control • Need to decide what degree of filtering is desired – Prefix filters – AS path filters – Both! • Register a maintainer object at chosen IRR – Usually a “manual” process and could be multi-stage if PGP key authentication required 12 © 2005, AARNet Pty Ltd Maintainer Object Maintainer objects used for authentication Multiple authentication methods NONE, MAIL-FROM, CRYPT-PW, PGPKEY mntner: MAINT-ASAARNET descr: Maintainers for AARNet and AARNet member objects admin-c: CS3692 tech-c: GT342-AU upd-to: [email protected] mnt-nfy: [email protected] auth: PGPKEY-FAD8C612 auth: PGPKEY-23B7F8EF remarks: Australian Academic and Research Network http://www.aarnet.edu.au/ mnt-by: MAINT-ASAARNET changed: [email protected] 20040113 source: RADB 13 © 2005, AARNet Pty Ltd Route Object Use CIDR length format Specifies origin AS for a route Can indicate membership of a route set route: descr: origin: mnt-by: changed: source: 14 © 2005, AARNet Pty Ltd 134.7.0.0/16 Curtin University of Technology AS7575 MAINT-ASAARNET [email protected] 20050818 RADB Route Set Object • Collects routes together with similar properties route-set: AS7575:RS-UNSW descr: University of New South Wales members: 129.94.0.0/16, 149.171.0.0/16, 203.10.48.0/24, 203.20.160.0/24, 203.20.160.0/19 remarks: List of routes accepted from AS7570 admin-c: MP151 tech-c: ANOC-AP mnt-by: MAINT-ASAARNET changed: [email protected] 20050427 source: RADB 15 © 2005, AARNet Pty Ltd AS Set Object (1) • Collect together Autonomous Systems with shared properties • Can be used in policy in place of AS as-set: AS7575:AS-EDGE descr: AARNet3 customers AS set members: AS1851, AS4822, AS6262, AS7575, AS7645, AS10148, AS17498, AS23654, AS23719, AS24101, AS24390, AS24431, AS24433, AS24434, AS24436, AS24437 remarks: List of customers on AARNet3 using public AS numbers remarks: http://www.aarnet.edu.au admin-c: MP151 tech-c: ANOC-AP mnt-by: MAINT-ASAARNET changed: [email protected] 20050819 source: RADB 16 © 2005, AARNet Pty Ltd AS Set Object (2) • RPSL has hierarchical names as-set: descr: members: remarks: remarks: admin-c: tech-c: mnt-by: changed: source: 17 AS7575:AS-CUSTOMER AARNet3 customers AS set AS7575:AS-EDGE, AS7575:AS-RNO List of customers on AARNet3 using public AS numbers http://www.aarnet.edu.au MP151 ANOC-AP MAINT-ASAARNET [email protected] 20050819 RADB © 2005, AARNet Pty Ltd Autonomous System Object • Routing Policy Description object • Most important components are – import – export • These define the incoming and outgoing routing announcement relationships • Instant Documentation! • whois –h whois.ra.net AS7575 18 © 2005, AARNet Pty Ltd Whois queries • whois –h whois.ra.net AS7575:CUSTOMER – members: AS7575:AS-EDGE, AS7575:AS-RNO • whois –h whois.ra.net AS7575:AS-EDGE – members: AS1851, AS4822, AS6262, AS7575, AS7645, AS10148, AS17498, AS23654, AS23719, AS24101, AS24390, AS24431, AS24433, AS24434, AS24436, AS24437 • whois –h whois.ra.net \!gAS1851 – 192.43.227.0/24 192.43.229.0/24 192.43.228.0/24 192.43.227.0/24 192.43.229.0/24 19 © 2005, AARNet Pty Ltd 129.127.0.0/16 203.9.156.0/24 129.127.0.0/16 192.43.228.0/24 203.9.156.0/24 Whois (2) • whois –h whois.ra.net AS7575:AS-PEER – members: AS24, AS42, AS174, AS226, AS297, AS703, AS1273, AS1982, AS2044, AS2152, AS2497, AS2516, AS3130, AS3303, AS3491, AS3557, AS3643, AS3699, AS3742, AS3786, AS3856, AS4134, AS4355, AS4513, AS4565, AS4716, AS4725, AS4739, AS4766, AS4788, AS5726, AS6327, AS6517, AS6539, AS6939, AS7132, AS8075, AS8121, AS8404, AS9156, AS9264, AS9277, AS9318, AS9505, AS10310, AS10557, AS11404, AS11726, AS11841, AS12111, AS12222, AS14277, AS14361, AS15169, AS15290, AS15412, AS16713, AS18530, AS21947, AS22822, AS23260, AS23265, AS23504, AS25700, AS25973, AS26228, AS27008, AS27318, AS29814, AS30092, AS31800, AS33529 20 © 2005, AARNet Pty Ltd Whois (3) •whois –h whois.ra.net \!gAS8075 A488 207.46.128.0/18 207.46.192.0/18 204.95.110.0/23 207.68.128.0/18 204.255.246.0/23 198.105.232.0/22 131.107.0.0/16 207.46.128.0/18 207.46.192.0/18 204.95.110.0/23 207.68.128.0/18 204.255.246.0/23 198.105.232.0/22 131.107.0.0/16 207.46.32.0/20 205.248.96.0/19 204.95.96.0/20 207.68.128.0/18 207.46.0.0/20 207.46.208.0/20 192.197.157.0/24 199.60.28.0/24 199.103.122.0/24 65.55.224.0/19 199.103.90.0/23 65.54.112.0/20 65.54.96.0/20 207.46.96.0/19 207.68.160.0/19 65.54.192.0/19 65.54.128.0/19 C • Can now build inbound prefix filters appropriately 21 © 2005, AARNet Pty Ltd Use of RPSL • Use RtConfig v4 (part of RAToolSet from ISI) to generate filters based on information stored in our routing registry – Avoid filter errors (typos) – Filters consistent with documented policy (need to get policy correct though) – Currently we use RAToolSet v 4.7.1 – Need to script our own tools for Procket and Juniper 22 © 2005, AARNet Pty Ltd Using RPSL to configure routers • Need to define “policy” for filtering – Inbound from customers & peers – Outbound to customers & peers • Need to be aware of shortcomings in router configuration and/or configuration generator – Command line length (on cisco this is 512 bytes) – Complexity of rules 23 © 2005, AARNet Pty Ltd AARNet’s filtering philosophy • Inbound – – – – Filter customer by prefix and AS path Filter peer by prefix filter Filter providers for prefixes longer than a /24 Don’t accept martians from anyone • Outbound – Filter by BGP community, which indicates the class of the prefix (customer, peer, etc) 24 © 2005, AARNet Pty Ltd RtConfig & IRRToolSet • Version 4.0 supports RPSL • Generates cisco configurations • Contributed support for Bay’s BCC, Juniper’s Junos and Gated/RSd • Creates route and AS path filters. • Can also create ingress/egress filters 25 © 2005, AARNet Pty Ltd RFC 1998 - Use of BGP communities import: 26 { from AS-ANY action community.append(7575:1000); } refine { from AS-ANY action pref=30; accept community.contains(7575:70); from AS-ANY action pref=20; accept community.contains(7575:80); from AS-ANY action pref=10; accept community.contains(7575:90); from AS-ANY action pref=0; accept ANY; © 2005, AARNet Pty Ltd RFC1998 (2) } refine { from AS65510 at 202.158.192.241 action community.append(7575:2241, 7575:3006, 7575:5001); accept { 134.7.0.0/16, 130.116.160.0/21,130.116.168.0/24, 139.230.159.0/24, 150.229.207.128/25 } AND <^PeerAS+$>; • 27 Now the routes are correctly tagged and the RFC1998 policy applied. © 2005, AARNet Pty Ltd Blackholes import: { from AS-ANY action community.append(7575:1000); accept ANY; } refine { from AS-ANY action next-hop=192.168.1.1; accept community.contains(7575:6); } refine { from AS65510 at 202.158.192.241 action community.append(7575:2241, 7575:3006, 7575:5001); accept { 134.7.0.0/16, 130.116.160.0/21,130.116.168.0/24, 139.230.159.0/24, 150.229.207.128/25 }^32 AND <^PeerAS+$>; } from AS24437 at 202.158.192.250 action community.append(7575:2250, 7575:3006); accept PeerAS^32 AND <^PeerAS+$>; 28 © 2005, AARNet Pty Ltd RtConfig command line options • Defaults to using RADB – -h whois.ra.net – -p 43 – -protocol irrd • Defaults to “cisco” style output – -config cisco • -suppress_martian • -s <list of IRR sources> – -s CCAIR,RADB,CW 29 © 2005, AARNet Pty Ltd RtConfig Configuration Template (1) ! RtConfig template for cpe-curtin-er1 router in AS7575 ! @RtConfig set cisco_map_first_no = 10 @RtConfig set cisco_map_increment_by = 10 @RtConfig set cisco_prefix_acl_no = 100 @RtConfig set cisco_aspath_acl_no = 130 @RtConfig set cisco_pktfilter_acl_no = 130 @RtConfig set cisco_community_acl_no = 30 @RtConfig set cisco_max_preference = 100 ! no ip access-list extended DENY-BOGON-SOURCE ip access-list extended DENY-BOGON-SOURCE @RtConfig printPrefixRanges " deny ip %p %K any\n" filter fltr-bogons permit ip any any ! 30 © 2005, AARNet Pty Ltd RtConfig Configuration Template (2) ! Curtin University ! router bgp 7575 neighbor 202.158.198.186 remote-as 65510 neighbor 202.158.198.186 description Curtin University neighbor 202.158.198.186 send-community neighbor 202.158.198.186 soft-reconfiguration inbound neighbor 202.158.198.186 ebgp-multihop 2 @RtConfig set cisco_map_name = "AS%d-IPv4-1-IMPORT" @RtConfig import AS7575 202.158.192.241 AS65510 202.158.198.186 @RtConfig set cisco_map_name = "AS%d-IPv4-1-EXPORT" @RtConfig export AS7575 202.158.192.241 AS65510 202.158.198.186 ! end 31 © 2005, AARNet Pty Ltd Cisco Configuration (1) ip ip ip ip ip ip ! ip ip ip ip ip ip ip ip ! ip ip ! ip ip 32 prefix-list prefix-list prefix-list prefix-list prefix-list prefix-list pl100 pl100 pl100 pl100 pl100 pl100 seq seq seq seq seq seq 5 permit 130.116.160.0/21 ge 32 10 permit 130.116.168.0/24 ge 32 15 permit 134.7.0.0/16 ge 32 20 permit 139.230.159.0/24 ge 32 25 permit 150.229.207.128/25 ge 32 30 deny 0.0.0.0/0 le 32 prefix-list prefix-list prefix-list prefix-list prefix-list prefix-list prefix-list prefix-list pl101 pl101 pl101 pl101 pl101 pl101 pl101 pl101 seq seq seq seq seq seq seq seq 5 permit 130.116.160.0/21 10 permit 130.116.168.0/24 15 permit 134.7.0.0/16 20 permit 134.7.230.0/24 25 permit 134.7.254.144/28 30 permit 139.230.159.0/24 35 permit 150.229.207.128/25 40 deny 0.0.0.0/0 le 32 prefix-list pl102 seq 5 permit 0.0.0.0/0 prefix-list pl102 seq 10 deny 0.0.0.0/0 le 32 prefix-list pl103 seq 5 permit 0.0.0.0/0 le 24 prefix-list pl103 seq 10 deny 0.0.0.0/0 le 32 © 2005, AARNet Pty Ltd Cisco Configuration (2) route-map AS65510-IPv4-1-IMPORT permit 10 match ip address prefix-list pl100 match as-path 130 match community 30 set ip next-hop 192.168.1.1 set community 7575:1000 7575:2241 7575:3006 7575:5001 additive ! route-map AS65510-IPv4-1-IMPORT permit 20 match ip address prefix-list pl101 match as-path 130 match community 31 set local-preference 70 set community 7575:1000 7575:2241 7575:3006 7575:5001 additive ! route-map AS65510-IPv4-1-IMPORT permit 30 match ip address prefix-list pl101 match as-path 130 match community 32 set local-preference 80 set community 7575:1000 7575:2241 7575:3006 7575:5001 additive ! 33 © 2005, AARNet Pty Ltd route-map AS65510-IPv4-1-IMPORT permit 40 match ip address prefix-list pl101 match as-path 130 match community 33 set local-preference 90 set community 7575:1000 7575:2241 7575:3006 7575:5001 additive ! route-map AS65510-IPv4-1-IMPORT permit 50 match ip address prefix-list pl101 match as-path 130 set local-preference 100 set community 7575:1000 7575:2241 7575:3006 7575:5001 additive Using RtConfig • RtConfig –cisco_use_prefix_lists < cpe-curtin-er1.rtconfig • Redirect output to a file • Upload by tftp to the router • Done! 34 © 2005, AARNet Pty Ltd Problems? • Policy can easily get very complex and result in even more complex router configuration • Line limit on cisco AS path filters (need to be careful when using as-sets) • Limited non-Cisco support • Need to develop scripts to implement on Procket and Juniper 35 © 2005, AARNet Pty Ltd Where next? • • • • • RPSLng http://www.radb.net/rpslng-08.html Adds IPv6 and multicast extensions to RPSL RADB & RIPE have implemented support Implemented in recent releases of IRRToolSet – ftp://ftp.isc.org/isc/IRRToolSet/IRRToolSet-4.8.2/ 36 © 2005, AARNet Pty Ltd References • RPSL - RFC 2622 – http://www.faqs.org/rfcs/rfc2622.html • Using RPSL in Practice - RFC 2650 – http://www.faqs.org/rfcs/rfc2650.html • IRRToolSet – ftp://ftp.isc.org.net/isc/IRRToolSet/ • RPSL Training Page – http://www.isi.edu/ra/rps/training • RADB – http://www.radb.net/ 37 © 2005, AARNet Pty Ltd Thank you! Any Questions?