New Version of the RIPE Database Andrei Robachevsky RIPE NCC Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net.
Download ReportTranscript New Version of the RIPE Database Andrei Robachevsky RIPE NCC Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net.
New Version of the RIPE Database Andrei Robachevsky RIPE NCC <[email protected]> 1 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Outline • Current status of the RIPE Database • New database software • Migration timeline • More information 2 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net RIPE Database Status • Contains • IP allocations/assignments • Domain registry • Routing registry • 3.7 Million objects • 80% person, 10% inetnum, 0.65% route • 6,700 updates/day • 770,000 queries/day (9 queries/s) • 38% IP addresses, 1% IP prefixes 3 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Distribution by object type (February 2001) route 0,66% person 78,62% Other 1,09% domain 10,43%role 0,11% inetnum 9,87% as-macro 0,04% aut-num mntner 0,11% 0,15% 4 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Queries =~ 9/sec average 9/sec 25.000.000 20.000.000 15.000.000 10.000.000 5.000.000 ap r99 ju n99 au g99 ok t-9 9 de c99 fe b00 ap r00 ju n00 au g00 ok t-0 0 de c00 fe b01 0 5 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net % of queries by object type (February 2001) domains 27% other 29% domains IP prefixes other prefixes 1% IP 43% 6 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Updates 21/min -> 5/min 1.000.000 900.000 800.000 700.000 600.000 500.000 400.000 300.000 200.000 100.000 0 9 0 0 1 9 0 9 0 9 9 0 0 9 0 9 0 0 0 9 0 9 9 0 0 t t r r c b g n ug n k k p p e ec feb e u u u o f o a j a j d d a a 7 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net RIPE Database • Whois service • http://www.ripe.net/ripencc/pub-services/db/ • Database Consistency Project • http://www.ripe.net/ripencc/pub-services/db/state/ • Routing Registry Consistency Check • http://www.ripe.net/ripencc/pub-services/db/rrcc/ 8 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net What’s wrong with current version? It’s good old software, but... • RIPE-181 for routing policy description • Lack of IRR security • Poor scalability • Performance limits • Hard to maintain 9 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net New version of the RIPE Database • Supports RPSL (RFC2622) • Extended syntax • New objects and attributes • Supports RPSS (RFC2725) • New authorization rules • Supports RAToolset • RtConfig -protocol bird • Code is completely rewritten 10 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net RPSL Support • Extended syntax rules apply to all object types • end of line comments • line continuation • order of attributes • New objects person: source: nic-hdl: address: + remarks: Test Person Object TEST TP-TEST # nic handle Nobody knows where he lives… be prepared to parse one • as-set (as-macro), route-set (community) • peering-set, filter-set, rtr-set • New attributes • member-of • mbrs-by-ref 11 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net RPSS support • New object • as-block • New attributes • mnt-routes: <mnt_name> [ rpsl list of prefixes | ANY] • referral-by: <mnt_name> • auth-override: YYYYMMDD • New authorization rules • route creation • aut-num • hierarchical names 12 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net RAToolset Support • New queries • -l <ip range> • -x <ip range> • -K • RtConfig -protocol bird • Patch is available • to parse RIPE-style comments (%) • ftp://ftp.ripe.net/ripe/dbase/software/RAToolSet/ 13 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net New software • Mainly in C, multithreaded • RDBMS as a back-end • MySQL, transaction support • In-memory radix tree for IP lookups • also more and less specific lookups for reverse delegation domains • MIME and GPG support • correct PGP keys are also accepted • Automatic access control • separate accounting for public and contact data 14 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Server architecture Message queues Syntax checks, acks, notifications queries Queue rules RDBMS Update FE Core Server E-mail Update FE NRTM clients Mirror Server RDBMS 15 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net What’s different ? • • • • • person: Test Person Object Modified objects: source: TEST mntner nic-hdl: TP-TEST # nic handle Access New control: objects: route address: Nobody knows where %ERROR:202: access control limit reached as-block New NRTM New protocol: attributes: % Youaut-num have reached the of returned contact information objects. helimit lives… rtr-set RDBMS (MySQL): member-of New query flags: %+ Thisas-set connection will be terminated now. (was: as-macro) peering-set CREATE TABLE mntner ( was: mbrs-by-ref -l <ip range> % remarks: Continued attempts to (was: return excessive amounts of contact route-set community) be prepared thread_id int(11) DEFAULT '0' NOT NULL, to parse filter-set UPD = (ADD mnt-routes + DEL) % information will-x result in permanent denial of service. <ip inet-rtr onerange> object_id int(10) unsigned DEFAULT '0' NOT NULL, referral-by -K DEFAULT '' NOT NULL, mntner varchar(80) inetnum will dummy be: tinyint(4) auth-override -d DEFAULT '0' NOT NULL, UPD =PRIMARY ADD KEY -q (object_id) sources [<source>] ); -q version Extended object syntax Modified objects New attributes New objects New query flags • New access control • New database format • New version of the mirroring protocol 16 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Who will be affected ? • Query users • new query flags • Update users • new syntax rules • new authorization rules • Scripts • new object format and syntax • new/modified objects and attributes • access control • NRTM clients • new software • new version of the mirroring protocol 17 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Transition timeline - Updates Production Prototype/Compatibility RIPE181 Updates in RIPE-181 to <[email protected]> Updates in RPE-181 to <[email protected]> Updates in RPSL to <[email protected]> RPSL TEST Updates in RIPE-181 to <[email protected]> Updates in RPSL to <[email protected]> Proposed dates: X=23 April Updates in RPSL to <[email protected]> Updates in RPSL to <[email protected]> Y=14 May Z=15 October 18 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Transition timeline - Queries Production Prototype RIPE-181 v2.x Querying RIPE DB in RIPE-181 at whois.ripe.net :43 RPSL v3.0 Querying RIPE DB in RPSL at rpsl.ripe.net :43 Additional flags available Proposed date: Querying RIPE DB in RPSL at whois.ripe.net : 43 Additional flags available X=23 April 19 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Transition timeline - NRTM Production Prototype RIPE181 v2.x Mirroring RIPE DB in RIPE-181 at whois.ripe.net :43 RPSL v3.0 Mirroring RIPE DB in RPSL at rpsl.ripe.net :4444 Proposed date: Mirroring RIPE DB in RPSL at whois.ripe.net : 4444 X=23 April 20 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Project Status • • • • • Version 3.0ß2 has been released Core server functionality is complete Infrastructure is under development Testing is in progress Portability issues are on our list • Solaris, Linux, FreeBSD, UnixWare(?), ... • Thanks to everyone who helps make it more portable • Special thanks to George Michaelson! 21 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Prototype servers • Near real-time mirror of the RIPE Database • whois -h rpsl.ripe.net • contains live RIPE Database in RPSL format • Test server for submissions • mail <[email protected]> • whois -h rpsl.ripe.net -p 4343 • NRTM • rpsl.ripe.net, port 4444 • please contact <[email protected]> 22 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net More Information • RIPE-181 to RPSL Migration page • http://www.ripe.net/rpsl • Documentation • • • • Transition to the RIPE DB v3.0 Whois Queries in the RIPE DB v3.0 Updates in the RIPE DB v3.0 Error codes in the RIPE DB v3.0 • Software • New whois client ftp://ftp.ripe.net/ripe/dbase/reimp/whoisRIP-1.0.tar.gz • Server software v3.0 http://www.ripe.net/ripencc/pub-services/db/reimp/latestbeta.html 23 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Questions? 24 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net New Version of the RIPE Database Andrei Robachevsky RIPE NCC <[email protected]> 25 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net New objects • • • • peering-set filter-set rtr-set as-block 26 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net New attributes • RPSL: • member-of, mbrs-by-ref • RPS-auth: • mnt-routes: <mnt_name> [ rpsl list of prefixes | ANY] • referral-by: <mnt_name> • auth-override: YYYYMMDD 27 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Modifications to all objects • • • • • Line continuation Attribute order is relevant Support for end of line comments Handling of empty attributes Legend: holes: member-of: cross-nfy: community: [optional] [optional] [optional] [optional] [multiple] [multiple] [multiple] [multiple] automatically translated new preserved deprecated 28 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Modified objects • mntner object mntner: descr: admin-c: tech-c: upd-to: mnt-nfy: auth: remarks: notify: mnt-by: auth-override: referral-by: changed: source: [mandatory] [mandatory] [mandatory] [optional] [mandatory] [optional] [mandatory] [optional] [optional] [mandatory] [optional] [mandatory] [mandatory] [mandatory] [single] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [single] [single] [multiple] [single] [primary/look-up key] [ ] [inverse key] [inverse key] [inverse key] [inverse key] [ ] [ ] [inverse key] [inverse key] [ ] *** RPS auth *** [inverse key] *** RPS auth *** [ ] [ ] 29 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Modified objects • route object route: [mandatory] descr: [mandatory] origin: [mandatory] holes: [optional] withdrawn: [optional] comm-list: [optional] advisory: [optional] member-of: [optional] inject: [optional] aggr-mtd: [optional] aggr-bndry: [optional] export-comps:[optional] components: [optional] cross-nfy: [optional] community: [optional] mnt-lower: [optional] mnt-routes: [optional] mnt-by: [mandatory] changed: [mandatory] source: [mandatory] [single] [multiple] [single] [multiple] [single] [multiple] [multiple] [multiple] [multiple] [single] [single] [single] [single] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [single] [primary/look-up key] [ ] [primary/inverse key] [ ] *** hole in RIPE 181 *** [ ] [ ] [ ] [inverse key] *** RPSL *** [ ] *** RPSL *** [ ] *** RPSL *** [ ] *** RPSL *** [ ] *** RPSL *** [ ] *** RPSL *** [inverse key] [ ] [inverse key] *** RPS auth *** [inverse key] *** RPS auth *** [inverse key] [ ] [ ] 30 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Modified objects • autnum object aut-num: as-name: descr: as-in: as-out: interas-in: interas-out: as-exclude: member-of: import: export: default: remarks: admin-c: tech-c: cross-mnt: cross-nfy: notify: mnt-lower: mnt-routes: mnt-by: changed: source: [mandatory] [mandatory] [mandatory] [optional] [optional] [optional] [optional] [optional] [optional] [optional] [optional] [optional] [optional] [mandatory] [mandatory] [optional] [optional] [optional] [optional] [optional] [mandatory] [mandatory] [mandatory] [single] [single] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [single] [primary/look-up key] [ ] [ ] [ ] [ ] [ ] [inverse key] *** New in RPSL *** *** as-in in RIPE 181 *** *** as-out in RIPE 181 *** [inverse [inverse [inverse [inverse [inverse [inverse [inverse [inverse key] key] key] key] key] key] key] key] *** RPS auth *** *** RPS auth *** 31 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Modified objects • as-set (previously as- macro) as-set: descr: members: mbrs-by-ref: remarks: tech-c: admin-c: notify: mnt-by: changed: source: [mandatory] [mandatory] [optional] [optional] [optional] [mandatory] [mandatory] [optional] [mandatory] [mandatory] [mandatory] [single] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [single] [primary/look-up key] *** as-macro in RIPE 181 *** *** as-list in RIPE 181 *** [inverse key] *** New in RPSL *** [inverse [inverse [inverse [inverse key] key] key] key] 32 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Modified objects • route-set (previously community) route-set: descr: members: mbrs-by-ref: remarks: tech-c: admin-c: notify: mnt-by: changed: source: [mandatory] [mandatory] [optional] [optional] [optional] [mandatory] [mandatory] [optional] [mandatory] [mandatory] [mandatory] [single] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [single] [primary/look-up key] *** community in RIPE 181 *** [inverse key] [inverse [inverse [inverse [inverse *** New in RPSL *** *** New in RPSL *** key] key] key] key] 33 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Modified objects • inet-rtr inet-rtr: descr: alias: local-as: ifaddr: peer: member-of: remarks: admin-c: tech-c: notify: mnt-by: changed: source: [mandatory] [mandatory] [optional] [mandatory] [mandatory] [optional] [optional] [optional] [mandatory] [mandatory] [optional] [mandatory] [mandatory] [mandatory] [single] [multiple] [multiple] [single] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [single] [primary/look-up key] [inverse key] [look-up key] [inverse key] [inverse [inverse [inverse [inverse *** New in RPSL *** *** localas in RIPE 181 *** *** New in RPSL *** key] key] key] key] 34 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Modified objects • inetnum inetnum: netname: descr: country: admin-c: tech-c: rev-srv: status: remarks: notify: mnt-by: mnt-lower: mnt-routes: changed: source: [mandatory] [mandatory] [mandatory] [mandatory] [mandatory] [mandatory] [optional] [generated] [optional] [optional] [mandatory] [optional] [optional] [mandatory] [mandatory] [single] [single] [multiple] [multiple] [multiple] [multiple] [multiple] [single] [multiple] [multiple] [multiple] [multiple] [single] [multiple] [single] [primary/look-up key] [lookup key] [ ] [ ] [inverse key] [inverse key] [inverse key] [ ] [ ] [inverse key] [inverse key] [inverse key] [inverse key] *** RPS auth *** [ ] [ ] 35 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net New object: peering-set • Peering-set peering-set: descr: peering: remarks: tech-c: admin-c: notify: mnt-by: changed: source: [mandatory] [mandatory] [mandatory] [optional] [mandatory] [mandatory] [optional] [mandatory] [mandatory] [mandatory] [single] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [single] [primary/look-up key] [inverse [inverse [inverse [inverse key] key] key] key] <= 36 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net New object: filter-set • defines a set of routes that are matched by its filter filter-set: descr: filter: remarks: tech-c: admin-c: notify: mnt-by: changed: source: [mandatory] [mandatory] [mandatory] [optional] [mandatory] [mandatory] [optional] [mandatory] [mandatory] [mandatory] [single] [multiple] [single] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [single] [primary/look-up key] [inverse [inverse [inverse [inverse key] key] key] key] <= 37 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net New object: rtr-set • defines a set of routers specified by inet-rtr names, ipv4_addresses or other rtr-set names rtr-set: descr: members: mbrs-by-ref: remarks: tech-c: admin-c: notify: mnt-by: changed: source: [mandatory] [mandatory] [optional] [optional] [optional] [mandatory] [mandatory] [optional] [mandatory] [mandatory] [mandatory] [single] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [single] [primary/look-up key] [inverse [inverse [inverse [inverse key] key] key] key] <= 38 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net New object: as-block • Defines a range of AS numbers delegated to a given repository as-block: descr: remarks: tech-c: admin-c: notify: mnt-lower: mnt-by: changed: source: [mandatory] [optional] [optional] [mandatory] [mandatory] [optional] [optional] [mandatory] [mandatory] [mandatory] [single] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [multiple] [single] [primary/look-up key] [inverse [inverse [inverse [inverse [inverse key] key] key] key] key] <= 39 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Queries • New queries • • • • • • -l <ip range> -x <ip range> -K -d -q sources [<source>] -q version • Inverse queries • Other differences 40 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net -l <ip range> • One level less specific • Does not return the exact match • Returns the smallest IP range that is bigger than the supplied range and that fully contains it • whois -r -Tin 193.0.0.0/23 • whois -r -Tin -l 193.0.0.0/23 • whois -r -Tin -L 193.0.0.0/23 41 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net -x <ip range> • • • • Exact match If no matching object is found nothing is returned whois -r -Tin 193.0.2.0/24 whois -r -Tin -x 193.0.2.0/24 42 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net -K • Only primary keys are returned • Exception is a set object, where the members attribute is also returned • Does not apply to person and role objects • whois -Trt -K -M 193.0.0.0/16 • whois -K -imo RS-HEPNET • whois -K AS-WORLD 43 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net -d (proposed) • Triggers inclusion of in-addr.arpa and ip6.int domain objects in the result of IP lookup • More/less specific lookups are possible • whois -r -d 193.0.2.0 • whois -d -Tdn -K -M 193.0.0.0/20 44 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Accounting and Access Control • Access to “public” and “contact” data is accounted differently • Is based on number of objects returned • • • • limit = f(max_limit1, query_rate) when limit is hit - the query is aborted and limit =0 limit recovers in time # of times the limit may be hit before permanent denial • Trusted proxies: accounting is based on client’s IP 45 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Authorization of route creation inetnum: 10.1.0.0 - 10.1.255.255 mnt-by: M1-MNT ... route: 10.1.0.0/16 mnt-by: M2-MNT ... mntner: M1-MNT auth: mntner: M2-MNT ... auth: ... mntner: M4-MNT auth: ... aut-num: AS65000 mnt-by: M3-MNT ... mntner: M3-MNT auth: ... route: 10.1.1.0/24 origin: AS65000 mnt-by: M4-MNT ... 46 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net Membership of set objects route-set: RS-FOO mbrs-by-ref: MNT-FOOBAR ... as-set: AS-BAR members: AS3333 mbrs-by-ref: MNT-FOOBAR ... route: origin: member-of: mnt-by: ... 193.0.0.0/22 AS3333 RS-FOO MNT-FOOBAR route: origin: member-of: mnt-by: ... 192.168.0.0/24 AS3333 RS-FOO OTHER-MNT aut-num: ... AS3333 aut-num: member-of: mnt-by: ... AS3267 AS-BAR MNT-FOOBAR 47 Andrei Robachevsky . APNIC/APRICOT2001, February 2001, Kuala Lumpur, Malaysia . http://www.ripe.net