Transcript RIPE Database Operations Update Shane Kerr RIPE NCC Shane Kerr . RIPE 45, May 2003, Barcelona . http://www.ripe.net.

RIPE Database
Operations Update
Shane Kerr
RIPE NCC
<[email protected]>
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
1
http://www.ripe.net
Outline
Statistics
Operations
Developments
Plans
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
2
http://www.ripe.net
Facts and Figures
• 1.9 M objects, 42% inetnum, 47% person
• 28 q/s on average, spikes of 200 q/s
• 2.3 updates/min, 61% inetnum, 28% person
Object counts, query loads, etc. available on-line:
http://www.ripe.net/db/dbconstat
http://www.ripe.net/db/mrtg/whois.html
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
3
http://www.ripe.net
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
20
20
20
20
20
20
20
20
20
20
20
20
20
20
19
19
19
19
19
03
03
02
02
02
02
01
01
01
01
00
00
00
00
99
99
99
99
98
-05
-02
-11
-08
-05
-02
-11
-08
-05
-02
-11
-08
-05
-02
-11
-08
-05
-02
-11
-03
-03
-03
-03
-03
-03
-03
-03
-03
-03
-03
-03
-03
-03
-03
-03
-03
-03
-03
Database Contents
Historical
6000000
5000000
4000000
3000000
person/role
domain
inetnum
other
2000000
1000000
0
http://www.ripe.net
4
Database Contents
Recent “Big Movers”
6%
6%
4%
irt (+6)
rtr-set (+1)
inet6num (+305)
3%
3%
9%
3%
2%
3%
2%
13%
10%
2%
2%
1%
0%
15%
24%
Shane Kerr
.
RIPE 45, May 2003, Barcelona
filter-set (+4)
key-cert (+157)
route-set (+32)
peering-set (+6)
as-set (+161)
inetnum (+43972)
mntner (+442)
route (+1309)
aut-num (+278)
domain (+6571)
person/role (+32905)
limerick (+4)
inet-rtr (+2)
as-block (-1)
.
5
http://www.ripe.net
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
20
20
20
20
20
20
20
20
20
20
20
20
20
03
03
03
03
03
03
03
03
03
03
03
03
03
-0
4
-0
4
-0
4
-0
4
-0
3
-0
3
-0
3
-0
3
-0
3
-0
2
-0
2
-0
2
-0
2
-2
6
-1
9
-1
2
-0
5
-2
9
-2
2
-1
5
-0
8
-0
1
-2
2
-1
5
-0
8
-0
1
Number of Updates
Updates by Method
8000
7000
6000
5000
4000
DIRECT
SYNC
MAIL
3000
2000
1000
0
http://www.ripe.net
6
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
2003-04-26
2003-04-19
2003-04-12
2003-04-05
2003-03-29
2003-03-22
2003-03-15
2003-03-08
2003-03-01
2003-02-22
2003-02-15
2003-02-08
2003-02-01
Percentage of Updates
Synchronous Update Usage
50
45
40
35
30
25
20
15
10
5
0
http://www.ripe.net
7
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
http://www.ripe.net
20
03
03
03
03
02
02
02
02
02
02
02
02
02
02
02
02
01
01
01
01
01
01
01
01
01
-04
-03
-02
-01
-12
-11
-10
-09
-08
-07
-06
-05
-04
-03
-02
-01
-12
-11
-10
-09
-08
-07
-06
-05
-04
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
-23
Number of Queries
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
How Many Queries are There?
Queries/Second
4500000
60
4000000
50
3500000
3000000
40
2500000
30
2000000
1500000
20
1000000
10
500000
0
0
8
What are the Queries For?
19%
IP
Domain
18%
Domain Referral
59%
Other
4%
Query Types 2003-02-01 to 2003-04-30
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
9
http://www.ripe.net
01
-04
01
-2
-05 3
20
01 -23
20 06-2
01
- 3
20 07-2
01
- 3
20 08-2
01
3
20 -09-2
01
- 3
20 10-2
01
3
20 -11-2
01
- 3
20 12-2
02
3
20 01-2
02
3
20 -0202 23
20 03-2
02
3
20 -04-2
02
- 3
20 05-2
02
3
20 06-2
02
- 3
20 07-2
02
- 3
20 08-2
02
3
20 -09-2
02
- 3
20 10-2
02
3
20 -11-2
02
- 3
20 12-2
03
3
20 01-2
03
3
20 -0203 23
20 03-2
03
3
-04
-23
20
20
What are Queries Returning?
10000000
9000000
8000000
7000000
6000000
5000000
Private Objects
Public Objects
4000000
3000000
2000000
1000000
0
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
http://www.ripe.net
10
Where are the Queries From?
Unique IP's for Whois Queries/Day
80000
70000
60000
50000
Real + Proxy IP
Real IP
40000
30000
20000
10000
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
200
3 -0
4-2
6
200
3 -0
4-1
9
200
3 -0
4-1
2
200
3 -0
4-0
5
200
3 -0
3-2
9
200
3 -0
3-2
2
200
3 -0
3-1
5
200
3 -0
3-0
8
200
3 -0
3-0
1
200
3 -0
2-2
2
200
3 -0
2-1
5
200
3 -0
2-0
8
200
3 -0
2-0
1
0
11
http://www.ripe.net
Outline
Statistics ✓
Operations
Developments
Plans
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
12
http://www.ripe.net
Database Operations
• Smooth sailing
– average response time is well below 1 second
– load approximately 70% higher than last year
• New ARIN mirroring scheme next week
– Use difference to generate NRTM, rather than full load
– No down-time, includes allocations and assignments
• Tertiary (i.e. off-site) server in planning phases
– Withstand catastrophic, city-wide failures
– Manual failover by updating DNS for whois.ripe.net
– Part of larger project to increase critical service reliability
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
13
http://www.ripe.net
How Responsive is the Server?
InnoDB table type goes on-line
Unoptimised MySQL
Desktop Upgrade
Shane Kerr
.
RIPE 45, May 2003, Barcelona
RIPE NCC DDoS
.
14
http://www.ripe.net
Unreferenced Person Cleanup
• Protect privacy of users
– Person objects remain undeleted when contacts change
• Person objects unreferenced for 90 will be deleted
– A modify of the object “resets” the counter
• E-mail sent after 60 days
– Limit to 2000/day to avoid overwhelming users & server
• Deletions begin 29 May 2003
http://www.ripe.net/db/unref-cleanup-200304.html
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
15
http://www.ripe.net
IPv6 query proxy status
• Operational issues resolved
– Solaris patches and move to Sun box
– DNS update hid AAAA record for 10 days
– Moving to standard monitoring and failover
• Source code public release made
• 90% of queries come from only 7 IP addresses
• Still see more 6bone than RIR addresses
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
16
http://www.ripe.net
RIPE Database Administration
Tiago Antao
Can Bican
Engin Gündüz
Katie Petrusha
Denis Walker
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
17
http://www.ripe.net
Recent RIPE DBM Activity
• Tickets, tickets, tickets
– About 90/day
• Spam becoming a larger problem
– Planning move to SpamAssassin
– Will reply if filtered – no e-mails dropped!
• Hostmaster taking over 20% of RIPE DBM
– Closer co-ordination between groups
• [email protected] is an open, general help mailbox
Please e-mail Questions or Problems!
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
18
http://www.ripe.net
ERX Update
• 4 /8’s transferred
– 141.0.0.0/8 and 151.0.0.0/8, RIPE NCC-majority
– 150.0.0.0/8 and 163.0.0.0/8, APNIC-majority
– Multiple transfers, with overlapping start/end dates
• One more trial to finalise processing
• “Long burn” scheduled to start afterwards
– No large issues
– Complete ordering, tentative scheduling TBD
• Moving to regular RIPE DBM and HM activity
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
19
http://www.ripe.net
Outline
Statistics ✓
Operations ✓
Developments
Plans
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
20
http://www.ripe.net
New Update Processing
• Improved acknowledgement messages
• Software maintenance eased
• Presentations at RIPE 45:
– Developer POV at Tools Working Group
– User POV at Database Working Group
http://www.ripe.net/db/dbupdate/
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
21
http://www.ripe.net
RPSLng
• IPv6 and multicast RPSL objects
• Prototype server on-line
– Based on last RPSLng draft
– Throw-away copy of RIPE Database
– Latest draft to be integrated Real Soon Now
• IRRToolSet changes
– More complicated than server changes
– Coming soon!
http://www.ripe.net/ripencc/pub-services/db/rpslng/
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
22
http://www.ripe.net
Semantic Changes
• Lots of mostly minor changes proposed:
–
–
–
–
–
default to protected inetnum/inet6num/domain
notification for more-specific
removal of cross notifications
"reclaim:"-like functionality
"mnt-lower:" on set objects
• Little discussion
• Will be on-line soon
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
23
http://www.ripe.net
Outline
Statistics ✓
Operations ✓
Developments ✓
Plans
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
24
http://www.ripe.net
Future Plans
• RPSLng IRRToolSet support
• Update processing semantic changes
• “status:” attribute changes?
• Deprecating NONE?
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
25
http://www.ripe.net
PKI
• Draft RIPE document, Improved Secure
Communication System for RIPE NCC Members
• LIR-Portal ready, time for Database integration
• New features needed:
– X.509 authentication
– Proxy authentication
• Proposals coming to mailing list
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
26
http://www.ripe.net
Outline
Statistics ✓
Operations ✓
Developments ✓
Plans ✓
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
27
http://www.ripe.net
Shane Kerr
.
RIPE 45, May 2003, Barcelona
.
28
http://www.ripe.net