Transcript ARIN Engineering Report Mark Kosters Engineering Theme • 2012 success is being aided by contractors (but not near as many) • The search is on.

ARIN Engineering
Report
Mark Kosters
Engineering Theme
• 2012 success is being aided by
contractors (but not near as many)
• The search is on to fill open
engineering slots
• Lots of work is done, but there is much
more to do
2
2
Staffing
• Operations
– 5 People + Manager (down 1 since ARIN 29)
– 2 DBA slots open
• Development
– 5 Developers + Manager
– 2 Contractors (down 3 more since ARIN 29)
• Quality Assurance
– 3 QA + Manager
– 4 Contractors
• Project Management
– 1 Filled! (up 1 since ARIN 29)
• Management
– 1 (me)
3
3
Operations
• Upgrading end-of-life equipment
• Maintaining the various environments we have
running (Production/OT&E/Dev/QA/Staging)
• Load Balancer challenges
• Moving production to the colocation facility
• IT support
• RPKI rollout
4
4
Whois-RWS Traffic Loads
• Running “normally” now at 405 queries
per second (QPS)
– down 70 QPS since last meeting
• RESTful calls have overtaken Port 43
calls since March
• 1.5 Billion RESTful calls for September
• 1.1 Billion Port 43 queries
5
5
2001-07
2001-10
2002-01
2002-04
2002-07
2002-10
2003-01
2003-04
2003-07
2003-10
2004-01
2004-04
2004-07
2004-10
2005-01
2005-04
2005-07
2005-10
2006-01
2006-04
2006-07
2006-10
2007-01
2007-04
2007-07
2007-10
2008-01
2008-04
2008-07
2008-10
2009-01
2009-04
2009-07
2009-10
2010-01
2010-04
2010-07
2010-10
2011-01
2011-04
2011-07
2011-10
2012-01
2012-04
2012-07
Queries Per Second
Whois-RWS Statistics
Queries on Port 43
4000.00
3500.00
6
3000.00
2500.00
2000.00
1500.00
1000.00
500.00
0.00
Months
6
Whois-RWS Statistics Queries
Total Queries (x10000)
1200000
1000000
800000
600000
400000
200000
0
Months
RESTful
Web
Whois Port 43
7
Whois-RWS – IPv6
Total Per Month
14000000
12000000
10000000
8000000
6000000
Port 80
4000000
Port 43
2000000
0
Month
8
Web Traffic V4 versus V6
2,815,272
IPv6
19,228,325
IPv4
12.77% Traffic on the website is IPv6
BUT….
9
Web Traffic IPv4 versus IPv6
893,698 IPv6
19,228,325
Ipv4
Take out the IPv6 connectivity testers
IPv6 is 4.05% of our total traffic
10
Development/QA
• Improvements to existing systems
• ARIN Online releases since ARIN XXVIII
– RPKI!
– Integrated payments
– Move from Red Hat JBoss to JBoss AS7
Community Edition
– Runout functionality enhancements for
staff
– Various minor bug fixes
11
11
Initiatives Currently Underway
• Implement delegated RPKI
• Extended statistics generation
• Improvements to internal billing
systems
• Move from Oracle to PostgreSQL
12
How is ARIN Online used?
• 62,998 accounts activated since
inception through Q3 of 2012
2012*
2011
2010
2009
2008
Number of Accounts Activated
10000
15000
5000
20000
* Through Q3 of 2012
13
Active Usage of ARIN Online
Logins
# of Users
30000
25000
20000
15000
10000
5000
0
0
1
2-5
6 - 10
11 - 15
>16
Times logged in
14
Reg-RWS (RESTful Provisioning)
At ARIN XXIX
Templates
REST
Transactions
0
200000 400000 600000 800000 1000000 1200000
Today
Templates
REST
Transactions
0
200000 400000 600000 800000 1000000 1200000
Cumulative totals since April, 2011
15
Evolution/Deployment of RPKI
• A brief look at
– Pilot participation
– Feedback
– The move to production
16
RPKI Pilot
• Pilot period
– Operational from 7/2009 until 9/2012
– 63 users
– 76 ROAs in the pilot
• Services are still hooked into the Pilot
– 15000 fetches per day at peak usage
– Let signatures expire after production
deployment
– 4000 fetches today on a empty repository
17
RPKI Pilot
• Feedback on the Pilot over the three
years
– Comments
• Just one -“weird passwords”
– Operational Learning
• People noticed for a time when signatures
expired
• A few data entries did not match global
routing entries
18
Production RPKI
• Getting the Trust Anchor
– 27 people have signed the RPA
– 17 people are non-RIR members
• Signing Resources
– 7 Organizations
– 19 ROAS
– 30 Networks/ASs
19
RPKI Results
• No real landrush to join
• Is it still in the experimental stage?
• Has been a multi-year effort
– $2.5 Million
– Very complex code
• Hope it has success in the long run
20
Maybe this will be like the IRR
• Low usage when started multi-month
project to upgrade the IRR
• Report from ARIN XXVIII showed low
usage which in turn called into
question the need to upgrade
21
IRR Usage From 2009-2Q/2011
Insertions/Changes
4000
2000
442 Insertions by
one maintainer
0
2009
2010
2Q/2011
400
200
In Year
Since 2009
0
Active Maintainers
22
Now the Traffic on IRR has gone
up after Upgrade in Sept 2011
• Maintainers
– 1,682 pre-conversion
– 1,812 today
• Routes
– 17,937 pre-conversion
– 20,513 today
• Networks
– 456 pre-conversion
– 539 today
23
RPKI Challenges
• Protocol is mature – kind of
– Validators do not allow for extension that we require (certificate
policies extension)
– Draft is to be written within the IETF to explicitly mention this
– Rsync may not be the best protocol to retrieve data from
repositories – quickly becomes a DDOS vector
– Work on a HTTP transport for getting data from repositories
• Challenges
– ERX and Inter-RIR Transfers
– Merging with the Global Trust Anchor
– Simultaneous operation of RIR Trust Anchor and Global Trust
Anchor
24
Schedule Pressure
– ACSP Suggestions (8 Pending)
– DNSSEC improvements
– Streamlined Transfer Service
– CMSD membership/voter functionality
– Integration of IRR within ARIN Online
– Lame Delegation reporting
– Additional OT&E services
– Alternative RPKI-like services
– Billing Management Improvements
25
Schedule Pressure
• Community needs/Policy
– Ways to better vet/implement community needs
– Need to hear from you
• Technical and Operational debt
– Many existing internal processes are inefficient and labor intensive
– Software changes
• Thought Leadership
– Whois-RWS
– RPKI
– Research
26
Comments?
27