Scalable Network Proximity Estimation Puneet Sharma ([email protected]) HP Labs, Palo Alto Joint Work with Sujata Banerjee, Sujoy Basu, Rodrigo Fonseca, Sung-Ju Lee, and Zhichen Xu Self-Managing.
Download ReportTranscript Scalable Network Proximity Estimation Puneet Sharma ([email protected]) HP Labs, Palo Alto Joint Work with Sujata Banerjee, Sujoy Basu, Rodrigo Fonseca, Sung-Ju Lee, and Zhichen Xu Self-Managing.
Scalable Network Proximity Estimation Puneet Sharma ([email protected]) HP Labs, Palo Alto Joint Work with Sujata Banerjee, Sujoy Basu, Rodrigo Fonseca, Sung-Ju Lee, and Zhichen Xu Self-Managing Networks Summit, June 1-2, 2005 © 2005 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice Network Proximity Estimation • Proximity estimation is key to finding “best” resources − closest game server, closest media service etc. − overlay neighbor selection, building distribution trees etc. • Challenges: − O(N2) probing overhead • Millions of globally distributed resources => 1012 probe messages − Dynamic environment => Faster re-computation 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 2 Related Work • Infrastructure support based − IDMaps, Dynamic Distance Map − King --- recursive DNS lookup − M-Coop – tracers link to each other that mimics BGP • Landmark based: − Landmark ordering • Cannot distinguish nodes with same (or similar) landmark order − GNP • Pre-computation of the landmark nodes • Sensitive to the measurement errors − Lighthouse • Quality depends on the choice of lighthouses − Vivaldi 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 3 Netvigator: Network Proximity Estimation • Efficient computation of the nodes in proximity to a given node − Find the “k closest nodes to a given node” • Proximity estimation not Distance estimation • Landmark based scheme: O(NL) v/s O(N2) measurement overhead • Uses widely deployed tools: ping, traceroute 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 4 Methodology: Landmarks-based Scheme d1 d1 d2 d2 d3… dn C1, C2, C3 d1d1 C1, C2, C3 d2 d2 …d3 dn d1 d2d2 … d3 C1, C2,d1 C3 dn Using Landmark Vectors Clustering instead Embed nodes in a of Global Embedding Cartesian Space using Landmark Vectors d1 d1 d2 d2 … d3 dn Leverage Route Information C1, C2, C3traceroute instead of ping Host 11/6/2015 Landmark C1, C2, C3 d1d1 d2d2 …d3 dn Landmark Vector Router (Milestone) Copyright © 2005 HP corporate presentation. All rights reserved. 5 NetVigator: Clustering Algorithms • min_sum minc C: l L(n,c) (dist(n,l) + dist(c, l)) • max_diff c maxc C: l L(n,c) ABS(dist(n,l) - dist(c, l)) • inner_product l l L(n,c) ((1/(dist(n,l)2) X (1/dist(c, l)2)) max∑c C: n min_sum: Minc C: l L(n,c) (dist(n,l) + dist(c, l)) 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 6 Netvigator Evaluation • Tested using large scale simulations, and implementation on the HP Intranet and PlanetLab. • Evaluation Metrics −Accuracy: Finding the best candidate in top “n” −Precision: How many correct top “n” ? −Penalty: How bad is the best found candidate? 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 7 Visualization using ZoomGraph: Top 5 closest nodes to planetlab1.cse.nd.edu planetlab1.cse.nd.edu NetVigator Result 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 8 Visualization using ZoomGraph: Partial Topology 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 9 Evaluation with PlanetLab data NetVigator performance •High accuracy: Over 90% accuracy •Low overhead: 15% measurement overhead •Robust to bad measurements 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 10 Distributed Querying of Internet Position Information Position Information Partitioning query specific search expansion A Query: closest node to me? Query: closest node to me? R(A) I.N. Distributed Infrastructure nodes Postition updates I.N. I.N. NetVigator: Landmark Vector Respository I.N. I.N. Hosts 11/6/2015 I.N. Position Updates Copyright © 2005 HP corporate presentation. All rights reserved. •Overloaded Central Server •Single Point of Failure •Query Latency 11 Distributed NetVigator • Closest Heuristic Partitioning − Simple idea: assign each node to its closest infrastructure node − infrastructure nodes know each other’s ‘positions’ − redirect position update/query to the closest infrastructure node − query-specific search expansion • Provides − load-balancing − efficient querying: low latency, low search expansion 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 12 Distributed Querying: Evaluation • Schemes Compared: − Closest mapping • Map each node to its closest infrastructure node − DHT based mapping • Map ‘position information’ to DHT geometry (we only looked at 1d initially) − Geometric Hilbert Mapping − Distance Vector Hilbert Mapping − Order Vector Recursive Partitioning • Evaluated on three datasets − PlanetLab: all-pairs-ping data − King Dataset: 1740 DNS servers − Transit-Stub synthetic topology 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 13 Results – King Dataset Closest Heuristic Partitioning • High hit ratio at Root •4-6 times lower latency to Root •Lower search expansion •Load-balanced 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 15 Results: Hops in search 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 18 Results: Load Balance 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 19 Concluding Remarks • NetVigator: computation of network proximity that: − − − − • is highly scalable as well as accurate is robust to bad measurements and choice of landmarks allows incremental computation has been tested using large scale simulations, and implementation on the HP Intranet and PlanetLab Next Steps: − Deploy as a PlanetLab service − Application to DARPA CHART (Control for High-throughput Adaptive Resilient Transport) project − Explore n-D DHT mappings − Scalable inference of other network/path properties such as bandwidth, loss etc. 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 20 More information… • Netvigator: Scalable Network Proximity Estimation, Zhichen Xu, Puneet Sharma, Sung-Ju Lee and Sujata Banerjee, HP Labs Technical Report, HPL-2004-28 • Distributed Querying of Internet Distance Information, Rodrigo Fonseca, Puneet Sharma, Sujata Banerjee, Sung-Ju Lee and Sujoy Basu, presented at IEEE Global Internet 2005 Symposium http://www.hpl.hp.com/research/mmsl/projects/net/ 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 21 11/6/2015 Copyright © 2005 HP corporate presentation. All rights reserved. 22