http://netlabs.accenture.com

Download Report

Transcript http://netlabs.accenture.com

Remote Network Labs: An On-Demand Network Cloud for Configuration Testing

Huan Liu, Dan Orban Accenture Technology Labs

Configuration is hard

• A few reasons – Many SW images for each box • IOS: 5+ trains, 8 packages for routers, 5 packages for switches – Primitive CLI interface – Commands may behave differently  a design may work on paper, but not in practice – Local configuration at a switch/router could have global impact • Sample manifestations – More outages are caused by operator errors rather than equipment failures 1,2 – 3 in 4 new BGP prefix advertisements are results of misconfiguration (200-1200prefixes/day) 3 – Average 7.17 to 9.63 errors per firewall config 4 1.

2.

3.

4.

“Configuration management delivers business resiliency,” The Yankee Group, Nov. 2002 D. Oppenheimer, A. Ganapathi , and D. Patterson, “Why internet services fail and what can be done about these,” in Proc. USENIX USITS, Oct. 2003.

R. Mahajan, D. Wetherall and T. Anderson, “Understanding BGP Misconfiguration”, SIGCOMM 2002 A. Wool, “A Quantitative Study of Firewall Configuration Errors”, Computer, 2004 2

Current solutions and their problems

• Solution 1: Simulator (e.g., RouterSim, OPNET) – Cannot capture all aspects of real hardware (e.g., many IOS images) – Support limited # of commands – Have simulation models for a limited number of routers • Solution 2: Emulator (Dynamips) – Limited set of interfaces are simulated – Support a limited set of Cisco routers • Solution 3: Build a smaller scale network in Lab to mimic the production network, and test in lab before roll out – Costly investment (routers are not cheap) – Poorly utilized (only during testing), yet necessary in case configuration changes again – May not be available when you need it (spare parts problem) – Time consuming to wire up – Hard to make sure the test network is the same as designed on paper. 3

What your lab looks like?

Our solution: Remote Network Labs

• Setup a virtual lab, then test – Equipment could be anywhere (no longer needs to be co-located) – Design from anywhere (no longer needs to be physically on site) – Quickly and easily reconfigurable • Advantages – Enable efficient sharing of equipment between projects • No more procurement delay • Essential zero cost to each project • No need for physical lab space – Enable design from a Web browser • Everything digitized, including the wiring – Save/restore legacy environment – Fully automate lab set up • No physical work to mount equipment and wire • Reduce travel (go

Green

) 5

Distributed architecture

http://netlabs.accenture.com

Web browsers San Jose

Internet

Chicago Client site 6

How it works

When press Deploy button, send instruction to build tunnel Front end Interface SW Interface SW

Internet

Back end 7

Making a router/firewall/server part of the lab inventory Internet

Interface SW 8

User defines mapping

Copyright © 2007 Accenture All Rights Reserved.

9

Key differences from prior work

Real router Distributed Wire

Emulab PlanetLab ONL VINI WAIL RNL N N N N Y Y N Y N Y N Y L2 tunnel L3 L2 tunnel L3 tunnel Real wire L2 + L3 tunnel Copyright © 2007 Accenture All Rights Reserved.

10

Use case 1 (layer 2 config)

• Configuring failover using Cisco Catalyst 6500 switches with a Firewall Services Module (FWSM) • Can also capture transient behavior Copyright © 2007 Accenture All Rights Reserved.

11

Use case 2 (test automation)

• Full automation (from setup to teardown) – Implementing web services API – Implementing library for traffic generation/capture, CLI parsing.

• Add delay/jitter • Inject/observe anywhere x x Add delay/jitter Inject x Observe Copyright © 2007 Accenture All Rights Reserved.

x 12

Challenges/drawbacks

• Additional delay • WAN bandwidth is not free • L2 interface diversity • Performance testing (see paper for more details) Copyright © 2007 Accenture All Rights Reserved.

13