Scalable Server Load Balancing Inside Data Centers

Dana Butnariu Princeton University Computer Science Department July – September 2010 Joint work with Richard Wang and Professor Jennifer Rexford

Client3 Client4 Client2

Getting started

Client1 Client5

The Internet

Server1 Server2

Switch

Server4 Server3 Client6

Getting started

• • • • • •

Client

– any device requesting a web service

Server

– device which handles a client request and provides the web service

Data center

– location containing a group of servers

Server load

– number of client requests a server must handle

Load balancing

– directing a client request to a particular server, managing server loads according to a certain algorithm

Switch

– device which enables client and server communication

Energy and Load balancing

• Servers:  Are located in data centers in different areas of the world.

 Energy cost and availability varies from one location to the other.

 Energy cost and consumption depends on client – server distance.

• Load balancing:  Tries to lower the energy cost and usage without affecting user perceived performance.

 Can achieve this goal by selecting close by data centers.

 Can achieve this goal by using only certain servers and powering down the rest.

Load balancing today

• •

Old approach

– have a separate device, the load balancer.

New approach

– implement the load balancing in devices already existent in the network. •

Old approach

:  Costly device  Consumes energy  Hard to program  Crashes easily •

New approach

:  Already existing device  No additional costs  Easy to program and customize  Stable and reliable

What, why and how?

•

What

: Scalable Server Load Balancing without sacrificing user perceived performance. •

Why

: To save energy and lower the cost of energy used to process client requests.

•

How

: Using a new emerging technology called OpenFlow which enables switch programming.

Project Steps

•

Establish

the network design.

•

Design

the load balancing application.

•

Implement

the load balancing application.

•

Test

the load balancing application.

Network Design

• Establish the network design.

 How many clients, servers, switches?

 How are they connected?

 What knowledge do they have of one another?

• There is a “Brain”.

 It is just another computer.

 It controls switch behavior.

 It installs rules in the switch.

 Rules tell switch which server handles a client request.

Clients

Network Design

Brain implementing design algorithm Client request Send request to servers Switch Install Load Balancing rules Server 1 Server 2 Server 3 Server 4 Data Center

How it works

• Client sends request for web service.

• Request arrives at switch.

• Switch decides server to handle request.

• Decision is based on:  Closest server so that less energy is used for transport  Cheapest path in terms of energy cost  Server usage so that less used servers can be powered down • Server sends a reply back to the client. • Client is provided with the web service.

Load Balancing Components

Partitioning Class Old server still handles old client requests Transitioning Class Install rules New servers handle new client requests Load Balancing Switch Decide servers to be powered down Statistics Class Monitor server usage

Application Components

• Partitioning:  Responsible for implementing the Load Balancing algorithm  Decides which server handles which client request • Transitioning:  Ensures that when a server is powered down all new client requests are handled by another server  Ensures that all old client requests are answered by the old server • Statistics:  Provides statistics regarding server usage

Prototype Implementation and Future Steps

• The first prototype for this project was created using:  OpenFlow programmable switches.

 OpenFlow, NOX and Mininet to program the switches.

 Applications written in Python.

 Applications tested using VMWare (Virtual Machine) and Debian version of Linux running on VMWare.

• Future steps:  Running the application on a web server and on a real network.

  Designing a more accurate partitioning component.

Adapting the partitioning component according to the statistics component.

Conclusions

• New solution which saves energy by:  Being implemented in already existing hardware, the switches.

 Finding the path which uses the lowest amount of energy at the lowest cost.

 Turning down severs which are handling a small amount of client requests.

• Solution offers:  flexibility due to the software component – the Load Balancing algorithm can be easily modified.

 speed due to the underlining hardware component – switch which applies rules.

Acknowledgements

Thank you:  Professor Jennifer Rexford   Richard Wang – team member Rob Harrison and David Shue – graduate students  Nate Foster – postdoctoral research fellow

Questions?