Application Delivery (ADX)

Creating Linux Logical Network Interfaces for DSR Load Balancing

by jack on ‎11-19-2009 02:06 PM (2,896 Views)

Problem

 

To use DSR you need to configure a logical interface on the real server. The instructions contained in the documentation for the ServerIron's may not work using Linux, although they are fine for Solaris.

 

With some Linux O/S's using the 2.6 kernel, the logical interface is considered "live" no matter which interface it is associated with. The host will reply to ARP requests for this logical IP. This effectively bypasses the load balancer as the real server will respond to ARP requests for this IP independent of the health of its service. If your machine is responding to ARP requests for the VIP IP address, you need to take a few additional steps in order to turn these ARP requests off.

 

A good indication of this problem is if you see the following in your log files on your load balancer.

 
Nov 6 11:20:54:WSmiley Very Happyuplicate IP address 10.10.10.10 detected sent from MAC address 0011.b6e1.e688 ...  
Nov 6 11:17:49:WSmiley Very Happyuplicate IP address 10.10.10.10 detected sent from MAC address 0011.b6e2.45ce ...  

A way to test for this problem is to try to bring up a logical interface with an unused IP using your normal steps and then attempt to ping it from another host on the same network. If you get a response to the ping, then you are likely to run into problems using DSR.

 

The following instructions are RedHat specific but the principles should apply to any Linux system with the 2.6 kernel. There may be other approaches but this was the least painful one that I could find. My recommendation is that you test these steps with an unused IP address. Once you are done, you should be able to ping this address from the host it is installed on but no other machines on the same network should be able to ping it.

 

Solution

 

Step #1 - Create the interface

 

First, put the following in a /etc/sysconfig/network-scripts/ifcfg-dummy0 file

 

DEVICE=dummy0  
ONBOOT=yes  
BOOTPROTO=static  
IPADDR=192.168.10.222  
NETMASK=255.255.255.255   

and change the IP address to one that you are interested in testing.

 

Step #2 - Modify kernel settings

 

Put the following into your /etc/sysctl.conf

 

net.ipv4.conf.default.arp_announce = 2  
net.ipv4.conf.default.arp_ignore = 1  

Now run 'sysctl -p /etc/sysctl.conf'. This will load the setting into your kernel and by putting them in this file, they should persist over reboots. These settings can be restricted to specific interfaces but it is not usually necessary to do so. The names might be slightly different depending on your O/S. This can be checked by running 'sysctl -a | grep arp' to get a list of the setting for your kernel & O/S.

Step # 3 - Restart the network service

 

This can be done by '/sbin/service network restart' or by rebooting the machine.

 

Done

 

At this point you should no longer see duplicate MAC warnings in your log files or have logical interfaces that respond to ping requests.

 

Multiple Logical Addresses

 

There may be circumstances where you want your real server to bind to multiple VIPs with DSR. Since there is only one dummy0 driver, you can use aliases to create the additional IP's. An alias is created by using by using additional files with the names of ifcfg-dummy0:0, ifcfg-dummy0:1, ... and then specifying a device of DUMMY0:0, DUMMY0:1, ... in the respective file. The ifup-aliases script will go through these files and create the appropriate interfaces. Here

  

DEVICE=dummy0:0  
ONBOOT=yes  
BOOTPROTO=static  
IPADDR=192.168.1.226  
NETMASK=255.255.255.255  

 

Summary

 

There is a debatable logic in being able to create a logical interface that responds to ARP requests. Personally, I believe that such interfaces should be bound to a physical device but perhaps there are situations where you need an IP address to respond over multiple NICs. The important lesson for me is that you can incrementally test a DSR setup to make sure the O/S component is working before getting the Load Balancer portion running.

 

Postscript - Multiple Physical Addresses on the same interface

 

There are situations where you may need additional more than one physical address on the same interface within the same subnet. An example might be that you have a machine that is both an SMTP server and DNS server but you need to want the servers in separate contexts. Since real servers can not be shared over contexts, one approach is to create another physical IP. Again, take advantage of the ifup-aliases script by putting additional files in /etc/sysconfig/network-scripts in the format of if-interface:0, if-interface:1. There are some important differences. The netmask should match the network, the interface the physical interface, do not include a GATEWAY, and set ONPARENT=yes. One example is

 

DEVICE=eth0:0  
BOOTPROTO=none  
BROADCAST=10.10.229.255  
IPADDR=10.10.229.148  
NETMASK=255.255.255.0  
TYPE=Ethernet  
ONPARENT=yes   

There is some decent additional information in the ifup-* scripts on the various settings.