Application Delivery (ADX)

Policy-Based Server Load Balancing (PBSLB)

by on ‎07-01-2009 09:00 PM - edited on ‎10-31-2013 03:32 PM by bcm1 (2,354 Views)

Policy-Based Server Load Balancing with ServerIron

 

Feature Brief Intro

 

The ServerIrons PBSLB feature provides the possibility to distribute requests to different server groups based on the source IP address of the requests.

PBSLB is somehow similar to Policy Based Routing (PBR) at traditional L3 devices.

PBSLB is available at the virtual service level. Traffic to a PBSLB enabled service of a virtual server is getting forwarded based on the source IP address fo the request and a list of IP addresses/subnets mapped to server groups.

The list linking IP addresses to server groups might be part of the configuration (very limited size) or it is downloadable via TFTP.

 

What are the reasons to use PBSLB?

 

Customer deploying PBSLB might want to

 

1. have special pools of servers for various client subnets (internal clients will end up at faster servers)

 

2. treat traffic from well-known IP addresses different from other traffic (like traffic from well-known spammers)

 

3. ...

 

Scope of this document

 

We are going to configure PBSLB with a  policy list coming from a TFTP server. There are three servers and three server groups - one server per group:

group 1 -> for internal clients (server: 192.168.8.101)
group 2 -> for well-known spammers (server: 192.168.8.102)
group 3 -> the rest (server: 192.168.8.103)

The IP/subnet to group assignment is going to be the following:


192.168.9.0/24               -> group 1
192.168.178.0/24          -> group 1
192.168.200.0/24          -> group 1
172.16.0.0/16                 -> group 1
172.31.31.31                  -> group 2
192.168.222.222           -> group 2
10.10.10.10                    -> group 2
10.11.12.12                    -> drop

PBSLB should hit for traffic coming in to virtual server 192.168.9.222 / port 80.

 

Topology

 pbslb.JPG

 

 

Sample Code/Configuration and Explanations

 

The ServerIron is going to look at the configured policy list for all incoming client requests. The policy list associates the source IP address of the request with a pool of real servers. If an entry for the IP address is found in the policy list, then the ServerIron ADX forwards the request to the associated real server group. If no entry for the IP address is found, the ServerIron ADX directs the request to a server group specified as the "default" server group.

There are two ways to get a policy list at the ServerIron ADX:

  1. create one using the CLI (feasible for small lists only)
  2. use TFTP or a USB flash to download a policy list to the ADX


We are going to use TFTP here.

A policy list is a simple ASCII file which consists out of one or more PBSLB entries with the following format (one entry per line in the file):

<ip-addr> [<network-mask>] <server-group-id>

The <ip-addr> can be a complete host address, or a network address followed by IP mask bits.

The <server-group-id> variable is alphanumeric and refers to one of the real server groups
configured on the ServerIron ADX.

The IP/subnet to group assignment shown above would look like:

192.168.9.0/24 1
192.168.178.0/24 1
192.168.200.0/24 1
172.16.0.0/16 1
172.31.31.31 2
192.168.222.222 2
10.10.10.10 2
10.11.12.12 0

I do assume there is a file with the PBSLB policy at the TFTP server already. The TFTP server IP address is 192.168.8.104 and the filename is list.txt. There is an example file attacked to this document. Use the following command to do a download of the policy list from the tftp server:

ServerIron(config)# server pbslb tftp 192.168.8.104 list.txt 5

Syntax: server pbslb tftp <tftp-server-ip-addr> <filename> <retry-count>

The <tftp-server-ip-addr> variable specifies the IP address of the TFTP server.
The <filename> variable specifies the name of the policy list file.
The <retry-count> variable specifies the number of times that the ServerIron ADX retries the download if the first attempt is not successful.

 

CLI output during the download:

 

telnet@ServerIron 4G(config)#server pbslb tftp 192.168.8.104 list.txt 5
Download of pbslb config from TFTP server is initiated.
telnet@ServerIron 4G(config)#
15:17:11 GMT+00 Wed Jul 01 2009
Download of pbslb config from TFTP server is done.
TFTP file size = 140, Entry count = 8,  Parse error = 0, Table full error 0

 

The list size is limited to 25000 entries by default and you have to use the following command to increas it:

 

ServerIron(config)# server pbslb max-entries<number>

 

The <number> variable specifies the amount of entries you would like to allows.

 

Example:

 

ServerIron(config)# server pbslb max-entries 50000

 

to allow a list of maximum 50K entries.

 

The policy download is not service affecting as long as the policy list does not exceed 1,000,000 PBSLB entries. The ServerIron maintains two tables to ensure there is no problem during the download - that means the ServerIron is able to download the new list first of all and it is going to swap the active list with the new list afterwards.

The ServerIron is going to block traffic during the policy download in case the list is bigger than 1,000,000 entries. Is it possible to send traffic to a "default" server group instead of blocking it. The following 3 steps are necessary to do so:

  1. set default group-id
  2. add real server ports to default group
  3. enable send-to-default-group-during-download


ServerIron(config)# server pbslb default-group-id 4
ServerIron(config)# server real rsdef1 a.b.c.1
ServerIron(config-rs-rsdef1)# port http group-id 4
ServerIron(config-rs-rsdef1)# server real rsdef2 a.b.c.2
ServerIron(config-rs-rsdef2)# port http group-id 4
ServerIron(config-rs-rsdef2)# exit
ServerIron(config)# server pbslb send-to-default-group-during-download

The maximum list size talking about the ServerIron ADX is 10 million entries. The last ServerIron generation (ServerIron 350/450/850 and ServerIron GT-C/GT-E) was able to store up to 5 million entries in releases prior to release 10.0.00a and up to 7 million entries starting with release 10.0.00a.

The PBSLB policy list is not getting saved to the ServerIrons flash by default when you enter a write memory. To write the policy list to the flash memory, enter the following command - we are not going to use this here:

ServerIron(config) server pbslb enable-config-gen

NOTE: The ServerIron ADX is NOT able to copy a policy list with more than 1,000 entries to the flash!

The PBSLB policy list is responsible for the association of client/source IPs with real server group IDs. It is necessary to assign real servers to the real server groups. We have been talking about the following real server to group assignment above:

group 1 -> for internal clients (server: 192.168.8.101)
group 2 -> for well-known spammers (server: 192.168.8.102)
group 3 -> the rest (server: 192.168.8.103)


Enter the following commands to create real servers and to put the http port of the real server into the groups mentioned above:

ServerIron(config)# server real rs101 192.168.8.101
ServerIron(config-rs-rs101)# port http group-id 1 1
ServerIron(config-rs-rs101)# exit
ServerIron(config)# server real rs102 192.168.8.102
ServerIron(config-rs-rs102)# port http group-id 2 2
ServerIron(config-rs-rs102)# exit
ServerIron(config)# server real rs103 192.168.8.103
ServerIron(config-rs-rs103)# port http group-id 3 3
ServerIron(config-rs-rs103)# exit

We want to use real server rs103 (part of real server group 3) as default real server / default group - this is getting done via:

ServerIron(config)# server pbslb default-group-id 3
ServerIron(config)# server pbslb send-to-default-group-during-download

according to the instruction above talking about a default-group.

The virtual server 192.168.9.222 needs to get configured now with port 80 (http) and it is necessary to enable PBSLB for port 80 - ALL of the real servers above need to get bound to port 80 of the virtual server:

ServerIron(config)# server virtual vs222 192.168.9.222
ServerIron(config-vs-vs222)# port http
ServerIron(config-vs-vs222)# port http sw-l4-pbslb
ServerIron(config-vs-vs222)# bind http rs101 http rs102 http rs103 http

PBSLB is now configured. Incoming traffic to virtual server 192.168.9.222 for port 80 is getting forwarded to one of the real servers bound and the forwarding decision is based on the PBSLB policy list.

 

Tips / Caveats

 

It is recommened to choose a download interval of at least 10 minutes in case of large PBSLB lists (a few million entries). It is wise to choose an even higher interval talking about a fully loaded ADX chassis due to the amount of BPs in such a system.

 

PBSLB is a Layer 4 feature and it is NOT possible to use PBSLB at a virtual server which is doing layer 7 switching as well (like URL switching and cookie switching).

 

PBSLB is as well NOT useable together with IP SLB (Layer 3 load balancing).

 

PBSLB is a feature which is getting enabled on a per-VIP basis - it is therefore possible to have PBSLB and non-PBSLB VIPs at the same ServerIron.

 

Config


server pbslb send-to-default-group-during-download
server pbslb tftp 192.168.8.104 list2.txt 5
server pbslb default-group-id 3
!
server real rs101 192.168.8.101
port http
port http url "HEAD /"
port http group-id  1 1
!
server real rs102 192.168.8.102
port http
port http url "HEAD /"
port http group-id  2 2
!
server real rs103 192.168.8.103
port http
port http url "HEAD /"
port http group-id  3 3
!
!
server virtual vs222 192.168.9.222
port http
port http sw-l4-pbslb
bind http rs101 http rs102 http rs103 http

 

 

 

Troubleshooting

 

I would recommend to use various test clients. The ServerIron statistics available via "show server real" and capture utility "debug filter" to analyse what is happening in a PBSLB setup. Use "show pbslb all <index>" to get the actual pbslb policy list - this is pretty handy as long as the list is not very large.

 

telnet@ServerIron 4G(config)#show pbslb all 0

 

Max Count: 25000        Total Count: 8

 

IP address           Mask                 Server Group ID
10.10.10.10          255.255.255.255      2
10.11.12.12          255.255.255.255      0
172.16.0.0           255.255.0.0          1
172.31.31.31         255.255.255.255      2
192.168.9.0          255.255.255.0        1
192.168.178.0        255.255.255.0        1
192.168.200.0        255.255.255.0        1
192.168.222.222      255.255.255.255      2

 

Syntax: show pbslb all <index>


The show pbslb all command displays 20 entries in the policy list, starting from the point specified with the <index> parameter.

Contributors