Ethernet Switches & Routers

Reply
Contributor
Posts: 44
Registered: ‎07-08-2011

Link Aggregate Priority HELP please

We have two separate stacks connected via fiber connections.

port 1/1/3 is connected to 1/1/3 in the other stack and port 2/1/3 is connected to the other stack's 2/1/3. below is the configuration.

right now 1/1/3 connected is saturated and 2/1/3 is at 40%. We realize for safety we need another port connection in the trunk in case one dies. However, our immediate goal is to get that 140% total utilization (out of 200% total) down so that it's 70% on one link and 70% on the other. We are pretty much only sending data one direction (from stack 1 to stack 2). These are FCX648S devices.

How do we make it so that it does not saturate one of the links while the other has plenty of bandwidth available? port or system priority? I'm new to all this. ports 1/1/4 and 2/1/4 are configured to belong as well ahead of time so we can simply connect an extra link and it will automatically be a part of the trunk. At least that is our hope. But since this connection is across cities, we have to get budget approval to add a third. 

ver 07.1.00aT7f3
!
stack unit 1
  module 1 fcx-48-port-management-module
  module 2 fcx-cx4-2-port-16g-module
  priority 128
  stack-port 1/2/1 1/2/2
stack unit 2
  module 1 fcx-48-port-management-module
  module 2 fcx-cx4-2-port-16g-module
  priority 128
  stack-port 2/2/1 2/2/2
stack enable
stack mac asdf.asdf.asdf.asdf

....

interface ethernet 1/1/3
port-name RepPorts
dual-mode
spanning-tree 802-1w admin-pt2pt-mac
link-aggregate configure key 20000
link-aggregate active
!
interface ethernet 1/1/4
port-name RepLink4
dual-mode
spanning-tree 802-1w admin-pt2pt-mac
link-aggregate configure key 20000
link-aggregate active
!

interface ethernet 2/1/3
link-aggregate configure key 20000
link-aggregate active
dual-mode
!
interface ethernet 2/1/4
dual-mode
spanning-tree 802-1w admin-pt2pt-mac
link-aggregate configure key 20000
link-aggregate active
!

Brocadian
Posts: 70
Registered: ‎03-14-2009

Re: Link Aggregate Priority HELP please

Hi Blake,

the traffic is distributed via the trunk according a hashing calculation based on the flow parameters (e.g. Source IP, Destionation IP, Source/Destination TCP/UDP port, L2, L3). Depending on the flows you have it may be that the distribution is uneven. Check the configuraiton guide chapter 14 for Trunk Groups and Dynmic Link Aggregation.

the only way you can change this is to add the MACs to the hashing via global config command:

trunk hash include-layer2

this is the same for the latest software release 7.4.

HTH

Alex


Contributor
Posts: 44
Registered: ‎07-08-2011

Re: Link Aggregate Priority HELP please

Looking for that doc. Do I put in the MAC address of the trunk ports of every host or destination NIC that passes traffic across this link? That's a LOT of mac addresses.

Contributor
Posts: 44
Registered: ‎07-08-2011

Re: Link Aggregate Priority HELP please

I'm not seeing that command in any doc. do you have a link? i did notice this paragraph below. Perhaps I should change to SERVER mode?

Trunk group load sharing

When you configure a trunk group, you specify whether the trunk group is a “switch” trunk group or a “server” trunk group:

Switch trunk group – Use this type of trunk group to load balance traffic based on destination MAC addresses.

Server trunk group – Use this type of trunk group to load balance traffic based on source IP and destination IP addresses.


Traffic distribution among trunk groups

Traffic distribution among trunk groups is described in Table 6. Traffic distribution among trunk groups is the same for switch and router code. LACP trunks default to trunk switch behavior.

Trunk Type

Traffic Type

Traffic distributed based on

Trunk Switch

Pass-through

Destination MAC, VLAN, EtherType, Source Module and Port ID

Trunk Switch

SLB

Destination MAC

Trunk Server

Pass-through.

Source IP, Destination IP, Source Port and Destination Port

Trunk Server

SLB

Source IP and Destination IP

Contributor
Posts: 44
Registered: ‎07-08-2011

Re: Link Aggregate Priority HELP please

Also, just noticed that priority feature is for a trunk setup. We are using Link-Aggregate. We can't use trunks because it's across cities through our ISP's devices. Won't work. Sounds like we're out of luck here unless they can enable their own load balancing.     

Contributor
Posts: 44
Registered: ‎07-08-2011

Re: Link Aggregate Priority HELP please

Anyone else have insight?

Occasional Contributor
Posts: 7
Registered: ‎12-22-2009

Re: Link Aggregate Priority HELP please

Hi Blake,

The load distribution in link aggregation depends on your traffic. The standard does not say how to distribute the traffic. But the order of the packets within a communication should not change. The switches are using some kind of hashing to assign a communication to a link. You can now influence this hashing algorithm (src ip, dst ip, sport, dport ...) . But with one TCP connection it will use only one link of the aggregation (to ensure the packet order). To get a 50 % / 50 % load balancing depends on your traffic and would be coincidence. 

VDX Brocade Trunks are different in this point. The will distribute the traffic more evenly.

Regards,

Benjamin

Contributor
Posts: 44
Registered: ‎07-08-2011

Re: Link Aggregate Priority HELP please

Thanks Benjamin.

Our scenario is that we have a ton of Dell Equallogic iSCSI SANs in production (24) replicating over this connection to two groups. One of which has 3 members and the other with 7. Each array has two controller cards, each with 4 nic ports. the IPs are virtual but if a controller card fails then the IPs just start using the other ports (now with new mac addresses). So what would be good is to say either:

A. Any traffic to these X IP addresses go over this link and the others go over this other one.

B. Same idea, but listing all the mac addresses. Do you know if that is possible with stacked FCX switches on each side? No idea how to do this.

Super Contributor
Posts: 1,087
Registered: ‎12-13-2009

Re: Link Aggregate Priority HELP please

Hi Blake,

     As I understand, who have two stacks connect together via two ports set in LACP and you want to be able to set conditions of what can talk over what individual port in the LACP group?

     If I have understood the question correctly then there is no way of doing this (at a switch level - a link load balancer would do this) as LACP is just a hashing protocol -  you only get good load balancing results with lots of flows of equal sizes.

Thanks

Michael.

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.