02-26-2012 02:40 AM
Let me describe issues with multicasting that we observe on FCX648 (7202e firmware).
The term “multicast” usually describes two different situations, i.e. multicast IP addresses and multicast MAC addresses.
Let us install Microsoft Forefront TMG 2010 in the NLB-cluster mode (it’s a pretty usual thing actually). What does Forefront TMG do? It acts as a gateway (or a border firewall), so our devices use it as their default gateway address.
The clustered Forefront TMG uses a unicast IP address for multiple servers in the NLB cluster. How is that possible? The trick is to map a multicast MAC address to the unicast IP address of the cluster. The NLB cluster members negotiate some shared rules, like “I’m number 1, you are number 2”, each cluster member handles its own part of traffic. Again, all cluster members receive the same traffic (it is possible because of the multicast MAC address), but only one cluster member replies at a time.
We met two scenarios.
1) The Forefront TMG cluster and other servers are in the same VLAN, so our FCX performs Ethernet switching between them. It delivers frames for the multicast MAC address to all Ethernet ports, which is normal for multicast MAC addresses. If we use “static-mac-address” command for the MAC address of the cluster on some ports of the FCX, where the cluster members reside, then the switch delivers the mentioned traffic to the configured ports only. Everything works as expected, packets are delivered to all Forefront cluster members, so every member routes its packets.
2) Other servers are in a different VLAN than the Forefront TMG cluster and the FCX acts as a router between the VLANs. We encounter issues in this scenario. It seems that the FCX neglects the multicast nature of the cluster MAC address in this case and delivers packets according to the ARP entries of the FCX. The problem is that the FCX has only one entry for the cluster MAC address and the entry points to one port only, i.e. only one cluster member sees the packets destined for the cluster. In order to work as expected all cluster members must see these packets.
We’d like to contact some specialist who understands how multicast things are implemented in FCXs.
02-26-2012 07:14 PM
I haven't messed with 2008 NLB's much, but with 2003 you should implement a local network for communication between the two systems, typically a crossover cable when in Unicast mode. This allows the servers to communicate to each other without involving the virtual mac addresses.
Microsoft NLB is evil, I typically just place the servers in a dedicated vlan to that NLB, tell them to use unicast mode and leave me alone.
02-26-2012 11:46 PM
if you were reading carefully, you would have been noticed that NLB sometimes works as expected (when Brocade FCX acts as a L2 device) and sometimes it doesn't work as expected (when Brocade FCX works as a L3 device). I know why it messes with NLB and I described the reason above.
02-26-2012 11:53 PM
Microsoft NLB is always a difficult thing to run clean on a switch network.
static MAC entries as long as the switch is in L2 mode only.
This works but is not flexible on server movemnts (= e.g. VMware) or for uplink redundancy in a meshed network
separate NLB VLAN with routing on the FastIron
But for this Brocade misses the feature "Multi-Port ARP" entries.
I have opened a Request for Enhancement on this about >1,5 years ago, and the information from Brocade was that it should be implemented into v7.3 code, but I have not found it yet.
I will ened to check with my local Brocade SE.
Hope this helps,
02-27-2012 01:12 AM
I described reasons why Brocade FCX behaves in a wrong way when acting as L3 device with NLB clusters. I think we should wait for someone from Brocade to answer. I do not understand why it should be ok with L2 mode and completely impossible with L3 mode. It is just buggy firmware, nothing else. Let Brocade engineers say what they think about that situation.
02-27-2012 01:24 AM
>>Let Brocade engineers say what they think about that situation.
Please keep us informed if you get an answers from Brocade on this!!!
Let's say... I am wating >2 years on this ;-)
02-27-2012 01:52 AM
The feature "Multi-Port ARP" for the FastIron, for which I have initiated a Brocade Request for Enhancement in 2010 and which I was told to come with FastIron v7.3, is still not there and still no confirmed release date.
I am looking forward to get the answer Broacde will give you!!!
02-27-2012 02:07 AM
multiple ARP entries are unneccessary. there's already working solution, when Brocade acts as L2 device it determines whether or not mac address is multicast ir unicast.
the same logic should be implemented in case of L3 routing. It determines mac address first, right ? The next step (which is obviously missing) should be "let us have a look at mac address, is it unicast or multicast?", ok, it is unicast, let us have a look at ARP table. no, not unicast ? well, let us deliver it according to multicast rules.
P.S. I'll inform you about Brocade engineers answer, what is your email ?
02-29-2012 05:21 PM
Layer 3 knows that IP as a route, it doesn't care what it's MAC address is, it only needs to know how to get to the next hop router that knows how to get to the destination.
If the router doesn't know the MAC, how do you want it to determine if it's a Multicast MAC or not?
Eventually the end router sees a unicast address responded to with a multicast MAC, therefore the packets will be discarded.
Additionally lets not ignore the fact that Microsoft is using a unicast IP address to talk to a multicast mac for clients.
Until Brocade implements the Multi-Port Arp as mentioned earlier you will only have those solutions. It's not buggy Brocade firmware, it's poor network considerations in MS NLB.
02-29-2012 10:06 PM
the abovementioned "multi port ARP" is TOO COMPLEX. I'd suggest something different. I understand that words do not give a picture and you wonder "how is mac related to routing ? you just need next-hop in case of routing..." have a look at pictures.
how switching works:
how routing works now:
how things should be (what I suggest instead of multi port ARP):