08-19-2011 11:53 AM
I'm setting up a primary PVLAN and two secondaries to give two groups of servers access to a shared iSCSI storage device. The problem is the servers can get any ARP responses from the storage because the broadcast traffic is by default disabled from a primary to secondary PVLANs on FCX.
Having looked in the FastIron Configuration Guide (I'm running L3 firmware 7.2.02) the commands to enable it are:
pvlan-preference broadcast flood
pvlan-preference unknown-unicast flood
The guide also informs pvlan-preference command is not supported on FCX platform without giving any alternatives.
Everything works fine when static entries are put in the servers ARP tables, but they need to be able to boot from this storage so that is not a viable solution.
Router interfaces are also unsupported on PVLANs so proxy-arp is not going to work.
Does anyone know how this can be solved on the FCX platform?
08-19-2011 03:57 PM
I do not think you can use PVLAN on FCX o do what you need.
Would standard L3 VLAN's with VE's do what you need?
vlan 10 name iSCSI
untag eth 0/0/1
router-inter VE 1
inter ve 1
ip address a.b.c.d/y
vlan 20 name group1
untag eth 0/0/2
router-inter VE 2
inter ve 2
ip address a.b.c.d/y
vlan 30 name group2
untag eth 0/0/3
router-inter VE 3
inter ve 3
ip address a.b.c.d/y
08-19-2011 05:07 PM
I can't use L3 VLANs and routing mostly because routing storage packets will very severely impact performance, and also I need the vlans separated for security reasons. It is the requirement which PVLANs were invented to fulfill.
PVLANs do what I need with the exception of allowing devices in secondary VLAN to get broadcast traffic from primary. This feature is critical for PVLAN usability and is supported on other devices by the pvlan-preferences command, but not on FCX. There has to be some other way the same functionality can be configured on FCX.
08-19-2011 05:29 PM
Ok, as I said I do not think PVLAN's will do what you need on the FCX for the reasons you stated.
How about l2 VLAN the group 1 and set for dual mode.
e.g. vlan iscsi set for tagged valn, vlan group 1 and group 2 set for untagged in there own VLAN but in dual port mode.
group 1 and 2 will not be able to talk to each other and both can access the storage.
Same type of setup when using VoIP phones.
Everything says at layer 2.
08-19-2011 06:06 PM
If I do that devices in both vlans will be ale to access each other through the iSCSI VLAN. I need to isolate the traffic for security reasons. Devices in your example group 1 and 2 will still be able to discover each other and send traffic to each other through the third (tagged)vlan. I also can't tag iscsi traffic on server side because my boot from san NICs don't support tagging during boot.
If I can't get the broadcasts from the storage to the devices I'll put an additional linux server with proxy-arp in each community vlan to answer those arp requests, but this is not what I would call a neat solution.
I can't understand why brocade would take out this functionality and leave severely limited PVLANs without providing some alternate method of accomplishing ARP resolution from secondary to primary. Short from reading the 1800 pages manual cover to cover I have no idea how this is supposed to be done.
08-23-2011 06:54 AM
It looks like community type PVLANs are completely unusable on FCX running firmware 7.2.
I have the following config:
vlan 50 name "Primary PVLAN test" by port
untagged ethe 1/1/44
pvlan type primary
pvlan mapping 51 ethe 1/1/44
vlan 51 by port
untagged ethe 1/1/43 ethe 3/1/43
pvlan type community
ARP requests are not forwarded not only from 1/1/44, but from anywhere else. A server connected to 1/1/43 is broadcasting and ARP request. It is not forwarded to 3/1/43 and it is not forwarded to 1/1/44.
Is this behavior by design? Is brocade expecting everyone to use static ARP with their poor PVLAN implementation?
08-23-2011 08:06 AM
I'll update this for the sake of anyone experiencing the same problem and finding this thread.
It seems this is not limited to broadcasts from primary to secondary VLAN. Broadcasts are not forwarded between member ports in a community type PVLANs as well. This is directly agains page 634 of the FastIron 7.2.02 configuration guide(4th paragraph from teh bottom) and most likely is a bug in FCXR07202d.
I'm opening a support call for this bug with our OEM. I'll update the thread if a solution is found.
08-26-2011 02:07 AM
Here is the promised update:
The issue is known to Brocade and is logged as 7202d firmware defect 364076 PVLAN (opened in August 2011). There is no fix date.
So the current status is Contrary to the datasheets and sales materials PVLANs are not officially supported on Brocade FCX :-(
That is very dissapointing news after buying those devices and wasting many hours of time troubleshooting the issue.
08-26-2011 02:46 AM
Thanks you for the update, this is good to know.
Did you ask the TAC in what version it was broken? E.G. if broken in dot d then maybe you could downgrade to get you working?
**** But check the release notes for 7.2.02d to see what was fixed against that version and see if you can live with that until the next patch.
Just a thought.
08-26-2011 02:51 AM
The defect was logged against 7.2.02d, but judging by the Wiki article it is not working on all 7.2 releases (I've updated the article with the details of the issue).
Unfortunately we can't use a different firmware as 7.0 has high latency issues on 10GB ports with our iSCSI storage and 7.1 has lots of stacking issues we can't risk appearing in our environment.
I'm trying to get our OEM to apply a bit of pressure on Brocade to get an estimated fix date for this. I'll update the thread when I have more information.