02-08-2013 05:13 AM
Here are a couple of interesting anomalies that we have run accross with the new Brocade VDX/FCX setup. This may save you troubleshooting time in your environment. What else have you all learned?
We ran into a very strange situation with Per VLAN Spanning Tree frames crossing between three vendors equipment. Our network (like most out there) is not a homogenious environment. We have equipment from many different vendors uplinked together. Brocade is now another vendor that we are supporting and we will continue have a very mixed environment until we can migrate all of our connections over to the new equipment.
Here is a simplified equipment chain.
* The Cisco 3500 Switch was running a version of PVST.
* Nortel was running its own Proprietary version of STP
* Brocade VDX Fabric was not running Spanning Tree (it is a Fabric)
* Brocade FCX was running PVST
In this scenario the PVST BPDU's were leaving the FCX and passing up towards the VDX. The VDX does not intercept these frames and just basically forward the BPDUs on as if they were any other layer2 frame. The PVST BPDU reached the Nortel switch which does not intercept these frames and just basically forwards them on (like the VDX did) on towards the Cisco 3500. The Cisco 3500 was not happy at all and it put ports into blocking mode when it really should not have.
So here is the multivendor anomaly - PVST BPDU's are normally sent only to your directly connected upstream neighbor and then that negibor processes the frames directly and does not continue sending up stream to veryone else, but puts its own BPDU together with the root info and sends out to directly connected neighbors.
Again we have a very similar layout with a Cisco router setup in the mix. Right now CDP on the FCX indicates that the Cisco Router is directly attached on the uplink port where it really connects to the VDX fabric. CDP is not being interpreted by both the VDX and the Nortel switches, but just forwarded along like any other frame. So from the CDP perspective the Cisco Router is directly attached to the FCX even though there are a number of other pieces of equipment setup in the mix.
Anyone have ideas for a work around. We were able to retire the Cisco Switch to eliminate that problem without much effort. The CDP anomaly is not really a problem but more of an nuisance.
02-08-2013 08:33 AM
First off, thank you for sharing your first hand experience across the multi-vendor topology danderson. I'm going to see if i can get some eyes to look this over. If you dont mind, could you share the firmware level you were at per each product?
02-13-2013 04:47 PM
just to say that this behaviour is expectable for me. Because a VCS fabric is not an ordinary switch and do not participate on any xSTP it will just forward all frames even this frames are layer-2 specific like xBPDU frames. I imagine the fabric more as a "wire" than a "switch" and take this into account for my network design.
Other setup but same effect: when I buy layer-2 WAN circuits from a carrier I'am glad to see 'my own' neighbours via CDP and can build my own xSTP and do not have to think a second about any system in the carriers cloud in between.
On the other hand it is good to have some config commands to tweak this if you really need to block those frames in special situations. But the defaults are correct in my opinion.
02-14-2013 05:27 AM
Thanks for the additional insight Gerald. I understand that this is functioning as designed, but it was something intially not expected. This is something that we had to work around in our setup and I suspect others may also have to do so. I'm curious how difficult it would be to add some features to be able to block the BPDU's/CDP at specific points. Here is something I ran into regarding BPDU's and Spanning Tree.
\ 1 /2
I have an FCX switch uplinked to the VDX fabric via a vLAG (FCX1 to VDX1-1 & FCX2 to VDX2-1). Initially I was told by Brocade support "Do not run spanning tree on the FCX uplinks facing the VDX switches". I set up the vLAG and everything was good. I then ran an extra cable from FCX port 3 to the VDX1 port 3. Now there was a loop in the network, I had the vLAG and the extra link (Port 3 >> Port 3) in the mix. Spanning Tree did not block the extra uplink as the FCX ports 1 & 2 did not participate in STP (even though port 3 did participate). From the STP perspective there was no loop.
To fix this problem, I had to enable STP on the FCX ports 1 & 2. Now when the extra cable gets plugged in I do not get a loop. During the problem period, the switches CPU went through the roof as traffic was continually looping (even with little traffic from our lab test running).
So now I do have STP uplinking towards the fabric, It would be nice to be able to block STP traffic flow at various places within the VDX Fabric. Maybe this will be a feature in a future code release.