Ethernet Fabric (VDX, CNA)

Reply
Occasional Contributor
Posts: 6
Registered: ‎08-24-2011

How does VCS Ethernet Fabric interact with Spanning-Tree?

Greetings,

One of our clients is planning to expand his network infrastructure to support a new hosted virtual server service offering.  I am looking at the Brocade 6720 switches with VCS as a possible choice.  I am curious to get some details about how a pair of 6720 switches running VCS (ethernet fabric) will interact when connected to a Cisco 6504 core running STP.

I've attached a quick sketch to help illustrate my questions.  Assume all the links are 802.1q trunks.  In the top drawing the two 6720 switches (labeled V1 and V2) are not running VCS so spanning tree blocks at V2 port 24 to prevent a loop.  This means traffic between SAN server A and Host server B has to traverse Core 1. 

What I'd like to know is how this scenario changes if you enable VCS on V1 and V2.

Here is my question: With VCS running on V1 and V2 , how is an ARP brodcast originating from SAN A for Host C located at Site 2 prevented from flooding out V2 port 24?  I can imagine VCS enabled ports blocking all ethernet type 0x806 (Standard ARP) packets and allowing only some type of custom VCS ARP packtes.  How does this work?

Thanks,

Greg

Super Contributor
Posts: 1,087
Registered: ‎12-13-2009

Re: How does VCS Ethernet Fabric interact with Spanning-Tree?

OkHi Greg,

     First when you have two or more VDX's setup in VCS mode (Ethernet Fabric) STP is running on the fabric.  Howeverif STP is running in the core and you have another noramal switch plugged intop one of the VDX's then the STP packets will carry over to that other swich.  as far as sTP is conserned the fabric is just a cable.

     Ok if I understand the question  correctly.  With VCS enable (bottom half of your picture, you would have Core 1 connect to both VDX' switcxhes via LACP.  The host to want SAN A to talk to is Host C - in another site and connected by Core 1?  If that is correct then the normal process if followed.   E.g. the VDX will follow the path back to core 1 and core 1 will route to site 2.  The path from SAN A back to Core1 will depend on of the result of the FSPF routing protocol.

     VDX also share the MAC table with each other - e.g. when a new host is connect to a VDX it sends an update says <new MAC add> can be found by my rbridge ID to all other VDX.  However as Host C is in a different site there is no learning of that in this case.

     Not sure if I understood your question correctly, please let me know.

Thanks

Michael.

Occasional Contributor
Posts: 6
Registered: ‎08-24-2011

Re: How does VCS Ethernet Fabric interact with Spanning-Tree?

Hi Michael,

Thanks for replying and trying to understand my question.

Just ignore Host C for now.  I put it there to show a Layer 2 connection exists between two datacenters.

I think I understand correctly that the VDX switches (in my scenario: V1 and V2) running in VCS mode will "carry over" STP BPDU packets to another non-VDX non-VCS switch connected to the fabric if there was one.  In this case there is not.

What I don't understand is what happens on the 802.1q trunk link between V1 port 23 and V2 port 24.  I thought the advantage of the ethernet fabric was that traffic from SAN server A connected to V1 could travel directly to Host server B connected to V2 across the V1-23 --> V2-24 link rather than having to take the path through Core 1 (as it would have to do if STP was running and blocked on V2 port 24).

So, using the lower scenario, with V1 and V2 running in VCS mode, will port V2-24 get blocked by STP?

Thanks

Super Contributor
Posts: 1,087
Registered: ‎12-13-2009

Re: How does VCS Ethernet Fabric interact with Spanning-Tree?

Hi Greg,

    

What I don't understand is what happens on the 802.1q trunk link between V1 port 23 and V2 port 24. I thought the advantage of the ethernet fabric was that traffic from SAN server A connected to V1 could travel directly to Host server B connected to V2 across the V1-23 --> V2-24 link rather than having to take the path through Core 1 (as it would have to do if STP was running and blocked on V2 port 24).

     You are correct; V1 port 23 and V2 port 24 will form a ISL (inter switch link).  There is no STP running on V1 or V2 when configured in VCS mode,

So, using the lower scenario, with V1 and V2 running in VCS mode, will port V2-24 get blocked by STP?

     No, as there will be no STP running on the fabric.

Also if you where to say add ports 17 through 24 on V1 to ports 17 though 24 on V2 you would have a 80G frame load balanced ISL giveing you a 3:1 oversubscription.  Unlike a LAG an ISL will split the load up accross all links in a ISL where a LAG would hash the session and send all data over a single 10G link.

Thanks

Michael.

Occasional Contributor
Posts: 6
Registered: ‎08-24-2011

How does VCS Ethernet Fabric interact with Spanning-Tree and ARP?

Excellent, that is the benefit I was hoping to see with VCS ethernet fabric.  So (using the lower drawn scenario), with 6720-V1 and 6720-V2 running in VCS mode and no STP blocking on either V1 port 23 or V2 port 24, traffic between SAN server A and Host Server B will travel across the ISL link (V1-23 --> V2-24) with out needing to traverse Core 1, yes?

I modified the subject line, because now I need to understand how ARP works if there is no STP blocking on either V1-23 or V2-24.  For example, assume that both SAN server A (connected to V1) and Host server B (connected to V2) are in VLAN 10 (which also exists on the core and is trunked to site 2).  Assume SAN A is 10.0.0.1/24 and Host B is 10.0.0.2/24. When SAN A needs to communicate with Host B using 10.0.0.2 for the first time, an ARP broadcast for 10.0.0.2 is generated by SAN server A.  Please correct me if there is something VCS does to change this behaviour, but here is what I imagine happening next.  The ARP broadcast from SAN server A connected to V1 is forwarded (see the F--> in the drawing) out port V1-1 towards Core 1.  Since B was just connected to the nextwork, Core 1 does not have and ARP entry.  As a result Core 1 forwards (F-->) the ARP broadcast for Host server B (10.0.0.2) out port 2 towards V2 port 1.  Once the ARP broadcast comes into port 1 on V2 it gets received by Host B which then sends a unicast response to SAN A out V2 port 1 to Core 1 port 2.  The unicast response travels back to SAN A in a reverse path so V1 ends up learning that the MAC for Host B is reachable via its port 1. 

Here is the part I don't understand.  I didn't draw it in, but at the same time the ARP broadcast from SAN A for 10.0.0.2 Host B is being forwarded out V1 port 1 it is also being forwarded out V1 port 23 and received by V2 on port 24?  If that is true, then the boradcast will be received by Host B and replied back as unicast out V2 port 24 to V1 port 23.  Now if that happens V1 will learn that the MAC for Host B is reachable out 23.  The problem is, as described above, V1 also learned that the MAC for Host B was reachable out port 1.  Port 1 or Port 23?  How is this problem resolved, or does it never come up becase I have misunderstood something?

Thanks

Super Contributor
Posts: 1,087
Registered: ‎12-13-2009

Re: How does VCS Ethernet Fabric interact with Spanning-Tree and ARP?

Hi greg,

So (using the lower drawn scenario), with 6720-V1 and 6720-V2 running in VCS mode and no STP blocking on either V1 port 23 or V2 port 24, traffic between SAN server A and Host Server B will travel across the ISL link (V1-23 --> V2-24) with out needing to traverse Core 1, yes?

     Yes no traffic destioned for another device connect to the frabric goes to the core - the fabric keep everything local.

Ok for the ARP learning, the first thing to understand is that it is a completly new way of building the ARP tables.

Switches in the fabric learn the MAC address of end devices connected to each switch in the fabric

The forwarding database is distributed to all switches in the fabric by eNS

A switches needs not be in the data path to learn the MAC address

Also eNS learns MAC's from both inside the fabric and outside of the fabric

I have attached a MAC learning example for you.

Thanks

Michael.

Occasional Contributor
Posts: 6
Registered: ‎08-24-2011

Re: How does VCS Ethernet Fabric interact with Spanning-Tree and ARP?

Michael,

That's awesome!  I'm pretty sure that will answer all my questions. I may get a few new ones after I read through the MAC learning attachments. 

Thanks!

Super Contributor
Posts: 1,087
Registered: ‎12-13-2009

Re: How does VCS Ethernet Fabric interact with Spanning-Tree and ARP?

You are wlecome Greg.  Have a good day.

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.