Design & Build

Data Center Solution-Design Guide: Connecting Brocade VCS Fabric with NOS 3.0 to Existing STP Networks

by on ‎10-14-2013 09:43 AM - edited on ‎05-08-2015 02:50 PM by eyoon@brocade.com (2,917 Views)

Synopsis: Designing the connection of a Brocade VCS Fabric with NOS 3.0.x to an existing STP network with Cisco Catalyst switches.

 

Contents

 

Preface

Overview

Network OS (NOS) 3.0.x,  Brocade VDX Switches support multiple methods to integrate with existing STP (Spanning Tree Protocol) networks:

  • In Standalone mode, Brocade VDX Switches support STP, and therefore, VDX Switches in Standalone mode actively participates in Root Bridge election and Designated Port election by sending STP BPDU (Bridge Protocol Data Units) frames and processing received STP BPDU frames. For how to connect VDX Switches in Standalone mode to existing STP networks, please use the following reference.
  •  

References


  • In VCS Fabric mode, by default Brocade VDX Switches passes through STP BPDU frames. Therefore, VDX Swtiches in VCS Fabric mode do not send STP BPDU frames, nor do they process received STP BPDU frames, but do flood received STP BPDU frames into ports in the same VLAN.

 

This document discusses how to connect Brocade VDX Switches with NOS 3.0.x in VCS Fabric mode to existing Cisco networks running STP.

 

NOTE:  VDX Switches running NOS 4.0.x in VCS Fabric mode support STP sending STP BPDU frames and processing received STP BPDU frames. This preserves the benefits of a VCS Fabric with STP interoperability. A design guide is planned for NOS 4.0.x support of STP when running VDX Switches in VCS Fabric mode. 

 

 

Purpose of This Document

This design guide covers how to connect Brocade VDX Switches running NOS 3.0.x in VCS Fabric mode to an existing STP network with Cisco Switches

 

The following Brocade and Cisco platforms are used in this deployment guide.

  • Brocade VCS Fabric Switches
  • Cisco Catalyst 6500 Switch and 3750 Switch

 

Audience

This document is intended for network design and operation staffs who want to add a Brocade VDX Switches running NOS 3.0.x in VCS Fabric mode to STP networks with Cisco Catalyst switches.

 

Objectives

Provide guidance and recommendations for connecting Brocade VDX Switches running NOS 3.0.x in VCS Fabric mode to existing STP networks with Cisco Catalyst switches.

 

Related Documents

The following documents are valuable resources for the designer. In addition, any Brocade release notes that have been published for Brocade VDX Switches should be reviewed.

 

References

 

About Brocade

Brocade® (NASDAQ: BRCD) networking solutions help the world’s leading organizations transition smoothly to a world where applications and information reside anywhere. This vision is designed to deliver key business benefits such as unmatched simplicity, non-stop networking, application optimization, and investment protection.

Innovative Ethernet and storage networking solutions for data center, campus, and service provider networks help reduce complexity and cost while enabling virtualization and cloud computing to increase business agility.

To help ensure a complete solution, Brocade partners with world-class IT companies and provides comprehensive education, support, and professional services offerings. (www.brocade.com)

 

Key Contributors

The content in this guide was developed by the following key contributors.

  • Lead Architect:          Chris Yoon, Strategic Solutions Lab
  • Technical Author:     Brook Reams, Strategic Solutions Lab

 

Document History

Date                  Version        Description

2013-10-02          1.0               Initial Release

 

 

Spanning Tree Protocol on VCS Fabric

Brocade NOS 3.0.x VCS Fabric passes through STP BPDU frames which means NOS 3.0.x VCS Fabric does not send STP BPDUs frames, nor does it process received STP BPDUs frames. So, it tunnels received STP BPDUs frames through the VCS Fabric and floods them into ports in the same VLAN instead of the VCS Fabric processing as shown in Figure 1-1. Therefore, the VCS Fabric is seen as a wire connection from other STP running switches’ point of view.

 

When in NOS 3.0.x VCS Fabric, a network administrator does not want the received BPDUs forwarded out edge ports directly connected to host devices, the “bpdu-drop enable” command is used to drop the received BPDUs on those edge ports, and therefore, BPDUs won’t be forwarded.

 

VCSFabric3-0WithSTPNetwork_BPDUForwardinginVCSFabricMode .jpg

 

  Figure 1-1 BPDU forwarding in VCS Fabric mode (click to enlarge)

 

Connecting VCS Fabric to Existing STP Network

This chapter offers test results from the Brocade test lab, to provide deployment guidance to how a NOS 3.0.x VCS Fabric can be connected to an existing Cisco Switches’ STP network. It validates that NOS 3.0.x VCS Fabric tunnels received BPDUs through the VCS cluster and they are flooded on the all the switch ports in the same VLAN. Therefore, the VCS Fabric is seen as a wire connection from Cisco Switches’ perspective.

 

 

Test Bed Diagram

Figure 2-1 shows the test bed network diagram.

 

VCSFabric3-0WithSTPNetwork_TestBedNetwork .jpg

   Figure 2-1 Test Bed Network (click to enlarge)

 

Test Methodology

The Cisco Switches’ environment network simulates existing customer infrastructure running STP, and it consists of 2 Cisco Catalyst 6500 Switches, and 2 Cisco Catalyst 3750 Switches. The Cisco Catalyst 6500 Switches are running IOS 122-33.SXI, and the Cisco Catalyst 3750 Switches are running IOS 15.0(1)SE or 12.2(46)SE.

 

VCS 1 simulates NOS 3.0.x VCS Fabric to be connected to existing customer infrastructure running STP, and it consists of 6 VDX Switches (2 VDX8770s, 2 VDX 6730s, and 2 VDX 6720s), and is running NOS 3.0.1a. To form the VCS Fabric, VDX 6720-A, VDX 6720-B, VDX 6730-A and VDX 6730-B each is connected to VDX6770-A and VDX6770-B via TenGigabitEthernet’s Brocade trunk. The VCS 1 are connected to Cisco Switches’ environment via two 2x10GE LACP-based vLAGs (that is, port-channel 1 and port-channel 2) which ensures high availability for multiple nodes and links failure scenarios. All these vLAGs are enabled as Layer 2 trunk using 802.1Q (dot1q), allowing all the VLANs.

 

All interconnections between Cisco Switches are 10GEs or 1GEs, and all the ports are enabled as Layer 2 trunk using 802.1Q (dot1q), allowing all the VLANs.

 

To simulate hosts or servers traffic sent through the network, flows are established using the Spirent Test Center (STC) tool. 300 Mbps traffic flows are sent on each STC port and distributed using full mesh, and the packet size of the flows is 128 bytes.

 

Testing VCS Fabric Connected to Catalyst Switches

 

This section discusses test results from a test network using a NOS 3.0.x VCS Fabric. It shows how a VCS Fabric can be seamlessly integrated with existing Cisco Switches' STP network configured with Rapid PVST+.

 

Configuration

The following are Rapid PVST+ configuration of the Catalyst 6500 Switches. The bridge priority is set to 0 for VLAN 501 on Cat6500-A in an effort to secure the root bridge position.

 

--------------------

Cat6500-A#sh running-config | in spanning-tree

spanning-tree mode rapid-pvst

spanning-tree vlan 501 priority 0

<Cat6500-B#sh running-config | in spanning-tree

spanning-tree mode rapid-pvst

--------------------

 

The following is the VCS Fabric vLAG configuration for VCS1. For more information about how todesign LAG and to verify LAG state on Brocade VDX Switches, see the following reference.

 

References

 

------------------

RB100# show running-config interface Port-channel 1

interface Port-channel 1

vlag ignore-split

switchport

switchport mode trunk

switchport trunk allowed vlan all

switchport trunk tag native-vlan

no shutdown

RB101# show running-config interface Port-channel 2

interface Port-channel 2

vlag ignore-split

switchport

switchport mode trunk

switchport trunk allowed vlan all

switchport trunk tag native-vlan

no shutdown

RB100# show running-config interface TenGigabitEthernet 100/4/34

interface TenGigabitEthernet 100/4/34

no fabric isl enable

no fabric trunk enable

channel-group 1 mode active type standard

lacp timeout long

no shutdown

RB101# show running-config interface TenGigabitEthernet 101/2/34

interface TenGigabitEthernet 101/2/34

no fabric isl enable

no fabric trunk enable

channel-group 1 mode active type standard

lacp timeout long

no shutdown

RB100# show running-config interface TenGigabitEthernet 100/4/33

interface TenGigabitEthernet 100/4/33

no fabric isl enable

no fabric trunk enable

channel-group 2 mode active type standard

lacp timeout long

no shutdown

RB101# show running-config interface TenGigabitEthernet 101/2/33

interface TenGigabitEthernet 101/2/34

no fabric isl enable

no fabric trunk enable

channel-group 2 mode active type standard

lacp timeout long

no shutdown

-----------------------

 

Test Results

 

STP Topology of Existing Cisco Switches' network with Default Port Cost

 

Figure 1-3 shows the STP topology of the existing Cisco Switches' network which is prior to connecting VCS Fabric. In the STP topology, Cat6500-A is elected as the root bridge since Cat6500-A has a bridge priority that is lower than all the other switches. STP calculates the path cost to the root bridge and selects the root port based on the lowest path cost to the root bridge. The root port is always in the ‘Forwarding’ state.

 

Therefore, in the STP topology, Gig 1/0/15 of Cat3750-D is in the ‘Blocking’ state since this port is on a segment where more than one designated port exist and its path cost to the root bridge is the higher

 

 

VCSFabric3-0WithSTPNetwork_STPNetworkwithBlockedPathBeforeConnecting.jpg

   Figure 1-3 STP Topology with Blocked Path Before Connecting Brocade VCS Fabric (click to enlarge)

 

In the Rapid PVST+ on Cisco Catalyst Switches, by default short mode port costs using a 16-bit value are used, and the port cost values from show spanning-tree command should be used to verify if Cisco Catalyst Switches use short mode port costs.

 

For the port-channel’s bandwidth is 20Gbps, its port cost is 1 as shown in Figure 1-3. By default, Cisco Catalyst 6500 Switches use long mode port costs using a 32 bit value in the MSTP instead of short mode port costs. To verify if Cisco Switches use short or long mode port cost, use the port cost values from the output of show spanning-tree and the spanning-tree cost “x” configuration under an interface.

 

Bandwidth

Short Mode STP

port cost value

Long Mode STP

port cost  value

10 Mbps

100

2,000,000

100 Mbps

19

200,000

1 Gbps

4

20,000

N X 1 Gbps

3

10,000

10 Gbps

2

2,000

100 Gbps

N/A

200

1 Tbps

N/A

20

10 Tbps

N/A

2

 

STP Port Cost Value

 

The output of show spanning-tree indicates the Catalyst Switch is using short mode port cost.

 

-----------------------

Cat6500-A#sh spanning-tree

VLAN0501

  Spanning tree enabled protocol rstp

  Root ID Priority    0

             Address     0017.0fec.f9f5

             This bridge is the root

             Hello Time   2 sec Max Age 20 sec  Forward Delay 15 sec

  Bridge ID Priority    0   

             Address 0017.0fec.f9f5

             Hello Time   2 sec Max Age 20 sec  Forward Delay 15 sec

             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type

------------------- ---- --- --------- -------- ----------------------

Gi1/14              Desg FWD 4         128.14   P2p

Gi1/17              Desg FWD 4         128.17   P2p

Te2/3               Desg FWD 2         128.131  P2p

Po1                 Desg FWD        128.1665 P2p

----------------------

 

STP Topology of the VCS Integrated Network with Default STP Port Cost

 

The STP network connected to the VCS Fabric has the default STP port cost. The Gig 1/0/15 port of Cat3750-D and TenGig 2/3 port of Cat6500-B are in a 'Blocking' state since these ports are on segments with more than one designated port and their path cost to the root bridge is higher.

 

As shown in Figure 1-3, prior to the VCS Fabric being connected to the Cisco Switches' STP network, TenGig 2/3 of Cat6500-B is in the 'Forwarding' state. However, after the VCS Fabric is connected, the STP topology changes as show in Figure 1-4 and it causes traffic disruptions inside the Cisco Switches' STP network. For example, the traffic forwarded through TenGig 2/3 of Cat6500-A and Cat6500-B is now blocked. Therefore, traffic between host 1 and host 4, host 2 and host 3, and host 3 and host 4 is disrupted and rerouted through the VCS Fabric.

 

VCSFabric3-0WithSTPNetwork_RapidPVST+NetworkConnectedtoBrocadeVCSFabric .jpg

  Figure 1-4 STP Topology after Connecting Brocade VCS Fabric and Resulting Blocked Paths (click to enlarge)

 

To avoid traffic disruption inside the existing Cisco Switches' STP network, prior to connecting the VCS Fabric, set a port cost value much greater than the largest path cost to the root bridge on the STP network ports connected to the VCS Fabric. This can be configured for port channel interfaces on the Cat6500-A and Cat6500-B as shown below.

 

--------------------------

Cat6500-A# sh run int port-channel 1

Building configuration...

Current configuration : 147 bytes

!

interface Port-channel1

switchport

switchport trunk encapsulation dot1q

switchport mode trunk

load-interval 30

spanning-tree cost 1000

Cat6500-B#sh run int port-channel 2

Building configuration...

Current configuration : 147 bytes

!

interface Port-channel2

switchport

switchport trunk encapsulation dot1q

switchport mode trunk

load-interval 30

spanning-tree cost 1000

-----------------------------

 

STP Topology of VCS Integrated Network with Large Port Cost

 

As shown in Figure 1-5, setting a large port cost (e.g. 1000) makes those port channel interfaces the least preferred path resulting in them being in the 'Blocking' state, and therefore, traffic disruption inside the existing Cisco Switches' STP network does not happen when the VCS fabric is connected up to the existing Cisco Switches' STP network.

 

A network designer should determine which active STP path is desirable for their datacenter architecture prior to connecting up a VCS Fabric to their existing Cisco Switches' STP network.

 

VCSFabric3-0WithSTPNetwork_STPNetworkUsingLargePortCostValue.jpg

   Figure 1-5 STP Topology using large Port Cost value to control STP path with Brocade VCS Fabric (click to enlarge)

 

STP BPDU Drop

 

In the VCS Fabric, the received BPDUs don’t need to be forwarded out edge ports directly connected to host devices. Therefore, the bpdu-drop enable command is enabled on the edge ports (TenGig 11/0/9, TenGig 12/0/9, TenGig 1/0/9, and TenGig 2/0/9).

 

The configuration tested used Cisco Switches running Rapid PVST+, but the same configuration can be applied to other vendors switches running various types of STP.

 

--------------------

RB1# show running-config interface TenGigabitEthernet 11/0/9

interface TenGigabitEthernet 11/0/9

no fabric isl enable

no fabric trunk enable

switchport

switchport mode trunk

switchport trunk allowed vlan all

switchport trunk tag native-vlan

bpdu-drop enable

no shutdown

---------------------

 

Brocade ISL Trunk Failover’s Impact to Existing STP Network

This chapter discusses, based on test results from the test bed network, how a Brocade ISL trunk failover of a NOS 3.0.x VCS Fabric could impact the STP of an existing Cisco Switches’ environment. 

 

 

Test Bed Diagram

The figure 2-1 shows the test bed network diagram.

 

13113_Network diagrm for Brocade ISL trunk failure.jpg

2-1 Test Bed Network (click to enlarge)


Test Methodology

The Cisco Switches’ environment network simulates existing customer infrastructure running STP, and it consists of 2 Cisco Catalyst 6500 Switches, and 2 Cisco Catalyst 3750 Switches. The Cisco Catalyst 6500 Switches are running IOS 122-33.SXI, and the Cisco Catalyst 3750 Switches are running IOS 15.0(1)SE or 12.2(46)SE.

 

 

VCS 1 simulates NOS 3.0.x VCS Fabric to be connected to existing customer infrastructure running STP, and it consists of 3 VDX Switches (2 VDX8770s, and 1 VDX 6720), and is running NOS 3.0.1a. To form the VCS Fabric, one VDX 6720, and two VDX6770 is connected via TenGigabitEthernet’s Brocade ISL trunk as shown above. The VCS 1 are connected to Cisco Switches’ environment via two 2x10GE LACP-based vLAGs (that is, port-channel 1 and port-channel 2) which ensures high availability for multiple nodes and links failure scenarios. All these vLAGs are enabled as Layer 2 trunk using 802.1Q (dot1q), allowing all the VLANs.

 

 

All interconnections between Cisco Switches are 10GEs or 1GEs, and all the ports are enabled as Layer 2 trunk using 802.1Q (dot1q), allowing all the VLANs.

 

 

To simulate hosts or servers traffic sent through the network, flows are established using the Spirent Test Center (STC) tool. 300 Mbps traffic flows are sent on each STC port and distributed using full mesh, and the packet size of the flows is 128 bytes.

 

 

Testing Brocade ISL Trunk Failover’s Impact to Existing STP Network

This section discusses, based on test results from the test bed network, how a Brocade ISL trunk's failover of a NOS 3.0.x VCS Fabric could impact STP BPDU exchange of an existing Cisco Switches’ environment running Rapid PVST+ when there is a redundant Brocade ISL trunk.

 

 

Configuration

The following are Rapid PVST+ configuration of the Catalyst 6500 Switches. The bridge priority is set to 0 for VLAN 501 on Cat6500-A in an effort to secure the root bridge position.

 

 

--------------------

Cat6500-A#sh running-config | in spanning-tree

spanning-tree mode rapid-pvst

spanning-tree vlan 501 priority 0

Cat6500-B#sh running-config | in spanning-tree

spanning-tree mode rapid-pvst

--------------------

 

 

The following is vLAGs’ configuration of VCS1, and for how to design LAG and how to verify LAG state on Brocade VDX Switches, please use the following reference.

 

 

References

 

------------------

RB100# show running-config interface Port-channel 1

interface Port-channel 1

vlag ignore-split

switchport

switchport mode trunk

switchport trunk allowed vlan all

switchport trunk tag native-vlan

no shutdown

RB101# show running-config interface Port-channel 2

interface Port-channel 2

vlag ignore-split

switchport

switchport mode trunk

switchport trunk allowed vlan all

switchport trunk tag native-vlan

no shutdown

RB100# show running-config interface TenGigabitEthernet 100/4/34

interface TenGigabitEthernet 100/4/34

no fabric isl enable

no fabric trunk enable

channel-group 1 mode active type standard

lacp timeout long

no shutdown

RB101# show running-config interface TenGigabitEthernet 101/2/34

interface TenGigabitEthernet 101/2/34

no fabric isl enable

no fabric trunk enable

channel-group 1 mode active type standard

lacp timeout long

no shutdown

RB100# show running-config interface TenGigabitEthernet 100/4/33

interface TenGigabitEthernet 100/4/33

no fabric isl enable

no fabric trunk enable

channel-group 2 mode active type standard

lacp timeout long

no shutdown

RB101# show running-config interface TenGigabitEthernet 101/2/33

interface TenGigabitEthernet 101/2/34

no fabric isl enable

no fabric trunk enable

channel-group 2 mode active type standard

lacp timeout long

no shutdown

-----------------------

 

Test Results

The VCS Fabric has redundant Brocade ISL trunks as shown above, and a Brocade ISL trunk is brought down to test how the trunks failover impacts to the STP of the existing Cisco Switches’ environment. When a Brocade ISL trunk (that is, Tengig 11/0/5 on the RB11) was brought down, the Spanning Tree topology of the existing Cisco environment was not impacted.

 

 

The traffic convergence time was of around 100msec when the Brocade ISL trunk was brought down, but STP BPDU frames are sent every 2 sec on the Cisco Switches. Therefore, it is expected behavior that the Spanning Tree topology of the Cisco environment was not impacted by the Brocade ISL trunk’s failover. 

 

 

Conclusion

Brocade NOS 3.0.x VCS Fabric passes through STP BPDU frames, so it floods received STP BPDU frames into ports in the same VLAN. Therefore, the NOS 3.0.x VCS Fabric is seen as a wire connection from the existing STP infrastructure’s point of view. This allows Brocade NOS 3.0.x VCS Fabric to be seamlessly integrated into the existing STP infrastructure. STP BPDU frames can also be dropped on edge ports in the NOS 3.0.x VCS Fabric for better control of STP topology and behavior.

 

When Brocade NOS 3.0.x VCS Fabric has redundant Brocade ISL trunks, the Spanning Tree topology of the existing STP infrastructure is not impacted by a Brocade ISL trunk's failover.