Ethernet Fabric (VDX, CNA)

Reply
Occasional Contributor
Posts: 7
Registered: ‎04-08-2011

Problem forming a VCS cluster with VDX6720

Hi,

we've just bought 4 VDX6720-24 systems and 4 VCS software licenses for our DC. The systems arrived with the NOS 2.1.1 software installed. After installing the VCS licenses on all boxes I started with a little test setup and run into the problem, that I can't build a VCS cluster.
The setup is quite simple: all 4 boxes are connected with ports 20 & 24 to the neighbours that forms a ring in the end.
--
  |            |
--
The VCS ID is "1" on all boxes and the rbridge ID's are distinct (1,2,3,4). If I understood the documentation in the right way, this should be enough to bring the cluster online. But all commands tells me, that this is not the case. The 4 1GE connections are ok, LLDP works on all boxes and I can see the neighbours. Here are some informations from the "vdx4" node - the output from the other nodes only differ in the rbridge ID's, interface names, WWN's and MAC's.
vdx4# sh lic
rbridge-id: 4
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
       Ports on Demand license - additional 8 port upgrade license
       Feature name:PORTS_ON_DEMAND_1
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
       VCS Fabric license
       Feature name:VCS_FABRIC
vdx4# sh lldp nei
Local Intf   Dead Interval  Remaining Life  Remote Intf                   Chassis ID        Tx           Rx
Te 4/0/20    120            116             TenGigabitEthernet 2/0/24     0005.33a2.5c40    8            6         
Te 4/0/24    120            99              TenGigabitEthernet 1/0/20     0005.33a0.2740    179          175      
vdx4# sh vcs
Config Mode    : Local-Only
VCS ID         : 1
Total Number of Nodes           : 1
Rbridge-Id       WWN                            Management IP   Status          HostName
----------------------------------------------------------------------------------------
4               >10:00:00:05:33:AD:0D:B8*       10.0.1.136      Online          vdx4
vdx4# show fabric isl    
No ISLs found
vdx4# show fabric islports
Name:         vdx4
Type:         95.2
State:         Online 
Role:         Fabric Principal
VCS Id:     1
Config Mode:Local-Only
Rbridge-id: 4
WWN:         10:00:00:05:33:ad:0d:b8
FCF MAC:    00:05:33:ad:0d:b8
Index      Interface     State   Operational State
===================================================================
...
20     Te 4/0/20         Down
...
24     Te 4/0/24         Down
...
vdx4# sh ip int brie
Interface                      IP-Address      Status                    Protocol
==========================     ==========      ====================      ========
...
TenGigabitEthernet 4/0/20      unassigned        up                        up
...
TenGigabitEthernet 4/0/24      unassigned        up                        up
Vlan 1                         unassigned        up                        down
vdx4# show run
...
!
interface TenGigabitEthernet 4/0/20
fabric isl enable
fabric trunk enable
no shutdown
!
!
interface TenGigabitEthernet 4/0/24
fabric isl enable
fabric trunk enable
no shutdown
!
One more thing :-) - when booting up the boxes, this two logging messages appear:
...
VCS mode is ENABLED (VCS ID 1)
VCS cluster mode is DISABLED
...
Is this a hint I don't understand?
Cheers,
Gerald

(cleaned the format for easier reading)

Super Contributor
Posts: 1,087
Registered: ‎12-13-2009

Re: Problem forming a VCS cluster with VDX6720

Hi support53,

vdx6720#vcs rbridgeid x vcsid 2 enable

Suggest running the above command on each switch - change x to values 1 to 4.

then check the results of 'show vcs'

If that still does not work, then drop back to 2 units only (as two does not reuire a license for VCS - this is in case there is an issue with the license.

Thanks

Michael.

Occasional Contributor
Posts: 7
Registered: ‎04-08-2011

Re: Problem forming a VCS cluster with VDX6720

Hi Martin,

thx for the quick reply. I've just tried that. BTW - I can't use the "enable" keyword because the VCS is already active:

vdx3# vcs rbridge-id 3 vcsid 2 enable

%Error: Cannot run this operation since VCS is already enabled. Try the command without the enable option set.
So I configured it without the trailing "enable" but after the reboot of all switches nothing has changed - same situation now with VCS ID 2 on all boxes.
Even with only 2 switches connected (vdx1/te20 <--> vdx4/te24) it's still the same.
vdx1# sh vcs
Config Mode    : Local-Only
VCS ID         : 2
Total Number of Nodes           : 1
Rbridge-Id       WWN                            Management IP   Status          HostName
----------------------------------------------------------------------------------------
1               >10:00:00:05:33:A0:27:40*       10.0.1.135      Online          vdx1

vdx1# sh lldp nei
Local Intf   Dead Interval  Remaining Life  Remote Intf                   Chassis ID        Tx           Rx
Te 1/0/20    120            91              TenGigabitEthernet 4/0/24     0005.33ad.0db8    26           24         
vdx1# show fabric islports
Name:       vdx1
Type:       95.2
State:      Online  
Role:       Fabric Principal
VCS Id:     2
Config Mode:Local-Only
Rbridge-id: 1
WWN:        10:00:00:05:33:a0:27:40
FCF MAC:    00:05:33:a0:27:40
Index   Interface      State   Operational State
===================================================================
20     Te 1/0/20         Down
vdx1# sh ip int brie
TenGigabitEthernet 1/0/20      unassigned        up                        up
Maybe I'll try to revert the switches back to the normal mode (no vcs enable) and then re-enable it...
Gerald
Super Contributor
Posts: 1,087
Registered: ‎12-13-2009

Re: Problem forming a VCS cluster with VDX6720

Yep that is what I would next.

disable VCS mode on all boxes.  then leave two not connected.

re enable the two that are still connected.   Brocade also recommands changing the deafults from VCSID 1 to another number and not to use the rbridgeid of 1. 

Thanks

Michael.

Occasional Contributor
Posts: 7
Registered: ‎04-08-2011

Re: Problem forming a VCS cluster with VDX6720

I've tried that now but without success.

After changing the systems vdx1 and vdx4 to normal switching mode with "no vcs enable" and then re-enable VCS on these 2 systems via "vcs rbridge-id (10|11) vcsid 10 enable" it's still the same. The boxes are connected on ports te20 <-> te24 (1G-TX-SFP) and they are seeing each other through LLDP but no cluster is formed.

I'am a little bit lost in the moment. Any further suggestions would be appreciated.

Otherwise I'am going to contact our distributor or brocade directly to find someone who is able to check the setup/systems (e.g. remotely via a TeamViewer session).

Gerald

--

#
vdx1# sh lldp nei
Local Intf   Dead Interval  Remaining Life  Remote Intf                   Chassis ID        Tx           Rx
Te 10/0/20   120            96              TenGigabitEthernet 11/0/24    0005.33ad.0db8    2056         2056       
#
vdx1# sh vcs
Config Mode    : Local-Only
VCS ID         : 10
Total Number of Nodes           : 1
Rbridge-Id       WWN                            Management IP   Status          HostName
----------------------------------------------------------------------------------------
10              >10:00:00:05:33:A0:27:40*       10.0.1.135      Online          sw0
#
#
vdx4# sh vcs
Config Mode    : Local-Only
VCS ID         : 10
Total Number of Nodes           : 1
Rbridge-Id       WWN                            Management IP   Status          HostName
----------------------------------------------------------------------------------------
11              >10:00:00:05:33:AD:0D:B8*       10.0.1.136      Online          sw0
#
vdx4# sh lldp nei
Local Intf   Dead Interval  Remaining Life  Remote Intf                   Chassis ID        Tx           Rx
Te 11/0/24   120            93              TenGigabitEthernet 10/0/20    0005.33a0.2740    2058         2058       
#
Retired-Super Contributor
Posts: 260
Registered: ‎05-12-2010

Re: Problem forming a VCS cluster with VDX6720

gkpop

I note in your post you twice mention GE ports rather than 10 GE.

"...The 4 1GE connections are ok"

"...are connected on ports te20 <-> te24 (1G-TX-SFP)"

If in fact you are using 1 GE ports, then I believe a fabric will not form.  A VCS fabric requires 10 GE ports.

Can you confirm if you have 10GE port optics?  The VDX can be configured with either 1GE or 10GE SFP.

Occasional Contributor
Posts: 7
Registered: ‎04-08-2011

Re: Problem forming a VCS cluster with VDX6720

Hi Brook,

very interesting hint! Because of a lack of LC/LC SMF patchcords I can't use my 10GE-LX SFP+ modules here in the lab and so I've tried all this using 1GE-TX SFP modules. I'am going to obtain some LC/LC SMF patches right now and come back with the result!

Thx,

Gerald

Occasional Contributor
Posts: 7
Registered: ‎04-08-2011

Re: Problem forming a VCS cluster with VDX6720

I have found some LC/LC MMF patches in the lab and noticed that I have 10GE-SX SFP+ modules as well. So I tried this modules instead of the 1GE-TX ones and it works!

I wasn't aware of the "VCS fabric needs 10GE ISL links" fact until your hint - thank you Brook!

Gerald

Retired-Super Contributor
Posts: 260
Registered: ‎05-12-2010

Re: Problem forming a VCS cluster with VDX6720

Gerard,

Cool beans :-)  Glad to hear you are up and running.

Best.

Brook

N/A
Posts: 1
Registered: ‎03-06-2012

Re: Problem forming a VCS cluster with VDX6720

I'm having this issue with a pair of 6710s. As the switches are booting they both say:

VCS mode is ENABLED (VCS ID 10)
VCS cluster mode is DISABLED

We're using 10Gbe twinax with brocade SFPs etc. The clustering comes up ok and the various command output looks alright but show vcs says that it's in local only config mode. Is there such a thing as non-local only config mode? The manuals don't seem to have much detail in this area.

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.

vADC is now Pulse Secure
Download FREE NVMe eBook