Fibre Channel (SAN)

Brocade Access Gateway Connectivity to Cisco MDS Fabrics

by on ‎02-02-2010 03:44 AM (81 Views)

Source: GA-TN-083-00 Feb 12, 2008

This SAN Tech Note describes in detail how to connect Brocade switches in Access Gateway (AG) mode to

Cisco switches. In many instances the connection is straightforward and no special handling of Cisco

switches is required. In certain cases, however, you need to configure the Cisco switch to ensure

interoperability with Access Gateway. If certain QLogic FC ASIC-based Host Bus Adapters (HBAs) (see list in

Table 1) are present behind the Access Gateway, then proceed to the “Special Cases” section of this

document.

If you are using Emulex HBAs or any other HBAs that are not based on QLogic FC ASIC technology, then just

ensure that N_Port ID Virtualization (NPIV) is enabled on the Cisco switch (default is enabled) and that the

switch is running SAN-OS 3.0 (1) or SAN-OS 3.1 (1) or later. To determine the SAN-OS version of the Cisco

switch and to see if NPIV is enabled on the switch, follow the steps in the “Standard Case” section, next.

STANDARD CASES

The standard case is a situation in which none of Organizationally Unique Identifier (OUI) IDs listed in Table

1 are present behind the AG. Make sure that you are using SAN- OS v3.0 (3) or SAN-OS 3.1 (3a) or later.

To determine the SAN-OS version, run the “show version” command. The SAN-OS version appears in the

“Software” section, on the line titled “system.” Below is an example:

ca243# show version

Cisco Storage Area Networking Operating System (SAN-OS) Software

TAC support: http://www.cisco.com/tac

Copyright (c) 2002-2007, Cisco Systems, Inc. All rights reserved.

The copyrights to certain works contained herein are owned by

other third parties and are used and distributed under license.

Some parts of this software may be covered under the GNU Public

License or the GNU Lesser General Public License. A copy of

each such license is available at

http://www.gnu.org/licenses/gpl.html and

http://www.gnu.org/licenses/lgpl.html

Software

BIOS: version 1.1.0

loader: version 1.2(2)

kickstart: version 3.2(1)

system: version 3.2(1) Version of SAN-OS running on the MDS switch

To enable NPIV on the MDS switch (by default NPIV is enabled). Furthermore, on Cisco MDS switches,

NPIV is enabled per switch and not per port.

config t

enable npiv

Press Ctrl-Z to exit

copy run start Saves MDS switch configuration

At this point, you should be able to connect Access Gateway to the Cisco switch.

SPECIAL CASES

Special handling is required when you are connecting the Access Gateway to Cisco fabrics in which certain

QLogic-based devices are present behind the AG. This is caused by incompatibility between the routing

mechanism used by AG and a workaround from Cisco to enable the switch to interoperate with certain

QLogic-based HBAs.

NOTE: This interoperability issue between Cisco switches and certain QLogic-based Fibre Channel (FC)

devices (targets or HBAs) is well documented. For additional information, refer to the “FCID Allocation for

HBAs” section on the Cisco Web site:

http://www.cisco.com/en/US/products/ps5989/products_configuration_guide_chapter09186a0080664cd1.html#wp1269334

Identifying Suspect Devices

The Cisco workaround is to maintain a data base of suspected devices (called “Company ID List” of OUI IDs)

and handle FCID allocation differently if such devices are detected. The most current Cisco Company ID List

is shown in Table 1. The OUI ID is usually the three middle bytes of the World Wide Name (WWN) as shown

in the figure to the right of the Table 1.

Table1. OUI IDs that require special treatment *

TABLE

* List from SAN-OS 3.2 (1) of Cisco MDS switch

** The OUI ID format shown is the one commonly used for Initiator devices. Refer to the Fibre Channel

specification for other possible formats.

How to Handle a Special Case

If there are no FC target devices (for example, storage arrays) present on the same Cisco switch, then follow

the steps in either Option A or B below. If target devices or the AG cannot be isolated to different switches,

then follow the steps in Option C or Option D.

Option A: No Target Devices Present – Edit Company ID List

Delete all of the OUIs of HBAs behind the AG from the Cisco switch Company ID list. See Option C for steps to edit the Company ID List.

Option B: No Target Devices Present – FLAT FCID mode

Alternatively place Cisco switch FCID allocation mode into FLAT mode by following these steps:

config t

fcinterop fcid-allocation flat

vsan database To enter VSAN mode

vsan <vsan#> suspend These two steps enable Flat FCID mode

no vsan <vsan#> suspend

Press Ctrl-Z to exit

copy run start Saves MDS switch configuration

NOTE: If there are any device(s) in that VSAN that you "suspend," it will take that device offline until you

“unsuspend” that VSAN.

Option C: Target Devices Present - Edit Company ID list

In this case you will need to add the OUI of all the target devices present on the switch to the Company ID

List and delete the OUI IDs of all the HBAs that are connected through the Brocade Access Gateway from

the Company ID List. You must remove the OUI IDs if and only if they are in the Company ID List.

First find out the OUIs in the Company ID List as follows:

switch#_show fcid-allocation area

FCID area allocation company id info:

00:50:2E Default entry

00:50:8B

00:60:B0

00:E0:79

00:0D:60 + User-added entry

00:09:6B + User-added entry

00:E0:8B * Explicitly deleted entry (from the original default list)

Total company ids 6

+ - Additional user configured company ids

* -Explicitly deleted company ids from default list.

The following example shows how to add or delete an OUI (0x112233) from the Company ID List.

config t

fcid-allocation area company-id 0x112233 Adds OUI id 0x112233 to list

no fcid-allocation area company-id 0x445566 Deletes OUI id 0x445566 from list

do show fcid-allocation area Displays the list

Press Ctrl-Z to exit

copy run start Saves MDS switch configuration

Make sure the OUI IDs of the attached target devices are listed in the updated Company ID List. Once you have updated the list, you are ready to connect the Access Gateway device. If any of the AG server ports

(F_ports) still report that the port is disabled with reason code “Duplicate ALPA Detected,” then follow

suggestions in the “General Notes” section, next.

Option D: Target Devices Present – Assign FCID

An alternative to Option C is to manually assign know “good” FCIDs to suspect devices behind the Access

Gateway module using the Persistent FCID field in the Cisco GUI tool. When using this method, ensure that

proper FCIDs are assigned that 1) have a different Area field than the target devices connected to the same

MDS switch and 2) meet the AG routing requirements (see Appendix A).

General Notes

Make sure that the debug FLOGI mode is not enabled; Cisco does not support NPIV when FLOGI debug

is set. Verify by running the “show debug flogi” command. To disable FLOGI debug:

config t

no flogi debug

Press Ctrl-Z to exit

copy run start Saves MDS switch configuration

By default, if this is a new or an existing VSAN to be used with the Access Gateway, the default policy

for access is "deny." Either set it to “permit” or zone the devices for access.

Access Gateway has been shown to be compatible with Cisco VSAN, Dynamic Port VSAN (DVPM), and

Inter-VSAN Routing (IVR) features through limited testing at Brocade. Users may need to utilize AG’s

advanced features, such as Port Grouping, to take full advantage of these MDS features.

APPENDIX A: BROCADE ACCESS GATEWAY ROUTING IN 4G SWITCH MODULES

To better understand the interoperability issue between Brocade AG and the Cisco Company ID List, you

may find it helpful to understand the routing mechanism on the Brocade platform. The AG uses the lower

8 bits of the FCID (that is, the ALPA/Port_ID field) to route the frames between its F_ports (connected to

servers) and N_Ports (connected to the fabric). Therefore Access Gateway cannot accept:

Two FCIDs with the same lower 8 bits on the same N_Port (for example, 0xaabb02 & 0xccdd02)

A “00” in the ALPA/Port_ID field of the FCID that is returned for F_ports logins (that is, server HBA

logins behind AG, also known as FDISC logins)

If either of these two situations is detected, the AG will persistently disable the server ports with the reason

code “Duplicate ALPA detected.”