For more details, please see ourCookie Policy.


Fibre Channel (SAN)

Reply
Contributor
Posts: 43
Registered: ‎10-18-2017

Re: FCIP configuration - to use VF or not ?

that is what i need to do.

 

Thanks for clarifying!

 

i have been reading about 4 different PDF's on configurations, but this sums it up nicely and i think i can figure out the nitty gritty details form here..

 

 

External Moderator
Posts: 5,541
Registered: ‎02-23-2004

Re: FCIP configuration - to use VF or not ?

@jmacdonald1

 

to avoid some confusion, in any case is this the same fabric.....

 

you ask feew mouth ago about crossport, do you have configured crossport on the same switch ?

 

if yes, do you have a single or multiple WAN between both side ?

 

Now is a question, why you want to create VF ?

TechHelp24
Contributor
Posts: 43
Registered: ‎10-18-2017

Re: FCIP configuration - to use VF or not ?

currently nothing at the moment is configured on either side, i am just trying to figure out the best way to configure this all

 

my initial design was to put all of the storage ports required for replication into a LSAN, and have a VEX port configured on 1 DCX site, however reviewing that, it appears that the non-VEX port site will be prone to WAN issues as described here

 

There may be cases in which an EBE topology cannot be accommodated; alternatively,
the main production fabric can be isolated from aberrant WAN behavior, while allowing
the backup site to remain exposed. This provides a greater degree of availability and less
risk compared to not using FCR at all. This type of architecture uses VEX_Ports that
connect to a remote edge fabric. The important point to observe here is that the remote
edge fabric continues to be connected to the WAN, and the fabric services span the
WAN all the way to the EX_Port demarcation point. This means that the fabric services
spanning the WAN are subject to disruption and repeated reconvergence, which can
result in an outage within the remote edge fabric. This may not be of great concern if the
remote edge fabric is not being used for production (but merely for backup), since such
WAN fluctuations are not generally ongoing.

 

The problem is that the "DR" site is actually a production site also, and if any WAN issues occur that will affect the fabric services on that site, would not be good.

 

So i got to looking at Virtual Fabrics and logical switches and after doing all the reading and suggestions from here, it looks like if i enable VF on both switches, on each switch create a base logical switch to act as the FCR router, connect the base logical switch to a "storage replication only" logical switch using EX ports, this may be what will give me the best solution for keeping things as isolated as possible.

 

i guess i could also just enable VF create 1 additional logical switch on each DCX, configure the IPIF and routes under the default logical switch, assign the ports (fc and ve) required to the "replication only" logical switch, create the tunnels and let that small fabric merge, it would not affect the default logical switch, or configure a vex port on one side and maybe even if there is WAN issues that would potentially affect the fabric services on the "DR" side, it would only affect the devices defined in that "replication only" logical switch....

 

 

 

Contributor
Posts: 43
Registered: ‎10-18-2017

Re: FCIP configuration - to use VF or not ?

Update on this setup

 

So we went ahead and configured everything up, and it all worked up until the point that the FX8-24 blade encountered an error, but i have a call open on that issue.  I want to just run what i did here across you all to ensure i did it right

the procedure was done on both DCX switches with exact same parameters except the DID's were set to unique domain ID's

 

1) VF was already enabled on both switches

 

2) enable FC routing service - fosconfig --enable fcr on both switches

 

3) issued lscfg  --create 8 -base  (base switch FID set to 8, DID set to unique#  - on other switch base switch FID set to 8, DID set to unique#

 

4) issued lscfg --create 16 (logical switch FID set to 16, DID set to unique# - on other switch, base switch FID set to 16, DID set to unique#

 

5) from FID 128 , issued these 3 commands 1 at a time -  a) lscfg --config 8 -slot 12 -port  b) lscfg --config 8 -slot 12 -port 6   c) lscfg --config 8 -slot 12 -port 22 (this is the VE port to be used) - repeated on other switch

 

6) set context to FID 8, issued portcfgexport 12/0  and then portcfgexport 12/6  - this set those ports to EX_ports - repeated on other switch

 

7) from FID 128, issued the following commands 1 at a time a) lscfg --config 16 -slot 12 -port 2-5  b) lscfg --config 16 -slot 12 -port 8-11  - This was to assign the FC ports to the logical switch - 2 of them will be E_ports, but they should register as E_ports once the cable is plugged into them between FID 8 EX_port and FID 16 E_port - repeated on the other swtich

 

8) set context to FID 128, issued the following  a) portcfg ipif 12/xge1 create <ip_address> netmask <net_mask> mtu 1500  b) portcfg ipif 12/xge0 create <ip_address> netmask <net_mask> mtu 1500 -x  (crossport)   - repeated on the other switch

 

9) still on FID 128 - issued the following  a) portcfg iproute 12/xge1 create <dest_network> netmask <net_mask> <gateway> b) portcfg iproute 12/xge0 create <dest_network> netmask <net_mask> <gateway> -x  - Repeated on the other switch

 

10) issued portcmd --ping against everything, got replies back all the way from both sides

 

11) issued setcontext 8 to log into the FID 8 base switch

 

12) issued portcfg fciptunnel 12/12 create -d <destination_ip> -s <source_ip> -b 500000 -B 2500000 -c auto - Repeated on other switch

 

13) issued portcfg fcipcircuit 12/12 create 1 -d <destination_ip> -s <source_ip> -b 500000 -B 2500000 -c aute - Repeated on other switch

 

After this was done, we brought up everything, and the tunnel came online, everything looked good

 

14) physically plugged in the ISL's between FID 8 and FID 16 logical switches  - this is were we saw our first issue. When we plugged in the first pair, when switchshow was issued on FID 8, I saw this

  96   11    0   1f0000   id    N8       Online      FC  EX-Port  10:00:xx:xx:xx:xx:xx:xx "switchname" (fabric id = 34 )(Trunk master)

I am very curious on why the Fabric ID is showing 34... we did not specify 34 for any FID

Same happened on the other switch, but i think the FID was different, but not anything we specified

 

15) we then tried to plug in the other EX-E pair, and it came up with a FID oversubscription error I think - so we just disabled those ports on each FID for now until i can figure out why that occured - not sure at this moment if there is a configuration error somewhere in the steps above.

 

16) plugged in our storage ports into the  FID 16 logical switch - they came online - repeated on the other switch

 

17) created aliases for both the local storage ports and the remote storage ports - repeated on the other switch

 

18) created LSAN zones to zone the storage to each other - repeated on the other switch with the same LSAN zone names

 

19) on FID 8 logical switch - issued LSANZONESHOW - S  - that resulted in showing the remote WWPN's as imported - repeated on the other switch

 

At this point all looked really good, everything was showing as it should be.  On the Storage side we created the FC partnerships and they actually paired up, but then crashed, as the FCIP tunnel dropped. looking at the swtich we saw the error regarding CHIP 1 Restarted.  On the target switch, we lost all of the IPIF and IPROUTEs

 

opened a support call and they identified a defect with the code level 8.0.2b, and to upgrade to 8.1.0 or 8.2.0

 

We issued a slotpoweroff 12 and then a slotpoweron 12,  this returned the IPIF and IPROUTES back, and everything came back online, tunnels, circuits, everything...  issued a lsanzoneshow -s from FID 8 base switch, the WWPN's were showing imported.

 

However, when we tried to partner the storage together again, it partnered for a few minutes, then died, and looking at the storage, the partner systems were no longer showing as candidates.  - no errors on the switches like what we saw before though, and tunnels stayed up.

 

So i am not sure what is going on now.. i half suspect something in my configuration is wrong, but not sure what

 

Any ideas?

 

Regular Contributor
Posts: 208
Registered: ‎04-04-2018

Re: FCIP configuration - to use VF or not ?

[ Edited ]

Hello,

it some time when i've configured FCR last time. But anyway....

 

Did you configure BACKBONE FABRIC ID ?

Did you specify EDGE FABRIC ID on your EX ports? ( You have to speciffy different Fabric ID on each switch)

 

 

Please mark this response as "Accept as Solution" if it answers your question.

Marian Bezeg
Contributor
Posts: 43
Registered: ‎10-18-2017

Re: FCIP configuration - to use VF or not ?

[ Edited ]

the setup is as i wrote out

 

I did not set a BBFID, as i thought the two logical base swtiches required the same FID to connect. is the BBFID set to be the same as the FID of the base switches (FID 8)? and is it set from FID 8 or FID 128

I think that FCR service will need to be disabled, set BBFID, renable FCR service?

 

as for the EX_port, I m not sure i follow. when i configurd the EXport, it the command was issued from FID 8 (base switch) and the syntax i used was just portcfgexport slot/port -a 1

 

was i supposed to enter portcfgexport slot/port -a 1 fidofotherlogicalswitch  - ie 16?  So portcfgexport slot/port -a 1 16, or is the FID supposed to be a unique number other than 8 or 16, which are in use now by the logical switches

Regular Contributor
Posts: 208
Registered: ‎04-04-2018

Re: FCIP configuration - to use VF or not ?

[ Edited ]

Hello, 

 

from my perspective the configuraiton is not correct.

 

If you are using FCR routing you are not merging fabrics together. You only allow communication between fabrcis based on LSAN zooning. EX port is only there for FC-NAT translation.

  

In normal SAN switch (as i know ) there is no posiiblity to change FID (except VF) as you dont need to do that in normaln operations.

You will specify on your EX port which Edge fabric is attached to this port. Each EX_Port should have defined to which edge fabric is it attached. Because of that you had FID confilict. For two EX ports connected to same EX fabric you have to configure same FID.

  

Its better to configure everything manualy. 

Both base switches should have same BBFID (as i remmember default is 1).

 

There is thee thing which you need to have in mind for FCR. 

And its Front Domain ID (FDID), Fabric ID (FID) and XLATE domain.

 

In this case you may have same FID for logical switches but you will specify on EX port the Fabric ID of the edge fabric which needs to be different. So its better to have different FID of logical switches only to have coresponding FID on EX port. You may have it same but on one EX port you have to specify another number,as we are not joining fabric, but we want to achive routing from one edge to another. 

 

So on SITE A you will have FID 16 so you may define FID 16 on EX port.

On Site B you have FID 16 buy you need to have define another FID on EX port for site B. (as you are want to connect two different fabrics.)

 

After that you have to specify Front domain ID which will is like projection of your edge fabric behind. (from my perspective like proxy device between router and your edge fabric). In this case i will use differen number as your FID.

 

Then you have to configure (or it ist configured automatically? really dont know i always do it manualy) xlate domain. From my perspective you will define to router how to connect to another edge fabric.)

 

So in your case i will change numbers little bit.

 

BBID= 1 

 

SITE A

FID 100

FDID 110

 

SITE B 

FID 200

FDID 210

 

So command will be 

 

SITE A router.

 

portcfgexport (slot/port) -a1 -f 100 -d 110

portcfgexport (slot/port) -a1 -f 100 -d 110

 

****

If you will keep your FID then:

portcfgexport (slot/port) -a1 -f 16 -d 160

portcfgexport (slot/port) -a1 -f 16 -d 160

****

 

 

SITE B router.

 

portcfgexport (slot/port) -a1 -f 200 -d 210

portcfgexport (slot/port) -a1 -f 200 -d 210

 

****

If you will keep your FID then there it should be different:

portcfgexport (slot/port) -a1 -f 17 -d 170

portcfgexport (slot/port) -a1 -f 17 -d 170

****

 

Then you have to speciffy xlate domain config.

 

fcrxlateconfig importedFabricID exportedFabricID preferredDomainID

SITE A router

 

fcrxlateconfig 100 200 210

 

SITE B router

 

fcrxlateconfig 200 100 110

 

To be honest its time ago when i have something with FCR so someone may help you more with this stuff.

  

The configruation about virtual fabric looks fine 

 

Please mark this response as "Accept as Solution" if it answers your question.

Marian Bezeg
Contributor
Posts: 43
Registered: ‎10-18-2017

Re: FCIP configuration - to use VF or not ?

issued an fcrconfigure --show from FID 8 on both swtiches,, Backbone Fabric ID shows 8 for both, so i think that is ok

 

when issuing prtcfgexport slot/port,  i see unique edge fabric ID's  on all EX_Ports and (on switch 1, export1 - 34, export2 - 36) on swtich 2 export1 - 39, export2 - 41)  so all FIDs are completely unique across everything save for the base switches, which require the same FID, and the two logical switches used for storage replication.

 

I am going to dig further and see what else i can find that may be amiss.....

Regular Contributor
Posts: 208
Registered: ‎04-04-2018

Re: FCIP configuration - to use VF or not ?

[ Edited ]

Hello

 

You cant have different FID on EX port attached to the same fabric. The FID must be the same!.

Because of wronf FID you got FID oversubscription error

BBFID must be same, thats true.

 

what is the output of lsanzoneshow command ?

Please mark this response as "Accept as Solution" if it answers your question.

Marian Bezeg
Contributor
Posts: 43
Registered: ‎10-18-2017

Re: FCIP configuration - to use VF or not ?

sanbase1:FID8:admin> lsanzoneshow -s
Fabric ID: 34 Zone Name: lsan_zone_replication_io0
    50:05:07:68:01:70:b9:0b  EXIST
    50:05:07:68:01:80:b9:0b  EXIST
    50:05:07:68:01:70:b8:86  EXIST
    50:05:07:68:01:80:b8:86  EXIST
    50:05:07:68:01:70:b8:82  Imported
    50:05:07:68:01:80:b8:82  Imported
    50:05:07:68:01:70:b4:99  Imported
    50:05:07:68:01:80:b4:99  Imported
Fabric ID: 34 Zone Name: lsan_zone_replication_io1
    50:05:07:68:01:70:b8:75  EXIST
    50:05:07:68:01:80:b8:75  EXIST
    50:05:07:68:01:70:b7:1e  EXIST
    50:05:07:68:01:80:b7:1e  EXIST
    50:05:07:68:01:70:b8:08  Imported
    50:05:07:68:01:80:b8:08  Imported
    50:05:07:68:01:70:b8:af  Imported
    50:05:07:68:01:80:b8:af  Imported
Fabric ID: 34 Zone Name: lsan_zone_replication_io2
    50:05:07:68:0c:53:22:65  EXIST
    50:05:07:68:0c:54:22:65  EXIST
    50:05:07:68:0c:53:22:63  EXIST
    50:05:07:68:0c:54:22:63  EXIST
    50:05:07:68:0c:33:8e:05  Imported
    50:05:07:68:0c:34:8e:05  Imported
    50:05:07:68:0c:33:8e:1a  Imported
    50:05:07:68:0c:34:8e:1a  Imported
Fabric ID: 34 Zone Name: lsan_zone_replication_io3
    50:05:07:68:0c:53:22:64  EXIST
    50:05:07:68:0c:54:22:64  EXIST
    50:05:07:68:0c:53:22:62  EXIST
    50:05:07:68:0c:54:22:62  EXIST
    50:05:07:68:0c:33:8e:19  Imported
    50:05:07:68:0c:34:8e:19  Imported
    50:05:07:68:0c:33:8e:62  Imported
    50:05:07:68:0c:34:8e:62  Imported
Fabric ID: 39 Zone Name: lsan_zone_replication_io0
    50:05:07:68:01:70:b8:82  EXIST
    50:05:07:68:01:80:b8:82  EXIST
    50:05:07:68:01:70:b4:99  EXIST
    50:05:07:68:01:80:b4:99  EXIST
    50:05:07:68:01:70:b9:0b  Imported
    50:05:07:68:01:80:b9:0b  Imported
    50:05:07:68:01:70:b8:86  Imported
    50:05:07:68:01:80:b8:86  Imported
Fabric ID: 39 Zone Name: lsan_zone_replication_io1
    50:05:07:68:01:70:b8:08  EXIST
    50:05:07:68:01:80:b8:08  EXIST
    50:05:07:68:01:70:b8:af  EXIST
    50:05:07:68:01:80:b8:af  EXIST
    50:05:07:68:01:70:b8:75  Imported
    50:05:07:68:01:80:b8:75  Imported
    50:05:07:68:01:70:b7:1e  Imported
    50:05:07:68:01:80:b7:1e  Imported
Fabric ID: 39 Zone Name: lsan_zone_replication_io2
    50:05:07:68:0c:33:8e:05  EXIST
    50:05:07:68:0c:34:8e:05  EXIST
    50:05:07:68:0c:33:8e:1a  EXIST
    50:05:07:68:0c:34:8e:1a  EXIST
    50:05:07:68:0c:53:22:65  Imported
    50:05:07:68:0c:54:22:65  Imported
    50:05:07:68:0c:53:22:63  Imported
    50:05:07:68:0c:54:22:63  Imported
Fabric ID: 39 Zone Name: lsan_zone_replication_io3
    50:05:07:68:0c:33:8e:19  EXIST
    50:05:07:68:0c:34:8e:19  EXIST
    50:05:07:68:0c:33:8e:62  EXIST
    50:05:07:68:0c:34:8e:62  EXIST
    50:05:07:68:0c:53:22:64  Imported
    50:05:07:68:0c:54:22:64  Imported
    50:05:07:68:0c:53:22:62  Imported
    50:05:07:68:0c:54:22:62  Imported


sanbase2:FID8:rayw> lsanzoneshow -s
Fabric ID: 34 Zone Name: lsan_zone_replication_io0
    50:05:07:68:01:70:b9:0b  EXIST
    50:05:07:68:01:80:b9:0b  EXIST
    50:05:07:68:01:70:b8:86  EXIST
    50:05:07:68:01:80:b8:86  EXIST
    50:05:07:68:01:70:b8:82  Imported
    50:05:07:68:01:80:b8:82  Imported
    50:05:07:68:01:70:b4:99  Imported
    50:05:07:68:01:80:b4:99  Imported
Fabric ID: 34 Zone Name: lsan_zone_replication_io1
    50:05:07:68:01:70:b8:75  EXIST
    50:05:07:68:01:80:b8:75  EXIST
    50:05:07:68:01:70:b7:1e  EXIST
    50:05:07:68:01:80:b7:1e  EXIST
    50:05:07:68:01:70:b8:08  Imported
    50:05:07:68:01:80:b8:08  Imported
    50:05:07:68:01:70:b8:af  Imported
    50:05:07:68:01:80:b8:af  Imported
Fabric ID: 34 Zone Name: lsan_zone_replication_io2
    50:05:07:68:0c:53:22:65  EXIST
    50:05:07:68:0c:54:22:65  EXIST
    50:05:07:68:0c:53:22:63  EXIST
    50:05:07:68:0c:54:22:63  EXIST
    50:05:07:68:0c:33:8e:05  Imported
    50:05:07:68:0c:34:8e:05  Imported
    50:05:07:68:0c:33:8e:1a  Imported
    50:05:07:68:0c:34:8e:1a  Imported
Fabric ID: 34 Zone Name: lsan_zone_replication_io3
    50:05:07:68:0c:53:22:64  EXIST
    50:05:07:68:0c:54:22:64  EXIST
    50:05:07:68:0c:53:22:62  EXIST
    50:05:07:68:0c:54:22:62  EXIST
    50:05:07:68:0c:33:8e:19  Imported
    50:05:07:68:0c:34:8e:19  Imported
    50:05:07:68:0c:33:8e:62  Imported
    50:05:07:68:0c:34:8e:62  Imported
Fabric ID: 39 Zone Name: lsan_zone_replication_io0
    50:05:07:68:01:70:b8:82  EXIST
    50:05:07:68:01:80:b8:82  EXIST
    50:05:07:68:01:70:b4:99  EXIST
    50:05:07:68:01:80:b4:99  EXIST
    50:05:07:68:01:70:b9:0b  Imported
    50:05:07:68:01:80:b9:0b  Imported
    50:05:07:68:01:70:b8:86  Imported
    50:05:07:68:01:80:b8:86  Imported
Fabric ID: 39 Zone Name: lsan_zone_replication_io1
    50:05:07:68:01:70:b8:08  EXIST
    50:05:07:68:01:80:b8:08  EXIST
    50:05:07:68:01:70:b8:af  EXIST
    50:05:07:68:01:80:b8:af  EXIST
    50:05:07:68:01:70:b8:75  Imported
    50:05:07:68:01:80:b8:75  Imported
    50:05:07:68:01:70:b7:1e  Imported
    50:05:07:68:01:80:b7:1e  Imported
Fabric ID: 39 Zone Name: lsan_zone_replication_io2
    50:05:07:68:0c:33:8e:05  EXIST
    50:05:07:68:0c:34:8e:05  EXIST
    50:05:07:68:0c:33:8e:1a  EXIST
    50:05:07:68:0c:34:8e:1a  EXIST
    50:05:07:68:0c:53:22:65  Imported
    50:05:07:68:0c:54:22:65  Imported
    50:05:07:68:0c:53:22:63  Imported
    50:05:07:68:0c:54:22:63  Imported
Fabric ID: 39 Zone Name: lsan_zone_replication_io3
    50:05:07:68:0c:33:8e:19  EXIST
    50:05:07:68:0c:34:8e:19  EXIST
    50:05:07:68:0c:33:8e:62  EXIST
    50:05:07:68:0c:34:8e:62  EXIST
    50:05:07:68:0c:53:22:64  Imported
    50:05:07:68:0c:54:22:64  Imported
    50:05:07:68:0c:53:22:62  Imported
    50:05:07:68:0c:54:22:62  Imported

Join the Broadcom Community

Get quick and easy access to valuable resources across the Broadcom Community Network.