Fibre Channel (SAN)

Reply
Occasional Contributor
Posts: 18
Registered: ‎10-14-2008

SAN80B IFL issue

Hello team,

I am configuring  new SAN setup consisting of SAN80B with Integrated Routing EX to E port link

after i configured EX port on production site and E port on DR site and enabled the port . both the ports are not coming online and i see many link error and sync loss error on both ports ( primary and DR switches )

I am investigation with with service provided for physical connectivity paralleled and in mean time have few queries regarding EX and E port setting .

Distance between 2 sites is >300 Mile connected via ciena 4200  and i have extended fabric license installed on Prod and DR switch

i want to know what if the setting shown below for EX and E port are correct or i need to make any change .

I tried to change The distance mode to LD with desired distance 300KM but i get and error that i don't have enough buffer available

also do i need different SFP for long distance ? currently i am using  LC SFP (part number 57-1000117-01) connectivity is show as

.site 1 (Prod Switch SAN80B) <.......Ciena 4200...............WDM.................Ciena 4200......> Site 2 ( DR Switch SAN80B )

PROD Switch:admin> portcfgshow 26   (EX port )

Area Number:              26

Speed Level:              AUTO(HW)

Fill Word(On Active)      3(A-A then SW I-A)

Fill Word(Current)        3(A-A then SW I-A)

AL_PA Offset 13:          OFF

Trunk Port                OFF

Long Distance             OFF

VC Link Init              OFF

Locked L_Port             OFF

Locked G_Port             OFF

Disabled E_Port           OFF

Locked E_Port             OFF

ISL R_RDY Mode            OFF

RSCN Suppressed           OFF

Persistent Disable        OFF

LOS TOV enable            OFF

NPIV capability           ON

QOS E_Port                AE

Port Auto Disable:        OFF

Rate Limit                OFF

EX Port                   ON

Mirror Port               OFF

Credit Recovery           ON

F_Port Buffers            OFF

Fault Delay:              0(R_A_TOV)

NPIV PP Limit:            126

CSCTL mode:               OFF

PROD Switch:admin>

-----------------------------------------------------------------

GR Switch :admin> portcfgshow 26 ( E port )

Area Number:              26

Speed Level:              AUTO(HW)

Fill Word(On Active)      3(A-A then SW I-A)

Fill Word(Current)        3(A-A then SW I-A)

AL_PA Offset 13:          OFF

Trunk Port                OFF

Long Distance             OFF

VC Link Init              OFF

Locked L_Port             OFF

Locked G_Port             OFF

Disabled E_Port           OFF

Locked E_Port             ON

ISL R_RDY Mode            OFF

RSCN Suppressed           OFF

Persistent Disable        OFF

LOS TOV enable            OFF

NPIV capability           ON

QOS E_Port                AE

Port Auto Disable:        OFF

Rate Limit                OFF

EX Port                   OFF

Mirror Port               OFF

Credit Recovery           ON

F_Port Buffers            OFF

Fault Delay:              0(R_A_TOV)

NPIV PP Limit:            126

CSCTL mode:               OFF

External Moderator
Posts: 4,857
Registered: ‎02-23-2004

Re: SAN80B IFL issue

you say:

->also do i need different SFP for long distance ? currently i am using  LC SFP (part number 57-1000117-01)

this is a SW = ShortWave SFP+ and as per my opinion cannot work when the port is set in LD Mode.

in such case you need mandatory LongWave SFP

from other side, what type of SFP do you use with WDM ?

TechHelp24
Occasional Contributor
Posts: 18
Registered: ‎10-14-2008

Re: SAN80B IFL issue

hi we use short range 850nm SFP on WDM

External Moderator
Posts: 4,857
Registered: ‎02-23-2004

Re: SAN80B IFL issue

gowhar,

as mentioned, I think - I'm at this point not 100% sure - ShortWave SFP+ cannot be set in LD Mode.

since you are use SW in WDM, i would try to set the Switch Port as default or LE.

try follow, and let me know

portcfgpersistentdisable -->> Port_Number

portcfgspeed --> fix the port as same speed with WDM Port

portcfglongdistance LE Set VC init 1

portcfgqos --> Set on Off

portcfgnpivport --> Set on Off.

portcfgpersistentenable -->> Port_Number

you must do this on both switch

TechHelp24
Occasional Contributor
Posts: 18
Registered: ‎10-14-2008

Re: SAN80B IFL issue

Hello Support,

As mentioned by you above i can set ports of switch as E port only on one switch (site ) as the other switch(site ) as Ex port

i am configuring IFL not ISL .. i set The speed of switch ports on both the site  to 4GB ( WDM ports were configured at 4GB only ) and link is up now

but if i run portshow i dont see wwn on connected device ( should be wwn of switch at other site ? )

fcrfarricshow also dons not show details of switch from other site .... is this normal?

GBXXXXXXXO01:admin> fcrfabricshow

FC Router WWN: 10:00:00:05:33:fc:30:3c, Dom ID:   1,

Info: 17X.VV.XXX.XX, "GBXXXXXXXO01"

   EX_Port       FID    Neighbor Switch Info (enet IP, WWN, name)

   ------------------------------------------------------------------------

      N/A

GXXXXXXXO01:admin> portshow 26

portIndex:  26

portName: port26

portHealth: HEALTHY

Authentication: None

EX_Port Mode:   Enabled

Fabric ID:      30

Front Phantom:  State: OK       Cur Dom ID: 160 WWN: 50:00:53:3f:c3:03:ce:1e

Pr Switch Info:         Dom ID: 3       WWN: 10:00:00:05:33:fc:c1:e0

Fabric params:  R_A_TOV: 0      E_D_TOV: 0      PID fmt: auto

Authentication Type: None

Hash Algorithm: N/A

DH Group: N/A

Edge fabric's primary wwn: N/A

Edge fabric's version stamp: N/A

portDisableReason: None

portCFlags: 0x1

portFlags: 0x10000903    PRESENT ACTIVE G_PORT U_PORT EX_PORT PHY_EX_PORT LOGICAL_ONLINE LOGIN

LocalSwcFlags: 0x0

portType:  18.0

POD Port: Port is licensed

portState: 1    Online

Protocol: FC

portPhys:  6    In_Sync         portScn:   1    Online

port generation number:    392

state transition count:    36

portId:    011a00

portIfId:    43022003

portWwn:   20:1a:00:05:33:fc:30:3c

portWwn of device(s) connected:    ?????????????????????? No WWN connected but still port is shown online

Distance:  normal

portSpeed:  4Gbps

External Moderator
Posts: 4,857
Registered: ‎02-23-2004

Re: SAN80B IFL issue

gowhar,

sorry my typo, please ignore the"portcfgeport" I've edited and remove the line.

with "fcrfabricshow" you should get a similarly output, see below

Brocade_7500:admin> fcrfabricshow

FC Router WWN: 10:00:00:05:1e:40:b8:49, Dom ID: 101,

Info: 172.21.61.84, "Brocade_7500"

   EX_Port     FID Neighbor Switch Info (enet IP, WWN, name)

------------------------------------------------------------------------

      14      6 172.21.61.81 10:00:00:05:33:ea:c4:34   "DCX_Core_B"

      15      7 10.65.39.17 10:00:00:05:1e:05:91:fd "RAE_SW2005_UP"

try "fcredgeshow" and post output here

however i noted you FCR pointed to

-> FC Router WWN: 10:00:00:05:33:fc:30:3c, Dom ID:   1,

Domain ID 1

can you post please details about DomainID both Switches ?

TechHelp24
Occasional Contributor
Posts: 18
Registered: ‎10-14-2008

Re: SAN80B IFL issue

Hello

GBXXXXXXO01:admin> fcredgeshow     <<<<< This is the switch that has EX port configured

                                                                   (SAN80B  with integrated routing     WITH DOMAIN ID 1             

  FID    EX-port        E-port   Neighbor Switch (PWWN, SWWN )

------------------------------------------------------------------------

  30       26             27     20:1b:00:05:33:fc:c1:e0        10:00:00:05:33:fc:c1:e0

  30       27             26     20:1a:00:05:33:fc:c1:e0        10:00:00:05:33:fc:c1:e0

GBNXXXXXO01:admin> switchshow

switchName:     GBNXXXXO01

switchType:     64.3

switchState:    Online

switchMode:     Native

switchRole:     Principal

switchDomain:   1

switchId:       fffc01

switchWwn:      10:00:00:05:33:fc:30:3c

zoning:         ON (MasterA)

switchBeacon:   OFF

FC Router:      ON

FC Router BB Fabric ID: 120

Address Mode:   0

Index Port Address Media Speed State     Proto

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

  26  26   011a00   id     4G   Online      FC  EX-Port  10:00:00:05:33:fc:c1:e0 "" (fabric id = 30 )

  27  27   011b00   id     4G   Online      FC  EX-Port  10:00:00:05:33:fc:c1:e0 "" (fabric id = 30 )

This Switch has E ports configured for IFL with Domain ID of Switch as DOMAIN ID 1


GXXXXXXXTO02:admin> switchshow

switchName:     GXXXXXXXTO02

switchType:     64.3

switchState:    Online

switchMode:     Native

switchRole:     Principal

switchDomain:   3

switchId:       fffc03

switchWwn:      10:00:00:05:33:fc:c1:e0

zoning:         ON (MasterC)

switchBeacon:   OFF

FC Router:      OFF

FC Router BB Fabric ID: 128

Address Mode:   0

Index Port Address Media Speed State     Proto

==============================================

  26  26   031a00   id     4G   Online      FC  E-Port  (unknown)

  27  27   031b00   id     4G   Online      FC  E-Port  (unknown)

If i try to change the port distance mode to LD or LS the links goes offline

External Moderator
Posts: 4,857
Registered: ‎02-23-2004

Re: SAN80B IFL issue

gowhar,

i believe some is a bit confused, and i think you have a mismatch in configured the FCR with correct FID.

please have a look in my both example, fcrfabricshow and fcredgeshow

Brocade_7500:admin> fcrfabricshow

FC Router WWN: 10:00:00:05:1e:40:b8:49, Dom ID: 101,

Info: 172.21.61.84, "Brocade_7500"

   EX_Port     FID Neighbor Switch Info (enet IP, WWN, name)

------------------------------------------------------------------------

      14      6 172.21.61.81 10:00:00:05:33:ea:c4:34   "DCX_Core_B"

      15      7 10.65.39.17 10:00:00:05:1e:05:91:fd "RAE_SW2005_UP"

Brocade_7500:admin> fcredgeshow

  FID     EX-port E-port      Neighbor Switch (PWWN, SWWN )

------------------------------------------------------------------------

  6       14          47 20:2f:00:05:33:ea:c4:34 10:00:00:05:33:ea:c4:34

  7       15          36 20:24:00:05:1e:05:91:fd    10:00:00:05:1e:05:91:fd

as you can see, in this example, EX_Port 14 Point to FID 6, where EX_Port 15 point to FID 7

please have a look here, for some details about how to configure FCR

http://wikibon.org/wiki/v/How_to_configure_FCIP_on_Brocade_7500's#Creating_FCIP_Connections

TechHelp24
Occasional Contributor
Posts: 18
Registered: ‎10-14-2008

Re: SAN80B IFL issue

Thanks  for your input . the issue of unknown E-port got resolved once i changed portcfgislmode to 1 on both E ports of IFL link . but now after i create LSAN the devices are shown as Initializing , not sure why

GBNPSXSTO01:admin> fcrrouteshow

Destination    NR_Port    FCRP Cost    WWN of Principal

Fabric Id       PID                    Switch in the

                                         Dest. Fabric

-----------------------------------------------------------

     30        011a00       1000    10:00:00:05:33:fc:c1:e0

     30        011b00       1000    10:00:00:05:33:fc:c1:e0

GXXXXXXXXX01:admin> fcrfabricshow

FC Router WWN: 10:00:00:05:33:fc:30:3c, Dom ID:   1,

Info: 172.16.173.12, "GBNPSXSTO01"

   EX_Port       FID    Neighbor Switch Info (enet IP, WWN, name)

   ------------------------------------------------------------------------

      26         30     172.30.86.15     10:00:00:05:33:fc:c1:e0   "GYYYYYYY01"

      27         30     172.30.86.15     10:00:00:05:33:fc:c1:e0   "GYYYYYYY01"

GXXXXXXXXX01:admin> lsanzoneshow -s

Fabric ID: 30 Zone Name: LSAN__N1P1

        50:05:07:68:02:15:83:f5  Configured

        50:05:07:68:02:15:84:a9  EXIST

Fabric ID: 30 Zone Name: LSAN__N2P1

        50:05:07:68:02:15:83:f6  Configured

        50:05:07:68:02:15:84:aa  EXIST

Fabric ID: 120 Zone Name: LSAN__N1P1

        50:05:07:68:02:15:83:f5  EXIST

        50:05:07:68:02:15:84:a9 Initializing

Fabric ID: 120 Zone Name: _N2P1

        50:05:07:68:02:15:83:f6  EXIST

        50:05:07:68:02:15:84:aa Initializing

Contributor
Posts: 32
Registered: ‎06-09-2012

Re: SAN80B IFL issue

I dont think so credit recovery is supported in Ex-port,

Can u try disabling the credit recovery on Ex-port and try if that fixes.

"I tried to change The distance mode to LD with desired distance 300KM but i get and error that i don't have enough buffer available "

It requires 606 buffer credits for 4Gbps speed over 300Kms. Are 606 buffer free credits available in Asic???

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.