Fibre Channel (SAN)

Reply
New Contributor
Posts: 3
Registered: ‎06-11-2008

Buffer credit problem

I have some problems with increasing tim_txcrd_z counters on all my isl's. I have 2 fabrics each with 2 48000 directors (Fab OS 5.3.0d). There are 2 8 gbit trunks on both fabrics.

I have a very high count of "Time BB credit zero" on all isl's (see below). The ISL's are configured with LE distance parameter (approx 3 km) and there is almost no utilization on the isl's. (no overload).

Just for info: We use HP EVA CA and are forced to set the aptpolicy to Port Based Routing.. DLS is set.

Porterrshow indicates no problems on the fibers. But to be on the safe side. Both cables (single mode) and SFP's (HP 1310 nm) are replaced. All connectors are inspected and cleaned if nessesary.

Any idea whatcauses my problem, and what to check next.??

:O)

/henrik

Example:

SANDIRECTOR_T:admin> portstatsshow 8/23

stat_wtx 720806480 4-byte words transmitted

stat_wrx 3523130592 4-byte words received

stat_ftx 296205109 Frames transmitted

stat_frx 4293919935 Frames received

stat_c2_frx 0 Class 2 frames received

stat_c3_frx 4293853327 Class 3 frames received

stat_lc_rx 33261 Link control frames received

stat_mc_rx 0 Multicast frames received

stat_mc_to 0 Multicast timeouts

stat_mc_tx 0 Multicast frames transmitted

tim_rdy_pri 0 Time R_RDY high priority

tim_txcrd_z 1222049738 Time BB credit zero

er_enc_in 0 Encoding errors inside of frames

er_crc 0 Frames with CRC errors

er_trunc 0 Frames shorter than minimum

er_toolong 0 Frames longer than maximum

er_bad_eof 0 Frames with bad end-of-frame

er_enc_out 0 Encoding error outside of frames

er_bad_os 0 Invalid ordered set

er_c3_timeout 0 Class 3 frames discarded due to timeout

er_c3_dest_unreach 0 Class 3 frames discarded due to destination unreachable

er_other_discard 0 Other discards

er_zone_discard 0 Class 3 frames discarded due to zone mismatch

er_crc_good_eof 0 Crc error with good eof

er_inv_arb 0 Invalid ARB

open 0 loop_open

transfer 0 loop_transfer

opened 0 FL_Port opened

starve_stop 0 tenancies stopped due to starvation

fl_tenancy 0 number of times FL has the tenancy

nl_tenancy 0 number of times NL has the tenancy

zero_tenancy 0 zero tenancy

External Moderator
Posts: 5,034
Registered: ‎02-23-2004

Re: Buffer credit problem

Henrik,

I had same problem in the past, are the Port both ISL Trunl set as Autonegotiate or Fixed Speed.

Can you set one this both ISL as Fixed Speed w/o any interrruption ?

TechHelp24

TechHelp24
New Contributor
Posts: 2
Registered: ‎07-25-2008

Re: Buffer credit problem

Thanx for ypur reply.

Well, in fabric 1 all ISL's are set to 4G. In fabric 2 all ISL's in the first trunk is set to 4G and the other trunk is mixed (4G and N4) (I hadn't noticed that) :|

But the problem is the same on all ISL's, despite different configuration..

:)/henrik

New Contributor
Posts: 3
Registered: ‎06-11-2008

Re: Buffer credit problem

I'm back again.. and I have done some digging:

FC max frame size is 2148 bytes and 1 max sized frame is approx. ½ km long on the fiber when running 4 Gbit. My ILS's are about 3 km long which requires 6 buffer credits.

The portstats64show on an isl port shows:

stat64_rateTxFrame 16498 Tx frame rate (fr/sec)

stat64_rateRxFrame 81453 Rx frame rate (fr/sec)

stat64_rateTxPeakFrame 89329 Tx peak frame rate (fr/sec)

stat64_rateRxPeakFrame 100071 Rx peak frame rate (fr/sec)

stat64_rateTxByte 7472962 Tx Byte rate (bytes/sec)

stat64_rateRxByte 42524674 Rx Byte rate (Bytes/sec)

stat64_rateTxPeakByte 43034423 Tx peak Byte rate (Bytes/sec)

stat64_rateRxPeakByte 51359907 Rx peak Byte rate (Bytes/sec)

****

max bytes divided by max frames: 43034423 (bytes/sec) / 89329 (fr/sec) = 481,75 bytes/fr

Max frame size divided by actual average frame size: 2148 (bytes/fr) / 475,75 (bytes/fr) = 4,51

My conclusion is: There is room for 4,51 more frames with my average frame size compared to max frame size, which requires 6 * 4,51 = 27,06 buffer credits and I only have 26 available..

Am I right ?? Do I make some wrong conlusions above? And any idea what to do about it, besides from reducing isl-port-speed to 2 Gbit or buying an extended fabric license..?

:8}/henrik

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.

vADC is now Pulse Secure
Download FREE NVMe eBook