Fibre Channel (SAN)

Reply
New Contributor
Posts: 4
Registered: ‎01-16-2012

Extreme rate of NoTxCredits

Hello,

FC switch: Brocade Silkworm 5000, Fabric OS v. 6.1.0d. Role: Subordinate.

On port 18 (SNMP port 19), I'm seeing an extremely high rate of NoTxCredits:

SW-MIB::swFCPortTxFrames.19 = Counter32: 3601318090
SW-MIB::swFCPortRxFrames.19 = Counter32: 4130306939
SW-MIB::swFCPortNoTxCredits.19 = Counter32: 142949157
(sleep 100 seconds)
SW-MIB::swFCPortTxFrames.19 = Counter32: 3601451025
SW-MIB::swFCPortRxFrames.19 = Counter32: 4130510092
SW-MIB::swFCPortNoTxCredits.19 = Counter32: 148706701

Deltas per second:
TxFrames:     1329
RxFrames:     2032
NoTxCredits: 57575

In Port Administration, Port 18 has type=F-port, speed=N4, status=Online, Health=Healthy. The port is connected to a Texas Memory Systems RamSan SSD-storage system.

There is nothing noteworthy in the the switch event log.

How can it be possible to have a rate of NoTxCredits which is an order of magnitude larger than the combined Rx/Tx frame rate? What could it indicate?

Valued Contributor
Posts: 931
Registered: ‎12-30-2009

Re: Extreme rate of NoTxCredits

How can it be possible to have a rate of NoTxCredits which is an order  of magnitude larger than the combined Rx/Tx frame rate? 

NoTxCredit recordes the number of times the TX credit reaches zero. It has nothing to do with the TX frames, let alone the RX, in a sense that you can add both and compare it to NoTxCredit.

In your case the switch is running out of buffer nearly 60k times a second out of it buffers.

What could it  indicate?

As buffers are a flow control mechanism, it means the switch (and thus hosts) is unable  to push more frames to your RAMSAN as credits are not returned fast  enough. For now you could conclude that your RAMSAN is not used fully to the capabilities it has.

Does the RAMSAN have more ports connected to the same or other switch(es) and does it show the same behaviour on those ports as well?

New Contributor
Posts: 4
Registered: ‎01-16-2012

Re: Extreme rate of NoTxCredits

Thanks for your response, dion.v.d.c.

You wrote:

Does the RAMSAN have more ports connected to the same or other  switch(es) and does it show the same behaviour on those ports as well?

The RamSan has four HBA ports. Two of these aren't connected to anything. The remaining two HBA ports go to different FC-switches (on different fabrics, but communicating with the same hosts). The large rate of NoTxCredits is also seen on port connected to the other active HBA.

Valued Contributor
Posts: 931
Registered: ‎12-30-2009

Re: Extreme rate of NoTxCredits

Perhaps it is best then, to attach those port as well to spread the load (should that be the case).

Your switch and FOS lvl unfortunatly don't support configurable f port buffers, you need FOS7 and an 8G or 16G switch.

No chance there to adjust and rule ruling those portbuffers out.

As both connections display the same behaviour I don't suspect a faulty port/cable etc.

I tried looking for some RAMSAN docs, but could find a general best practice or howto configure your RAMSAN and or SAN to get the best performance.

New Contributor
Posts: 4
Registered: ‎01-16-2012

Re: Extreme rate of NoTxCredits

Thanks again for the reply.

We have had some communication with RamSan, and it turns out that we are quite far behind firware-version-wise, and they suggest that a firmware upgrade will help with the NoTxCredits issue.

If that doesn't help, we may have to equip the RamSan box with more HBA port connections.

New Contributor
Posts: 4
Registered: ‎01-16-2012

Re: Extreme rate of NoTxCredits

After having upgraded the RamSan firmware, the NoTxCredits issues went away.

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.