Fibre Channel (SAN)

Reply
New Contributor
Posts: 2
Registered: ‎06-25-2012

How to calculate Buffer Credits when you don't know the payload size

Hi, any advice on setting buffer credits for long distance ISLs is greatly appreciated!

We are planning on stretching the FC storage fabrics across DC’s for storage replication

If I understand the non qos calc below you need 56 Buffer Credits when transmitting a full payload @ 2Gbps over a 50KM distance

(Reserved Buffer for Distance Y) = (Distance * LinkSpeed / 2) + 6

(50 km * 2 Gbps / 2) + 6 = 56 buffer

But how do you calculate the correct buffer credits to allocate to the ISLs when the payload can be variable? And you don’t know an average

As I understand it, the smaller the payload the larger the buffer credits you require to ensure you don’t run in to performance problems

2112 is not a realistic number to work with.

I’m wondering if we have to apply the suck it and see approach – he’s why I’m think this;

we can’t work out the average payload size cause storage replication isn’t enabled because the stretched ISL isn’t in place because the buffer credits aren’t set, can’t work out the buffer credits cause I don’t know the average payload size

Cheers

Mike

Contributor
Posts: 27
Registered: ‎10-05-2011

Re: How to calculate Buffer Credits when you don't know the payload size

try and error,

use twice the amount you calculated and check the bb credit zero counters.

Pls be aware that additional equipment in the signal pathe may add some delay too ( e.g WDM) Thus you might add additional credits for this too

hope this helps

Uli

Valued Contributor
Posts: 931
Registered: ‎12-30-2009

Re: How to calculate Buffer Credits when you don't know the payload size

The try and error approach wil get the job done eventually. So is allocating a zillion credits to ports.

However there's is another approach to remove some of the guesswork.

You can calculate the average framesizes yourself.

There are statistics kept on a per port basis. Of those statistics we are looking at the frames counters and word counters.

Usually those counters have some values already, so best to reset the port counters to zero

Depending on the traffic you have to wait a while to get some decent numbers, be aware that older gear/firmware lacks a 64 bit counter. Counters rolling over to 0 invalidates the calculations.

Now with valid frame and word counters you can calculate avg words per frame,

By dividing {word counter}/{framecounter} you end up with the average words sent per frame.

A Brocade word is 4 bytes long. So multiply your {average words per frame} value by 4 and you have the average bytes per frame.

And that value is to be used with the BB credit calculation. In past calculation I've got an average of 1700 bytes for a frame. It gives you an more exact number for your buffer reservation, making sure you do not starve you're ISLs from credits or reserve more credits then actually needed. Of course you can add some more credit to be save.

Occasional Contributor
Posts: 11
Registered: ‎07-04-2013

Re: How to calculate Buffer Credits when you don't know the payload size

I have a similar setup and went for the 'calculate a number and double it' technique initially, given that with an unknown average frame size, the average frame size must be 2112/2 .

I have subsequently used the calculation method described by dion.v.d.c and interestingly got a similar average frame size of around 1700 Bytes, so stuck with my original BB Credits to give some headroom

I'd assume that you won't be going live with your described environment the moment it is setup, so hopefully you have access to resources to be able to present some test LUNs to a 'test' host and use an IO tool to generate various Write IO profiles that may then enable you to better approximate the average frame size.

Happy to be shot down/corrected, but I have always assumed that FC replication frames tend to be predominantely 'Full Size' or 'Small' given that on the Enterprise arrays I work on, data is replicated in chunks bigger than the FC frame payload size. So the frames tend to be full of data, or 'small' amounts of data at the end of a chunk, and then 'small' SCSI write acknowledgements (enable FC Fast Write on the array or switch to reduce the number of these).

Valued Contributor
Posts: 931
Registered: ‎12-30-2009

Re: How to calculate Buffer Credits when you don't know the payload size

I know what you mean with "FC replication frames", but there is no such frame specifically tailored for replication.

As for the Full size or small, I agree. Replication traffic is usually in measured in bigger chunks, for instance HP EVA does it in 16kb chunks. That's an exchange of 8 frames, 7 fullsize and the rest of data in the 8th.

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.

vADC is now Pulse Secure
Download FREE NVMe eBook