For more details, please see ourCookie Policy.


Fibre Channel (SAN)

Reply
New Contributor
Posts: 4
Registered: ‎09-26-2017

DIstance limits for FABRICS

Hi, can anyone point me to documention containing Brocade distance limits for a SAN fabric spanning two Datacentres?

We may be looking at creating a 3rd datacentre 300 miles away.

 

Our current fabric spans 15 km via a WDM.

 

thanks rob

Broadcom Moderator
Posts: 106
Registered: ‎03-29-2010

Re: DIstance limits for FABRICS

Search on my.brocade.com for: SOLUTION DESIGN GUIDE Brocade Extension Solution Design Guide 53-1004573-01 9 September 2016 2016

 

This is the latest regarding distance solutions. If you are planning to go native FC over DWDM, there are a lot of tests that would be implemented before turning it over to production. It's a big subject, but I would strongly recommend knowing a lot about the traffic type, and payload size of the data being sent. Small data payloads of 100-800 bytes is not going to work well over distance due to the buffers required, and the distance latency. Brocade uses buffers for flow control. Getting the right setup may take several trial and refinement jobs.

doc

Any and all information provided by me is for entertainment value and should not be relied upon as a guaranteed solution or warranty of mechantability. All systems and all networks are different and unique. If you have a concern about data loss, or network disconnection, please open a TAC service request for service through Brocade, or through your OEM equipment provider. If this provided you with a solution to this issue, Please mark it with the button at the bottom "Accept as solution".

Contributor
Posts: 50
Registered: ‎05-26-2010

Re: DIstance limits for FABRICS

[ Edited ]

Regarding buffer flow over long distant links. Brocade switch ports configured in "default" mode provide a default number of buffers which is typically sufficient for local data center i/o. Long distance links can often require an adjustment to quantity of buffers allocated. The number of buffers required to keep a long distant link full when operating at max link speed capacity is dependent on: link speed, distance and frame size.

 

Every fibre channel frame requires a buffer. The smaller the average frame
size the more frames can be "fitted" onto the long distant link to keep it full and therefore,

putting more (smaller) frames on a link requires more buffers.

As an example, a 10km 8Gb link requires 40 buffers to keep the link fully utilized when using full size frames (2048 payload bytes). If the same 10km 8Gb link has average frame size of 512 bytes, the number of configured buffers would be 160.

You can find the tx and rx average frame size from the portstats64show output or by calculating the number of tx frames vs.
tx 4-byte words from portstatshow command. Suggest you first clear the port counter and capture new output while the expected
i/o is running over the port to ensure the measurement represents the actually characteristic of the i/o.

Other thoughts;
If your long distance equipment (ie., DWDM) is passive, which is most common, you may want to explore the Brocade portlongdistance and portbuffercalc commands to aid in setting up the number of buffers for
the long distance port.

If your long distance equipment is "spoofing" (providing it's own credits for the long distant link), which is rare in my experience,
then a default port config should be fine.


You can find how many buffers are available for the Brocade switch using portbuffershow. Directors will have output for each line
module.

Lastly, Brocade has an Excel spreadsheet to aid in Buffer Credit calculations:
https://community.brocade.com/t5/Fibre-Channel-SAN-Storage/Fibre-Channel-Buffer-Credit-calculator-spreadsheet-February-2015/ba-p/70873

The comments and advise included are the author's and do not mirror the views and opinions of any company. Any comments, advise, statements or references are without warranty, liability or claims of expertise. You should seek the assistance of the professional entity for subject matter being pursued.
Occasional Contributor
Posts: 14
Registered: ‎06-01-2016

Re: DIstance limits for FABRICS

Also try this guide for FOS 8.1.0:

https://docs.broadcom.com/docs/12381243

 

 

Regards

Ulf

Join the Broadcom Community

Get quick and easy access to valuable resources across the Broadcom Community Network.