03-22-2010 07:49 AM
We have recently begun the process of replacing our old McData 6140 and DS-3232 switches with Brocade 5300s for our Tape SAN. The two switches in the environment are about 16km apart and pass through a DWDM. There are a total of 10 x 2Gb ISLs between the two sites.
I have moved two of the 10 ISLs from the McData to the new Brocades (on each end, of course) using the same DWDM ports and cabling. However, when I bring the Brocade ports online, I begin seeing many errors on the ISL switch ports. Some ports show very few errors (a dozen per minute) while other ports show many errors (hundreds per second). There are no errors on the DWDM units. All of the ISL ports are using longwave GBICs (1380nm). Both of the Brocades are running firmware 6.1.0h.
In this situation, the McData switch ports do not show any errors, but when I move those same cables to the Brocades, I see the errors.
Finally, I would just like to clarify that we are NOT connecting the Brocades and the McData switches into a single fabric. I am disconnecting ISLs from the McDatas (on both ends) and then using those cables to connect the Brocades together. I have already had the fibre cables at each end tested and some of them replaced and have had no further success.
Any help would be greatly appreciated!
03-22-2010 09:04 AM
--->>>...with Brocade 5300s for ...
are in both site 5300 ?
--->>> I have moved two of the 10 ISLs from the McData to the new Brocades (on each end, of course) using the same DWDM ports and cabling. However, when I bring the Brocade ports online, I begin seeing many errors on the ISL switch ports.
It see to me the error is caused from wrong set of BB-Credit beetwen beetwen the two site, because.....
--->>>> In this situation, the McData switch ports do not show any errors, but when I move those same cables to the Brocades, I see the errors.
when you begin to move from 2GB switch to 8GB, a correct BB-Credit must be set.
Have you excluded this Issue ?
03-22-2010 09:55 AM
Thanks for the reply.
Yes, both sites have identical 5300s installed.
I never did actually move from 2Gb to 8Gb. The DWDM is rather old and will not support speeds greater than 2Gb. Therefore, I have hard-coded the ISLs on the Brocades to 2Gb.
As for the BB credit, I used the 'portcfglongdistance' command on all ISL ports and used the same parameters for each port (In this case, I used "LD" mode with a desired distance).
One thing of which I am unsure is the physical cabling. The McData uses 9u longwave GBICs and the Brocades use 1310nm. As far as I know, both wavelengths are supported on the single-mode fibre we're using, but I was wondering if the tolerances are different for these wavelengths. In other words, is it possible that a fibre with certain defects/sharp corners is within tolerance at one wavelength and not within tolerance at another?
03-22-2010 10:13 AM
--->>> I never did actually move from 2Gb to 8Gb.
I mean when you move from Old 2GB Switch to NEW Brocade.....
--->>>The McData uses 9u longwave GBICs....
i think your mean SFP ?
......and the Brocades use 1310nm.
9u longwave and 1310nm are the same
--->>> In other words, is it possible that a fibre with certain defects/sharp corners is within tolerance at one wavelength and not within tolerance at another?
sure, but i don't think this caused the problem, hovewer is probable both SFP are incompatible.
can you tell me please the exact SFP Part Number and Manufacrurung ?
About BB-Credit, see a table.
03-22-2010 10:27 AM
the table indicate BB value for Condor chipset, 5300 use GoldenEye2 Chipset.
as will check for the exact Value and coming back.
As mentioned, I'm 99% quite sure the error is caused by the BB-Credit.
03-22-2010 10:27 AM
The buffers were set automatically when I ran the portcfglongdistance command. I admit to being more of a McData guy than Brocade, but I have not yet found a way to set the buffers arbitrarily on the Brocades as can be done on, say, the McData 6140.
Yes, I meant SFP.
The SFPs at both ends (i.e. on the Brocades) are identical and were received in the same shipment. They are HP part numbers (these Brocades were re-sold from HP):
HP P/N: AJ717A
03-22-2010 11:58 AM
I appreciate your confidence that the issue it related to BB credits and I have not ruled that out. However, I have done some testing and I am not able to confirm your belief that BB credits are to blame.
At the moment, I only have one ISL up and running since the other one gets so many errors that it causes the fabric to become unstable. On the active ISL, the local switch receives very few errors (fewer than one error per minute). On the remote switch, the same ISL receives about 600 errors per minute.
These error rates are almost exactly the same no matter what the BB setting is. So, I use this command to set the reserved buffers to 39:
portcfglongdistance 0 ld 33
And that command is run on both switches (both switches are using port 0 for this ISL). If I lower the buffers to 16 with this command (i.e. setting the distance to 10 km, which is as low as I can go in LD mode):
portcfglongdistance 0 ld 10
the error rates stay the same.
Finally, if I increase the buffers to 106 using this command:
portcfglongdistance 0 ld 100
the error rates still stay the same.
If I am missing something, I would be happy to be corrected. Thanks!
03-22-2010 01:21 PM
Just read your post where you say that it is an old DWDM device. Which device exactly is it?
You can set up the specific number of buffers for a port with portcfgfportbuffers if you need a different value.
03-22-2010 01:32 PM
The DWDM device is Nortel Optera Metro 5200. Honestly, that's about all I can tell you about it. All of the people who set it up originally are no longer with the company, though we do have one guy on the Telecom side who seems to know a bit about it.
It seems the command you suggested does not exist:
portcfg portcfgeport portcfglongdistance portcfgnport portcfgshow
portcfgalpa portcfgexport portcfglosstov portcfgpersistentdisable portcfgspeed
portcfgcreditrecovery portcfggport portcfglport portcfgpersistentenable portcfgtrunkport
portcfgdefault portcfgislmode portcfgnpivport portcfgqos portcfgvexport
03-22-2010 11:08 PM
I asked a Brocade SE about you problem, i think we found the problem. Step2
Can you please create a simple Visio Sheet, Site A 5300 Switch - optera - Site B 5300 Switch, etc....
You say to have configured the port with the command "portcfglongdistance" , hovewer this command requiered a Extended Fabric License, so i assume both switch hare licensed. Is correct ? can you please confirm ?
"portcfgfportbuffers" as mentioned by neverdie is correct, but is supported in FOS 6.2 and above for F_Port, the command is not Support for
Long Distance Port.
Can you please use "portcfglongdistance" "LS" mode with a desired distance ?
About the 5200 2 GB, this is continued supported by Brocade but not longer listed in the Compatibility Matrix as 2GB, can you please use in the Brocade 5300 other 1310 SFP ?
Example use here Finisar 2GB or 4GB or equivalent 1310 SFP ?