07-04-2013 03:26 AM
we have a 5100 connected to a Ciena DWDM and are seeing many CRC errors in conjunction with the same number (ish) of 'too long' errors being displayed when we run porterrshow.
too long = Frames longer than maximum: The number of frames that are longer than the maximum frame size (36 bytes + data frame size). Data frame size is a variable from 0 to 2112. These errors are in the sum for the LLI errors
the complete path is for a replication link between 2 x HDS VSPs which are talking to an 8Gb/s SFP then into a 4Gb/s ISL but running at 1Gb/s into a Ciena DWDM. When we push load through the 5100 links we are getting 'too long' errors and suspect it might be due to the step down from 8Gb/s to 1Gb/s but dont really know!
07-04-2013 03:38 AM
I would guess the Ciena has more than one 1G port and thus is doing some sort of TDM.
How does your portcfg look like ?
Try switching the ISL to R_RDY mode and set VC Link init to 0 ( @ portcfglongdisatnce)
07-04-2013 04:09 AM
Hi, We are already in R_RDY mode and everything is fine until a load of about 25MB/s on the link then we get an imbalance with 2 out of the 4 links seemingly capped at this mark (both links in the same 5100) but the other ones can go up to 80MB/s (pretty much the max for the 1Gb/s link speed. This is not a constant state though, as we see the low rates on varaible paths but are always seeing the 'too long' errors following the low bandwidth.
We are just trying to understand actually what causes these 'too long' errors so we can work out a plan to fix it! Currently that plan includes a change to 2Gb/s on the link that most often 'bad' but the customer is wanting some reasoning behind the change!
BTW the Ciena is running FC over SONET in the card:
The Ciena kit is5000 Optocal MetroThe card in question is
5000 series 10Gb MOTR VCATsoftware 11.1
and we have V7.1.0a in the 5100s
07-04-2013 04:36 AM
Sorry, I don't know about the exact meaning of the too long errors.
But I have seen similar behavior using one of our 10:1 GFP-T cards in conjunction with two 6510's. Could you post the portcfgshow of the link please?
07-04-2013 04:46 AM
ODC-Rep-B:admin> portcfgshow 0
Area Number: 0
Speed Level: 1G
Fill Word(On Active) 0(Idle-Idle)
Fill Word(Current) 0(Idle-Idle)
AL_PA Offset 13: OFF
Trunk Port ON
Long Distance LD
VC Link Init OFF
Desired Distance 100 Km
Reserved Buffers 56
Locked L_Port OFF
Locked G_Port ON
Disabled E_Port OFF
Locked E_Port ON
ISL R_RDY Mode ON
RSCN Suppressed OFF
Persistent Disable OFF
LOS TOV enable OFF
NPIV capability ON
QOS Port OFF
Port Auto Disable: OFF
Rate Limit OFF
EX Port OFF
Mirror Port OFF
Credit Recovery OFF
F_Port Buffers OFF
Fault Delay: 0(R_A_TOV)
NPIV PP Limit: 126
CSCTL mode: OFF
07-04-2013 06:04 AM
--->>>......which are talking to an 8Gb/s SFP then into a 4Gb/s ISL but running at 1Gb/s.....
I'm here a bit confused, can you please tell me exactly the SFP is inserted in SAN Switch ?
07-04-2013 06:13 AM
Re-reading it I am not surprised!
The storage ports are connected into the 5100 through 8Gb/s SFPs at a fixed speed of 8Gb/s - no problem on these.
The two 5100s are then E-port connected via 4Gb/s SFPs which are running at a fixed speed of 1Gb/s to communicate with the Ciena box - it is these 1Gb/s ports that are giving us the 'too long' errors plus we have CRC errors and enc_out errors as well on these ports.
We have replaced the 4Gb/s SFPS with the same errors being posted.
Light levels are good from -3.5Db on Tx to -6Db on Rx so we are pretty confident we don't have cable issues.
07-04-2013 06:47 AM
Hmm, seems alright to me, perhaps you can also switch off trunking and go to LS mode instead of LD. (to get closer to the FC standard).
But my best guess is that the bit/byte stuffing used by the GFP-T algorithm (used by the MOTR card) causes this issue since it changes the bit stream actively. Not quite sure if VCAT could make it worse, but in principal VCAT should work if implemented correctly on the SDH transport layer .
07-04-2013 10:51 AM
thanks for you comment.
->Re-reading it I am not surprised!
but in entire Thread I not see a bit of word about SFP's you are used.
07-05-2013 03:44 AM
OK. The 8Gb/s SFPs are Brocade 57-1000117-01 and the 4Gb/s SFPs are Brocade 57-1000013-01
The 8Gb/s connect to the storage at 8Gb/s fixed and the 4Gb/s connect to the Ciena at 1Gb/s fixed
The errors are occurring on the 1Gb/s links.
Last night we upped the Ciena speed on one of the 4 links from 1Gb/s to 2Gb/s, replaced the 4Gb/s SFP with an 8Gb/s SFP and fixed its speed to 2Gb/s. All errors disappeared!
So it looks like the 'too long' errors were either an incompatibility between the 8G and 4G (but running at 1G) or the interaction between the 1G in the Brocade and the 1G at the Ciena. This is the clarification we are after.