01-21-2015 09:54 PM
I have a pair of SX800 switches, each with two SX-FI-8XG (8x10gig) gen3 cards, single gen2 management module SX-FIZMR6 and dual switch fabric modules. I'm aware that the SX-FI-8XG card is oversubscribed in some manner, but what I can't figure out is where the bottleneck is and if it is avoidable if you are careful with your setup. Some of the documentation leaves me with the impression that only traffic coming in that is destined for a port on a different card would see the 20 gbps bottleneck getting off the card to the backplane, but other things I've read have made me think all traffic is bottlenecked, or perhaps only traffic crossing port groups (1-4 talking to 5-8, or off the card to the backplane). I can't determine what the real answer is.
The majority of the traffic these units carry is non-routed iSCSI between a Cisco UCS cluster and storage arrays with a unique vlan per storage subnet. Before now, the bottleneck issue was a non-factor because our storage couldn't saturage a single 10gig link. We've just added a flash array to the mix and now we have storage that I've found can easily sustain greater than 20 gbps.
I'm trying to determine if there is a configuration I can use that would avoid any bottleneck the cards have. For example, the array has four 10gig iSCSI interfaces, which are all active. If I put one storage vlan per module, and distribute the initiator side for each vlan on the same modules, does that mean I'd get wire speed from all of them? Or would I need to keep them isolated to ranges of ports? For example, vlan 10 initiators and targets on ports 1-4 of one interface card, vlan 20's initiators and targets on ports 5-8, etc. Or perhaps none of it matters and I can put them wherever I feel like?
01-30-2015 10:02 AM
That card creates 4 virtual 10G ports (mapped to physical ports) in ports 1-4 in one group and ports 5-8 the other group.
Each port group is 4-1 oversubscribed to the actual backplane physical 10G port if that makes sense?
Basically each port group has 10G available to it, maxed. It sounds like you are hitting this issue.
Here's the specifics if they help.