01-18-2011 01:36 PM
I have a situation where I am trying to figure out the source of network traffic from an unknown source. Recently we upgraded our ESX links to 10Gb across NetIron MLXs and have started noticing a steady 1.4% OutUtilization across all links that lead to all ESX boxes. When looking at the statistics from the traffic coming into the switch and as compared to the traffic going out to the ESX boxes, the math just doesn't add up.
A few steps we have tried include: Ports have been removed from from all vlans outside of management, Alternate switches, Cable replacement, ESX hosts have had all VMs removed from the machines, ESX host removed from cluster, Redundancy was broken on the ESX host, and etc. The behavior is the same across all switches and ESX hosts. The traffic isn't bound for outside of the local switches since ISL traffic doesn't reflext the increased usage.
Is this normal behavior? If not, is there are way I can track down the source of this traffic considering I don't have 10Gb packet sniffing capabilities at this point.
01-18-2011 02:24 PM
The only thing I can suggest is as traffic is only about 1.4% then mirror the port to see what traffic is comming out.
Are the port setup as router only ports or switch ports? What version of MLX codde are you using?
01-19-2011 05:21 AM
They are running as switch ports and are running 4.1b software. Updating to the 5.x code is off the table for the time being unless it is absolutely necessary.
We are waiting on equipment to be able to properly sniff traffic across a mirrored 10Gb port, but I was hoping there was a simple answer along the lines of management overhead.
01-19-2011 07:23 AM
ok firstly I do not think this is normal. If know the apps (vmware in this case) should not output more traffic then something is. I do not think it would be controll traffic.
From memeory I do not think you can mirror a 10 GB port to a 1GB port. so if you have no 10gb poert host for a miror, then do you have a switch (like a FESX) that you can set the arp age to 0 (this will make it a hub and not a switch that way you can sniff the traffic and see what it is.
01-30-2011 06:15 AM
Looks like your port still in default vlan and default vlan broadcasts are transmitting over port.
Try to put just created untagged vlan to your port and look what will happend.
Another way - try to apply "route-only" command on port.