03-18-2014 01:19 PM
I was called by our repair group with reports of sporadic traffic flow thru our MLX in Sparta.
I tried pinging it. I could not reach it. I tried pinging devices on the other side of the MLX and
they would respond sporadically. All Upstream Interfaces looked good. I checked the upstream BGP sessions
and they would cycle thru and would stay up for around 2 minutes. I had a local tech go into the
Craft port and He issued the command SH CPU LP . He reported the craft port was running slow. The SH CPU LP
showed 1% across the board. I had him run SH CPU and where it normally says 98% idle it was reporting
2% Idle and SNMP 98% We tried the NO SNMP SERVER command but when we tried to enter config mode
the active processor would try to synch with the secondary. Then it would say synch complete and then you would try to enter config t
again it would repeat the same process. We decided to reboot the shelf. I had a constant ping going and they reported the MLX was
acting right from the command line. I had a steady ping going to the shelf. I saw the shelf come up with the 1st ping at 1ms second ping was 3 ms followed Fast forward 3 more pings and we were at 4237ms. The shelf then stopped responding to pings. The craft INterface began to slow down. A quick SH CPU showed as before snmp 98% utilization. Trying to shut snmp-server off repeated the same issue as above. When it rebooted it came up on the left processor. I thought something might be wrong with it. I powered the shelf down and removed the left processor and rebooted the shelf. Same as above. This time I watched the pings come back up and the SH CPU. I saw the snmp process climb again and I had the local tech issue the NO SNMP-SERVER COMMAND. This calmed everything back down to normal.
We run 2 Vrf's on this Mlx. 1 for inside equipment and 1 for internet traffic and both were affected.
The logs were not very helpful. I shut down both Vrf interfaces that have snmp servers talking across and it did nothing to ease the situation. Any thoughts?