Ethernet Switches & Routers

Reply
Contributor
Posts: 29
Registered: ‎12-08-2011

Telnet connection slowed

Hi all,

I have MLX-32 that I can access remotely via telnet, but lately I have problem access it, the response is slow, the response is delayed. for instance if I issued a command I will have to wait for a second or two before I get the response.

It usually happening in peak traffic it is about morning to 05pm. so at first I think the traffic is the cause of this. but then I check the cpu, it just about 20-30% used.

I need your suggestion if you have experienced some or similar problem.

Super Contributor
Posts: 1,087
Registered: ‎12-13-2009

Re: Telnet connection slowed

Suggest you have a look at the two commands below;

     * show tasks      <-- CPU usage statistics for tasks

     * show bm          <-- CPU buffer health

This might give you some insight as to what is happening

Thanks

Michael.

Contributor
Posts: 29
Registered: ‎12-08-2011

Re: Telnet connection slowed

Hi michael, thanks again for your help.

#show task, shows me that lp is busy, it used ~40% cpu.

#show bm is unrecognized, any other helpful command i should try?

do lp have an impact to telnet connections?

Super Contributor
Posts: 1,087
Registered: ‎12-13-2009

Re: Telnet connection slowed

Hi mate,

     No, the IP traffic should not interfere with telnet, assuming you are connecting to the management port for telnet? 

     The only time I have seen slow telnet performance was when I had a broadcast storm and CPU was sitting at 100%, then it was slow.

     The "show bm" command was in ver 5.2.00 - not sure what version you are running.

You can use the "show sfm-links all" to do a quick check on how the frabics are going.

Also "show lp-cpu packet statistics" will let you know about packet drops

But that is about as much as I know here.

Thanks

Michael.

Contributor
Posts: 29
Registered: ‎12-08-2011

Re: Telnet connection slowed

Hi Michael,

I see that IPv4 protocol drop(PEE) is the highest caused of drop from #show lp-cpu packet statistic.

Now I've been thinking, actually it is happening in my MPLS-LDP network, recently I have problem with high CPU utilization at the same time with the telnet connection.

I have about 10 MPLS-node(CES-2024) connected with ringed topology, I see high utilization in mpls-interface, not sure that LDP could cause broadcast storm, because it used IGP ttl to prevent the loop right.

Every MPLS-node connected to its own workgroup switch that have untagged access from VPLS connection. in every workgroup switch I have about 3VLAN that used q-n-q to traverse it into MPLS network.

Here what I think the possible caused.

STP : i dont thinks so, because using L3 link in MPLS network.

LDP : used IGP mecanism to prevent loop

External packet : not sure what it is.

kindly please share your thought?

Many thanks in advance,

Chairul

Super Contributor
Posts: 1,087
Registered: ‎12-13-2009

Re: Telnet connection slowed

Hi mate,

     Not much more I can tell you. But can you confirm you are see this from Telnetting to the dedicated management port on the MLX or on a normal data port?  As a data port will be affected by IP traffic (would need a LOT of traffic to have an affect though).

     If you are using the management port on the management card/s then I would also log this one with the TAC.  Below is some info on DROP code PE.

Packet Error (PE) Drop - A packet is dropped because it is received with errors because the packet is corrupted, or the packet contains incorrect information such as an invalid SA or DA MAC address.

Download the Diagnostic guide from Brocade (goto software download and it should be in the manuals area) and have a look at TABLE 7 Packet drop causes for LP CPU counters.


Thanks

Michael.

Contributor
Posts: 29
Registered: ‎12-08-2011

Re: Telnet connection slowed

Hi Michael,

Thank you for your response,

I am using data port as telnet connection, so maybe yes, it affected by data traffic.

By now I am still have telnet access problem. but looking at #show tasks, it shows that cpu is ~99%idle, so I should not have a telnet problem right.

Below is #show task result,

telnet@QDC_Metro_Dinas#sh task

08:34:36 GMT+07 Sun Jul 19 1903

  Task Name    Pri  State  PC        Stack       Size  CPU Usage(%)  task vid

  -----------  ---  -----  --------  --------  ------  ------------  --------

  idle           0  -      -         -              0           100         0

  con           27  wait   00056f8c  041bcdd0   16384             0         0

  mon           31  wait   00056f8c  04a40f10    8192             0         0

  flash         20  wait   00056f8c  04b4cf40    8192             0         0

  dbg           30  wait   00056f8c  04a46ec0   16384             0         0

  boot          29  wait   00056f8c  04a63df8   65536             0         0

  main           3  wait   00056f8c  244a8730  131072             0       101

  itc            6  wait   00056f8c  244ddb20   16384             0       101

  tmr            5  wait   00056f8c  24792670   16384             0       101

  ip_rx          5  wait   00056f8c  24797f70   16384             0       101

  scp            5  wait   00056f8c  24fac670   32768             0       101

  lpagent        5  wait   00056f8c  24fb2670   16384             0       101

  console        5  wait   00056f8c  24fd3658  131072             0       101

  vlan           5  wait   00056f8c  25000660   32768             0       101

  mac_mgr        5  wait   00056f8c  25008670   16384             0       101

  mrp            5  wait   00056f8c  25060670   16384             0       101

  vsrp           5  wait   00056f8c  25066668   16384             0       101

  erp            5  wait   00056f8c  2506e670   16384             0       101

  mxrp           5  wait   00056f8c  25077670   24576             0       101

  snms           5  wait   00056f8c  2509a670  131072             0       101

  rtm            5  wait   00056f8c  250a1670   16384             0       101

  rtm6           5  wait   00056f8c  252d8670   16384             0       101

  ip_tx          5  wait   00056f8c  2531f670   16384             0       101

  rip            5  wait   00056f8c  25464668   16384             0       101

  l2vpn          5  wait   00056f8c  254cf670  131072             0       101

  mpls           5  wait   00056f8c  254f0670  131072             0       101

  nht            5  wait   00056f8c  25511670  131072             0       101

  mpls_glue      5  wait   00056f8c  25532670  131072             0       101

  bgp            5  wait   00056f8c  255395d8   24576             0       101

  bgp_io         5  wait   00056f8c  25582f28   16384             0       101

  ospf           5  wait   00056f8c  25587670   16384             0       101

  ospf_r_calc    5  wait   00056f8c  255edf38   16384             0       101

  isis           5  wait   00056f8c  255f2670   16384             0       101

  isis_spf       5  wait   00056f8c  255f9f08   16384             0       101

  mcast          5  wait   00056f8c  255fe670   16384             0       101

  msdp           5  wait   00056f8c  2568d670   16384             0       101

  vrrp           5  wait   00056f8c  25697670   16384             0       101

  ripng          5  wait   00056f8c  256a0670   16384             0       101

  ospf6          5  wait   00056f8c  256ab670   20480             0       101

  ospf6_rt       5  wait   00056f8c  256b1f38   20480             0       101

  mcast6         5  wait   00056f8c  256b6670   16384             0       101

  vrrp6          5  wait   00056f8c  256f4670   16384             0       101

  bfd            5  wait   00056f8c  256fb670   16384             0       101

  ipsec          5  wait   00056f8c  25723670   16384             0       101

  l4             5  wait   00056f8c  25749668   16384             0       101

  stp            5  wait   00056f8c  257a1668   65536             0       101

  gvrp_mgr       5  wait   00056f8c  257ad670   16384             0       101

  snmp           5  wait   00056f8c  25ad6670  131072             0       101

  rmon           5  wait   00056f8c  25af7670  131072             0       101

  web            5  wait   00056f8c  25b19670  131072             0       101

  lacp           5  wait   00056f8c  25b1e670   16384             0       101

  dot1x          5  wait   00056f8c  25b24668   16384             0       101

  dot1ag         5  wait   00056f8c  25b30670   32768             0       101

  loop_detect    5  wait   00056f8c  25b35670   16384             0       101

  ccp            5  wait   00056f8c  25b3b670   16384             0       101

  cluster_mgr    5  wait   00056f8c  25b43660   16384             0       101

  hw_access      5  wait   00056f8c  25b53670   32768             0       101

  ntp            5  wait   00056f8c  25b5a670   16384             0       101

  b10b_ovsd      5  wait   00056f8c  25b61660   16384             0       101

  b10b_vswitc    5  wait   00056f8c  25b68670   16384             0       101

  dhcp6          5  wait   00056f8c  25b6f670   16384             0       101

  ospf_msg_ta    6  wait   00056f8c  262b5678   16384             0       101

  lp             3  ready  00056f8c  22828f08  131072             0       102

  LP-I2C         3  wait   00056f8c  23c06ee0    4096             0       102

  telnet_5       5  wait   00056f8c  2849e670  131072             0       101

  telnet_0       5  ready  00056f8c  284bf670  131072             0       101

#show lp-cpu packet statistic

telnet@QDC_Metro_Dinas#sh lp-cpu packet statistics

MODULE 1

----------

Total Packets Received:                  79992994

MPLS uplink packets received:            24830105

    VPLS packets received:               24829329

    VLL packets received:                0

    L3 VPN packets received:             0

    Other MPLS packets received:         776

ARP packets received:                    65910

    ARP request packets received:        32961

    ARP response packets received:       32961

IPV4 packets received:                   11125634

    IPv4 unicats packets routed:         39

    IPv4 protocol packets received:      11125581

    GRE tunnel packets received:         0

    6to4 tunnel packets received:        0

IPV6 packets received:                   0

    IPv6 unicats packets routed:         0

    IPv6 protocol packets received:      0

IPv4 multicast packets routed:           0

IPv6 multicast packets routed:           0

L2VPN endpoint packets received:         43971319

    VPLS endpoint packets received:      43971319

    VLL endpoint packets received:       0

    Local-VLL endpoint packets received: 0

L2 packets received:                     16

    L2 known unicast packets forwarded:  996382

    L2 unknown unicast packets flooded:  0

    L2 broadcast Packets flooded:        0

    L2 multicast Packets flooded:        0

    Packets received for SA learning:    205948

Other packets received:                  0

Total Packets dropped:                   439109

Packet drop causes:

30 ( 0-Unknown cause)

14 (44-IP TTL too small(PE))

439065 (60-VPLS processing drop(PFE))

ARP packets captured for DAI:            65919

ARP packets failed DAI:                  0

The topology shown below.

All configuration are similar to PE1 attached also in picture. Now I am telneting Core via internet, that means the traffic is flow from MLX32.

Currently link 10.1.1.16/30 and 10.1.1.8/30 are down due to high utilization.

Kindly please share your though michael,

Best regards,

Chairul

Contributor
Posts: 29
Registered: ‎12-08-2011

Re: Telnet connection slowed

Hi Michael,

The problem is solved now, It STP problem.

Firstly I check the packet statistic, it shows high utilization of unicast packet, while in time I check it, I am sure there is no possibilities to get such much IP traffic in off office hour. So I do packet capturing, from packet capturing, I see excessive STP packet, I am sure that I put #no spanning tree in global configuration,,, so after that I issue #no span on interface basis and also with #no route only. and then the utilization go down to ~0.1%.

About the telnet packet, yes I have move the connection to management port and found no problem now.

Many thanks for your help Michael.

Super Contributor
Posts: 1,087
Registered: ‎12-13-2009

Re: Telnet connection slowed

Most welcome mate, glad you found the issue.

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.

Click to Register
Download FREE NVMe eBook