Fibre Channel (SAN)

Reply
Occasional Contributor
Posts: 18
Registered: ‎11-15-2012

strange vmstat output

Hello,

I am experiencing a strange situation on my DCX-8510.

When I start monitoring CPU performance no matter with vmstat or top command cpu usage peaks up to 100%

It seems cnmd is using the most part of cpu. Do you have any idea about this issue ?

My FabOS is : v6.4.2b

Here is vmstat output:

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----

r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa

3  0      0 582148  28328 839412    0    0     0     1    1    0 19 49 32  0

1  0      0 582148  28328 839412    0    0     0     0   10  233 29 71  0  0

2  0      0 582156  28328 839412    0    0     0     0   13  226 30 70  0  0

1  0      0 582156  28328 839412    0    0     0     0   14  230 29 71  0  0

1  0      0 582156  28328 839412    0    0     0     0   12  224 33 67  0  0

Here is top output:

top - 15:20:32 up 366 days,  2:06,  2 users,  load average: 4.37, 3.63, 3.07

Tasks: 112 total,   2 running, 110 sleeping,   0 stopped,   0 zombie

Cpu(s): 38.2%us, 60.5%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  1.3%si,  0.0%st

Mem:   1863172k total,  1424000k used,   439172k free,    29536k buffers

Swap:        0k total,        0k used,        0k free,   796864k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND

2068 root      16   0  139m 7504 5088 S 87.0  0.4 312895:23 cnmd

1718 root      16   0 92240 6940 4384 S  6.3  0.4  19730:38 emd

2014 root      16   0 93096  17m 3676 S  3.3  0.9   5314:40 trafd

1982 root      16   0  114m  50m  47m S  2.0  2.8   1198:43 iswitchd

3929 root      16   0  2440 1196  940 R  0.3  0.1   0:00.04 top

    1 root      16   0  1696  592  524 S  0.0  0.0   0:32.44 init

    2 root      34  19     0    0    0 S  0.0  0.0   4:28.77 ksoftirqd/0

Regards,

Niyazi

Valued Contributor
Posts: 761
Registered: ‎06-11-2010

Re: strange vmstat output

Hi,

Do you see any errors in errdump?

in how many switchs you see this behavior?

rgds

Occasional Contributor
Posts: 18
Registered: ‎11-15-2012

Re: strange vmstat output

Hi,

I don't see any errors in errdump.

In all of 4 san directors I see this behavior. They are all at the same FOS level and same hw type.

Sometime I see other processes in top as follows:

  PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  COMMAND
2013 root  16   0  121m  14m 9548 S 39.1  0.8   5316:18 snmpd
1954 root  16   0  153m  23m 9.8m S 27.4  1.3   8871:08 cald
2068 root  16   0  139m 7504 5088 S 16.0  0.4 316041:23 cnmd
1718 root  16   0 92240 6940 4384 S  4.9  0.4  19894:47 emd

Regards,

Niyazi

Valued Contributor
Posts: 761
Registered: ‎06-11-2010

Re: strange vmstat output

Hi,

I reckon that your switchs are experiencing the behavior reported in DEFECT000370943. The fix to this defect has been added to FOS 6.4.3.

Kind regards,

Felipe

Occasional Contributor
Posts: 18
Registered: ‎11-15-2012

Re: strange vmstat output

Hi,

I upgraded to 6.4.3d on DR site and yes cpu problem seems to resolved.

Thanks a lot !

Regards,

Niyazi

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.