Fibre Channel (SAN)

Reply
Occasional Contributor
Posts: 7
Registered: ‎11-15-2011

DCX SAN switches: CPU utilization reaching 100%

Hello,

 

We have multiple DCX SAN switches with the firmware 7.2.1c1. We are mainly getting high CPU utilization when we run supportsaves and rarely on some other occations.

 

Is this known issue ? we have opened case with our Switch vendor IBM but they could not find the root cause, it has been happening for the last 4 or 5 months.

 

Regards

 

Guru

 

 

External Moderator
Posts: 4,974
Registered: ‎02-23-2004

Re: DCX SAN switches: CPU utilization reaching 100%

collect please output of "errshow" command in a .txt file, and post here.

 

 

TechHelp24
Occasional Contributor
Posts: 7
Registered: ‎11-15-2011

Re: DCX SAN switches: CPU utilization reaching 100%

Based on my observation, snmpd daemon utilizing high cpu, but this daemon is not running always, it runs once in every 10 mins. whenever it runs, CPU hitting > 85 %. If we start the supportsave when snmpd runs, it is hitting 100% utilization.

 

errshow output given below.

 

 

 errshow
Fabric OS: v7.2.1c1
2015/10/06-06:18:37, [LOG-1003], 1, SLOT 7 | CHASSIS, INFO, NG01_CORE-04_B, The log has been cleared.

Type <CR> to continue, Q<CR> to stop:

2015/10/06-13:40:17, [MAPS-1003], 2, SLOT 7 | FID 92, WARNING, VA_HAR_DR_NG01_B768_CORE-04_B, Switch, Condition=SWITCH(FLOGI/min>6), Current Value:[FLOGI,10 Logins], RuleName=Switch_FLOGI, Dashboard Category=Fabric State Changes.

Type <CR> to continue, Q<CR> to stop:

2015/10/06-14:27:11, [LOG-1000], 5, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Previous message repeated 3 time(s).

Type <CR> to continue, Q<CR> to stop:

2015/10/06-15:13:11, [MAPS-1003], 6, SLOT 7 | FID 92, WARNING, VA_HAR_DR_NG01_B768_CORE-04_B, Switch, Condition=SWITCH(FLOGI/min>6), Current Value:[FLOGI,10 Logins], RuleName=Switch_FLOGI, Dashboard Category=Fabric State Changes.

Type <CR> to continue, Q<CR> to stop:

2015/10/06-15:44:17, [LOG-1000], 8, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Previous message repeated 2 time(s).

Type <CR> to continue, Q<CR> to stop:

2015/10/06-15:49:23, [SEC-1203], 9, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 30.117.12.180

Type <CR> to continue, Q<CR> to stop:

2015/10/06-15:51:36, [SEC-1203], 10, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 30.117.12.180

Type <CR> to continue, Q<CR> to stop:


2015/10/06-15:59:23, [MAPS-1003], 11, SLOT 7 | FID 92, WARNING, VA_HAR_DR_NG01_B768_CORE-04_B, Switch, Condition=SWITCH(FLOGI/min>6), Current Value:[FLOGI,10 Logins], RuleName=Switch_FLOGI, Dashboard Category=Fabric State Changes.

Type <CR> to continue, Q<CR> to stop:
2015/10/06-22:56:18, [SEC-1203], 12, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 30.117.12.181

Type <CR> to continue, Q<CR> to stop:
2015/10/07-06:41:05, [MAPS-1003], 13, SLOT 7 | FID 92, WARNING, VA_HAR_DR_NG01_B768_CORE-04_B, Chassis, Condition=CHASSIS(CPU>=80.00), Current Value:[CPU,91.00 %], RuleName=Chassis_CPU, Dashboard Category=Switch Resource .

Type <CR> to continue, Q<CR> to stop:
2015/10/07-06:43:05, [MAPS-1003], 14, SLOT 7 | FID 92, WARNING, VA_HAR_DR_NG01_B768_CORE-04_B, Chassis, Condition=CHASSIS(CPU>=80.00), Current Value:[CPU,98.00 %], RuleName=Chassis_CPU, Dashboard Category=Switch Resource .

Type <CR> to continue, Q<CR> to stop:
2015/10/07-06:45:05, [MAPS-1003], 15, SLOT 7 | FID 92, WARNING, VA_HAR_DR_NG01_B768_CORE-04_B, Chassis, Condition=CHASSIS(CPU>=80.00), Current Value:[CPU,100.00 %], RuleName=Chassis_CPU, Dashboard Category=Switch Resource .

Type <CR> to continue, Q<CR> to stop:

2015/10/07-06:47:05, [MAPS-1003], 16, SLOT 7 | FID 92, WARNING, VA_HAR_DR_NG01_B768_CORE-04_B, Chassis, Condition=CHASSIS(CPU>=80.00), Current Value:[CPU,100.00 %], RuleName=Chassis_CPU, Dashboard Category=Switch Resource .

Type <CR> to continue, Q<CR> to stop:

2015/10/07-06:49:05, [MAPS-1003], 17, SLOT 7 | FID 92, WARNING, VA_HAR_DR_NG01_B768_CORE-04_B, Chassis, Condition=CHASSIS(CPU>=80.00), Current Value:[CPU,96.00 %], RuleName=Chassis_CPU, Dashboard Category=Switch Resource .

Type <CR> to continue, Q<CR> to stop:


2015/10/07-06:51:05, [MAPS-1003], 18, SLOT 7 | FID 92, WARNING, VA_HAR_DR_NG01_B768_CORE-04_B, Chassis, Condition=CHASSIS(CPU>=80.00), Current Value:[CPU,91.00 %], RuleName=Chassis_CPU, Dashboard Category=Switch Resource .

Type <CR> to continue, Q<CR> to stop:
2015/10/07-06:52:24, [SS-1000], 19, SLOT 7 | CHASSIS, INFO, NG01_CORE-04_B, supportSave has uploaded support information to the host with IP address 30.117.12.180.

Type <CR> to continue, Q<CR> to stop:

2015/10/07-07:55:18, [SEC-1203], 20, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 30.117.12.180

Type <CR> to continue, Q<CR> to stop:

2015/10/07-22:56:17, [SEC-1203], 21, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 30.117.12.181

Type <CR> to continue, Q<CR> to stop:


2015/10/08-22:56:11, [SEC-1203], 22, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 30.117.12.181

Type <CR> to continue, Q<CR> to stop:
2015/10/09-07:51:01, [MAPS-1004], 23, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, SFP 1/0, Condition=ALL_16GSWL_SFP(VOLTAGE<=3000), Current Value:[VOLTAGE,0 mVolts], RuleName=All_16GB_SFP_SWL_Volt_Below, Dashboard Category=Port Health.

Type <CR> to continue, Q<CR> to stop:

2015/10/09-22:56:08, [SEC-1203], 24, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 30.117.12.181

Type <CR> to continue, Q<CR> to stop:

2015/10/10-08:20:59, [MAPS-1004], 25, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, SFP 5/29, Condition=ALL_QSFP(VOLTAGE<=2940), Current Value:[VOLTAGE,0 mVolts], RuleName=All_QSFP_Volt_Below, Dashboard Category=Port Health.

Type <CR> to continue, Q<CR> to stop:

2015/10/10-22:56:07, [SEC-1203], 26, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 30.117.12.181

Type <CR> to continue, Q<CR> to stop:

2015/10/11-21:44:57, [MAPS-1004], 27, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, SFP 1/8, Condition=ALL_16GSWL_SFP(VOLTAGE<=3000), Current Value:[VOLTAGE,0 mVolts], RuleName=All_16GB_SFP_SWL_Volt_Below, Dashboard Category=Port Health.

Type <CR> to continue, Q<CR> to stop:

2015/10/11-22:56:12, [SEC-1203], 28, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 30.117.12.181

Type <CR> to continue, Q<CR> to stop:


2015/10/12-22:56:06, [SEC-1203], 29, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 30.117.12.181

Type <CR> to continue, Q<CR> to stop:
2015/10/13-16:07:14, [HTTP-1002], 30, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Zoning transaction initiated by User: admin, Role: admin completed successfully.

Type <CR> to continue, Q<CR> to stop:

2015/10/13-16:07:16, [ZONE-1022], 31, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, The effective configuration has changed to VA_HAR_CORE_04B. .

Type <CR> to continue, Q<CR> to stop:

2015/10/13-16:07:16, [HTTP-1002], 32, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Zoning transaction initiated by User: admin, Role: admin completed successfully.

Type <CR> to continue, Q<CR> to stop:

2015/10/13-16:08:22, [HTTP-1002], 33, SLOT 7 | FID 92, INFO, VA_HAR_DR_NG01_B768_CORE-04_B, Zoning transaction initiated by User: admin, Role: admin completed successfully.

Type <CR> to continue, Q<CR> to stop:

 

 

 

External Moderator
Posts: 4,974
Registered: ‎02-23-2004

Re: DCX SAN switches: CPU utilization reaching 100%

the last message is 2015/10/13

 

is this the last message ? is a bit suspect.

 

BTW, the message you reported here, are all in range.

 

 

TechHelp24
Contributor
Posts: 25
Registered: ‎01-20-2010

Re: DCX SAN switches: CPU utilization reaching 100%

It happens here as well. IMHO the CPU in the DCX and sadly unchanged in the 8510 is way to weak for the features included in the FOS. Especially when using snmpv3 with BNA and another monitoring tool and running scheduled ssh queries against the switches. The only impact I have seen is slow responses when logging in through SSH so I can live with it.

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.