Fibre Channel (SAN)

Reply
Occasional Contributor
Posts: 11
Registered: ‎07-23-2009

DCX firmware upgrade failed (during upgrade!)

Bad, I know.

The network went down whilst upgrading a DCX from version 6.1.0h to 6.2.0g

It failed during a firmwarecommit command. Here is how it is looking:

_____________________________________________________________

GLSDCX02B:admin> firmwareshow 
Slot Name     Appl     Primary/Secondary Versions       Status
------------------------------------------------------------------------
  6  CP0      FOS          v6.1.0h                                  ACTIVE *
                                   v6.1.0h                                 
  7  CP1      FOS          v6.2.0g                                  STANDBY
                                   Unknown                                
*  Local CP

WARNING: The local CP and remote CP have different versions
of firmware, please retry firmwaredownload command.

_____________________________________________________________

After the failure, it was active on 6.2.0h, I tried a firmwarecommit, it failed. It told me to reboot hte switch and try again. Rebooted. Now active on 6.1.0h. firmwarecommit does nothing now.

Here is the output of hashow:

_____________________________________________________________

GLSDCX02B:admin> hashow
Local CP (Slot 6, CP0): Active, Cold Recovered
Remote CP (Slot 7, CP1): Standby, Healthy
HA enabled, Heartbeat Up, HA State synchronized

_____________________________________________________________

The HA status is currently non-redundant (via GUI)

I can't seem to re-try the firmwaredownload:

_____________________________________________________________

GLSDCX02B:admin> firmwaredownload
Server Name or IP Address: 10.80.102.102
User Name: <username>
File Name: /
Network Protocol(1-auto-select, 2-FTP, 3-SCP) : 2
Password:
Checking system settings for firmwaredownload...
Sanity check failed because firmwareDownload is already in progress.

_____________________________________________________________

The earlier, failed firmwaredownload was initiated via DCFM

I've tried a firmwarerestore too:

_____________________________________________________________

GLSDCX02B:admin> firmwarerestore
Cannot perform firmwarerestore because either the  new firmware has already been committed, or there is no firmwaredownload performed.
Firmwarerestore failed - No firmwarerestore is needed. (0x1f)

_____________________________________________________________

This switch is NOT yet live, so I can perform disruptive commands on it.

Any idea's on how to get the partitions syncronised again?

Any other dumps needed?

Thanks in advance.

Richard.

Occasional Contributor
Posts: 11
Registered: ‎07-23-2009

Re: DCX firmware upgrade failed (during upgrade!)

Quick update:

Rebooted the switch again. Now we are on the 6.2.0g partition. Ran another firmwarecommit command:

_____________________________________________________________________________

switch_128:admin> firmwarecommit
Validating the filesystem ...
Repairing the secondary partition now.
Please wait ...
rsync: read errors mapping "/lib/libkmclient_shared.so": Input/output error (5)
rsync: read errors mapping "/lib/libkmcryptolib.so": Input/output error (5)
rsync: read errors mapping "/lib/libkmclient_shared.so": Input/output error (5)
rsync: read errors mapping "/lib/libkmcryptolib.so": Input/output error (5)
ERROR: lib/libkmclient_shared.so failed verification -- update discarded.
ERROR: lib/libkmcryptolib.so failed verification -- update discarded.
rsync error: some files could not be transferred (code 23) at main.c(977)
rsync: read errors mapping "/lib/libkmclient_shared.so": Input/output error (5)
rsync: read errors mapping "/lib/libkmcryptolib.so": Input/output error (5)
rsync: read errors mapping "/lib/libkmclient_shared.so": Input/output error (5)
rsync: read errors mapping "/lib/libkmcryptolib.so": Input/output error (5)
ERROR: lib/libkmclient_shared.so failed verification -- update discarded.
ERROR: lib/libkmcryptolib.so failed verification -- update discarded.
rsync error: some files could not be transferred (code 23) at main.c(977)
rsync: read errors mapping "/lib/libkmclient_shared.so": Input/output error (5)
rsync: read errors mapping "/lib/libkmcryptolib.so": Input/output error (5)
rsync: read errors mapping "/lib/libkmclient_shared.so": Input/output error (5)
rsync: read errors mapping "/lib/libkmcryptolib.so": Input/output error (5)
ERROR: lib/libkmclient_shared.so failed verification -- update discarded.
ERROR: lib/libkmcryptolib.so failed verification -- update discarded.
rsync error: some files could not be transferred (code 23) at main.c(977)
Failed to synchronize the partitions (0x1700).
Please reboot the switch and run the firmwarecommit command. If the issue persists, Replace the CF card.
switch_128:admin>

_____________________________________________________________________________

Still can't firmwaredownload, keeps saying that a firmware update is still in progress.

Regards

Richard

External Moderator
Posts: 4,809
Registered: ‎02-23-2004

Re: DCX firmware upgrade failed (during upgrade!)

Richard,

its appear 99,99% the installation was interrupted during the Upgrade procedure.

Your can reinstall the FOS with the command -s for single CP for the Corrupted Blade, but I don't think your are able to solve this problem you self !!!!

At this point is the best way you contact the Brocade Support to procede with the Recovery.

TechHelp24
External Moderator
Posts: 4,809
Registered: ‎02-23-2004

Re: DCX firmware upgrade failed (during upgrade!)

How you have solved the FOS Problem ?

TechHelp24
Occasional Contributor
Posts: 11
Registered: ‎07-23-2009

Re: DCX firmware upgrade failed (during upgrade!)

Yes thanks,

EMC had to swap out the CP in the end.

All good now.

Richard

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.