Fibre Channel (SAN)

Reply
Regular Visitor
Posts: 1
Registered: ‎01-17-2012

Issues with DCX8510 CP blade's secondary partition reported after upgrade from 8.1.0 to 8.1.1

 

 

Just now seeing error on console. any workaround?

It seems partition on CF card is having issue. 

 

rsync: read errors mapping "/tftpboot/SWBD88/kernel-debug-2.6.14.2-3282828861.ppc.rpm": Input/output error (5)

dcx85108-fcs01:FID128:admin> rsync: read errors mapping "/boot/bootrom.bin": Input/output error (5)

rsync: read errors mapping "/fabos/modules/pluto-zentron-module.ko": Input/output error (5)

rsync: read errors mapping "/tftpboot/SWBD88/kernel-debug-2.6.14.2-3282828861.ppc.rpm": Input/output error (5)

ERROR: boot/bootrom.bin failed verification -- update discarded.

ERROR: fabos/modules/pluto-zentron-module.ko failed verification -- update discarded.

ERROR: tftpboot/SWBD88/kernel-debug-2.6.14.2-3282828861.ppc.rpm failed verification -- update discarded.

rsync error: some files could not be transferred (code 23) at main.c(977) [sender=2.6.9]

 

 

 

----------

 

Here’s what it says..

 

dcx85108-fcs01:FID128:admin> hashow

Local CP (Slot 7, CP1): Standby

Remote CP (Slot 6, CP0): Active

HA enabled, Heartbeat Up, HA State synchronized

rsync: read errors mapping "/fabos/modules/pluto-zentron-module.ko": Input/output error (5)

 

dcx85108-fcs01:FID128:admin>

dcx85108-fcs01:FID128:admin> firmwareshow

Slot Name       Appl     Primary/Secondary Versions               Status

--------------------------------------------------------------------------

  6  CP0        FOS      v8.1.1                                   ACTIVE

                         v8.1.1

  7  CP1        FOS      v8.1.1                                   STANDBY *

                         Unknown

*  Local CP

 

WARNING: Firmwaredownload is in progress.

 

 

 

 

 

----------

 

 

 

FAILED again.

 

rsync: read errors mapping "/tftpboot/SWBD88/kernel-debug-2.6.14.2-3282828861.ppc.rpm": Input/output error (5)

rsync: read errors mapping "/boot/bootrom.bin": Input/output error (5)

rsync: read errors mapping "/fabos/modules/pluto-zentron-module.ko": Input/output error (5)

rsync: read errors mapping "/tftpboot/SWBD88/kernel-debug-2.6.14.2-3282828861.ppc.rpm": Input/output error (5)

ERROR: boot/bootrom.bin failed verification -- update discarded.

ERROR: fabos/modules/pluto-zentron-module.ko failed verification -- update discarded.

ERROR: tftpboot/SWBD88/kernel-debug-2.6.14.2-3282828861.ppc.rpm failed verification -- update discarded.

rsync error: some files could not be transferred (code 23) at main.c(977) [sender=2.6.9]

 

dcx85108-fcs01:FID128:admin> rsync: read errors mapping "/boot/bootrom.bin": Input/output error (5)

 

 

 

 

-

----

 

It seems trying to recover secondary partition but then its failing.

 

--- Partition /dev/hda1 is inconsistent.

 

dcx85108-fcs01:FID128:admin> rsync: read errors mapping "/fabos/modules/pluto-zentron-module.ko": Input/output error (5)

Service instances out of sync

 

[64]: Mon Oct 16 17:01:55 2017

Slot 7 (CP1, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[65]: Mon Oct 16 17:06:31 2017

Slot 7 (CP1, standby): Firmware commit operation has started to restore the secondary partition.

 

[66]: Mon Oct 16 17:06:31 2017

Slot 7 (CP1, standby): The firmware commit operation has started. This may take up to 10 minutes.

 

dcx85108-fcs01:FID128:admin> Repairing the secondary partition now.

Please wait ...

rsync: read errors mapping "/boot/bootrom.bin": Input/output error (5)

 

 

 

 

dcx85108-fcs01:FID128:admin> hashow

Local CP (Slot 6, CP0): Standby

Remote CP (Slot 7, CP1): Active

HA enabled, Heartbeat Up, HA State synchronized

dcx85108-fcs01:FID128:admin> firmwareshow

Slot Name       Appl     Primary/Secondary Versions               Status

--------------------------------------------------------------------------

  6  CP0        FOS      v8.1.1                                   STANDBY *

                         v8.1.1

  7  CP1        FOS      v8.1.1                                   ACTIVE

                         Unknown

*  Local CP

 

WARNING: Firmwaredownload is in progress.

dcx85108-fcs01:FID128:admin> exit

logout

 

Fabric OS (dcx85108-fcs01)

 

 

telnet> quit01 console login:

Connection to ts41h41.gdl.englab.netapp.com closed.

bash-3.00$ rconsole dcx85108-fcs01-cp1

dcx85108-fcs01-cp1(=>dcx85108-fcs01-cp1.gdl.englab.netapp.com) - telnet requested to ts41h41.gdl.englab.netapp.com 2044

Trying 10.228.1.68...

Connected to ts41h41.gdl.englab.netapp.com.

Escape character is '^]'.

 

 

 

Fabric OS (dcx85108-fcs01)

 

 

dcx85108-fcs01 console login: admin

Password:

dcx85108-fcs01:FID128:admin>

dcx85108-fcs01:FID128:admin> slotshow

 

Slot   Blade Type     ID    Status

-----------------------------------

  1     SW BLADE     96     ENABLED

  2     UNKNOWN             VACANT

  3     SW BLADE    153     ENABLED

  4     UNKNOWN             VACANT

  5     CORE BLADE   98     ENABLED

  6     CP BLADE     50     ENABLED

  7     CP BLADE     50     ENABLED

  8     CORE BLADE   98     ENABLED

  9     UNKNOWN             VACANT

10     SW BLADE     97     ENABLED

11     UNKNOWN             VACANT

12     UNKNOWN             VACANT

 

 

dcx85108-fcs01:FID128:admin>

dcx85108-fcs01:FID128:admin> reboot

Warning: This command is being run on a control processor (CP)

based system and will cause the active CP to reboot.

This will cause disruption to all traffic in this chassis.

Are you sure you want to reboot the active CP [y/n]?y

2017/10/16-17:03:42, [RAS-1007], 17074, SLOT 7 CHASSIS, INFO, Brocade_DCX, System is about to reload.

Rebooting! Mon Oct 16 17:03:52 EDT 2017

 

Broadcast message from root (ttyS0) Mon Oct 16 17:03:52 2017...

 

The system is going down for reboot NOW !!

INIT: Switching to runlevel: 6

INIT: Sending processes the TERM signal

dcx85108-fcs01:FID128:admin> rsync: read errors mapping "/fabos/modules/pluto-zentron-module.ko": Input/output error (5)

Service instances out of sync

2017/10/16-17:03:55, [FSSM-1003], 17075, SLOT 7 CHASSIS, WARNING, Brocade_DCX, HA State out of sync.

eth0: Link is down at 31696881 jiffies

eth3: Link is down at 31696897 jiffies

bonding: bond0: released all slaves

eth1: Link is down at 31696920 jiffies

eth2: Link is down at 31696928 jiffies

##RSCMON exiting due to signal: 19, pending signals: 0x0, 0x0

##exiting due to signal: 19, pending signals: 0x0, 0x0

Stopping diagnostics

Unmounting all filesystems.

Please stand by while rebooting the system...

reboot_event: Calling bus0 flush

Restarting system.

.

 

The system is coming up, please wait...

 

 

U-Boot 1.1.3 (Jun  1 2017 - 11:38:13)

 

CPU:   8548_E, Version: 2.1, (0x80390021)

Core:  E500, Version: 2.2, (0x80210022)

Clock Configuration:

       CPU:1199 MHz, CCB: 399 MHz,

       DDR: 199 MHz, LBC:  49 MHz

L1:    D-cache 32 kB enabled

       I-cache 32 kB enabled

Board: Zentron

CPU Board Revision 255.198 (0xffc6)

    PCI2: disabled

I2C:   ready

DRAM:  initdram: Initializing DDRSDRAM

initdram: Normal boot in progress

dimm_spd_sdram 2001: memsize = 800

initdram: Call ddr_enable_ecc dram size -2147483648

    DDR: 2048 MB

POST RAM test disabled.

Adjusted memory 0x7d000000 2097152000

Now running in RAM - U-Boot at: 74f7d000

trap_init : 0x0

system inventory subsystem initialized

FLASH:  4 MB

L2 cache 512KB: enabled

CPLD: Init complete

ATA interface setup upm 256: ffcff00, ffcfc04, ffffc00, fffffc01

PCI: gd->brcd_flags = 0, PCI init

Skip our host bridge

        00  11  8086  b555  0680  1a

        00  12  8086  b555  0680  1a

        00  13  8086  b555  0680  1a

        00  14  1131  1561  0c03  3b

        00  14  1131  1562  0c03  3b

           CPLD rev: 0xd0

  CPU0 CPLD version: 0xba

In:    serial

Out:   serial

Err:   serial

Net:

ENET2: PHY is not applicable

ENET1: PHY is Broadcom BCM5461S 10/100/1000 BaseT PHY (2060c1)

ENET0: PHY is Broadcom BCM5461S 10/100/1000 BaseT PHY (2060c1)

ENET3: PHY is Broadcom BCM5461S 10/100/1000 BaseT PHY (2060c1)

 

board_init_r: Adjusted ram 2097152000 0x7d000000 2048000k

Checking system RAM - press any key to stop test

 

Checking memory address: 00100000

 

System RAM test using Default POST RAM Test succeeded.

 

set_bootstatus: BS_LOAD_OS, platform_idx = 2

Hit ESC to stop autoboot:  0

Map file at LBA sector 0x2b8100

do_bootm 171: load_addr 0x02000000 addr 0x00400000

do_bootm 174: ## Booting image at 00400000 load_addr 0x02000000 ...

   Image Name:   Linux-2.6.14.2

   Image Type:   PowerPC Linux Multi-File Image (gzip compressed)

   Data Size:    2675443 Bytes =  2.6 MB

   Load Address: 00000000

   Entry Point:  00000000

   Contents:

   Image 0:  1596767 Bytes =  1.5 MB

   Image 1:  1078663 Bytes =  1 MB

   Uncompressing Multi-File Image ... do_bootm_linux 565: ## Current stack ends at 0x74F5BB38 => set upper limit to 0x00800000

## initrd at 0x00585DAC ... 0x0068D332 (len=1078663=0x107587)

   Loading Ramdisk to 1fef8000, end 1ffff587 ... OK

initrd_start = 1fef8000, initrd_end = 1ffff587

## Transferring control to Linux (at address 00000000) ...

tlbcam_index=11

mpc85xx_setup: Doing Pcie bridge setup

cpld_init: Map PCI RAS device

 

Installing Linux 2.6 Kernel

Attempting to find a root file system on hda2...

INIT: version 2.78 booting

Bypassing firmware validation.

--- Partition /dev/hda1 is inconsistent.

--- Its content will be restored to be the same as that of /dev/hda2.

--- Please check the version and re-load firmware if necessary after the system boots up.

INIT: Entering runlevel: 3

loading bonding module

Enabling FOS failure detection feature

enable fabos log

Starting Fabric OS Services...

FIPS off, RRD...

 

 

Enter trace_initialize

fablog module loaded with mtracer!

 

sys_chip_init: class 0 max 24 Done

ethsw_ha_enabled = 1

main-cpu in Neptune...

HASM enable auto-reboot

 

Detected 2 CPs in Chassis

Probe for PCI timeout detect

.

 

 

Fabric OS (dcx85108-fcs01)

 

 

dcx85108-fcs01 console login: admin

Password:

*****************************************************************

Logging into STANDBY CP, not all commands are fully supported !!

*****************************************************************

unknown(0):FID128:admin> slotshow

rbash: slotshow: command not found

unknown(0):FID128:admin> firmwareshow

Slot Name       Appl     Primary/Secondary Versions               Status

--------------------------------------------------------------------------

  7  CP1        FOS      v8.1.1                                   STANDBY *

                         Unknown

*  Local CP

 

WARNING: Firmwaredownload is in progress.

unknown(0):FID128:admin> Proxy: ip2ipc started (run=1, rc=0), debug=0

Proxy: ipc2ip started (rc=0)

HAM: HAM:ham_open_htbt:Open heartbeat: 3e0006

HAM: HAM:ham_open_htbt:Open heartbeat: 3f0107

Oct 16 17:05:36  Heartbeat to 3e0006 Up!

2017/10/16-17:05:35, [HAM-1004], 17076, SLOT 7 CHASSIS, INFO, Brocade_DCX, Processor rebooted - Reboot.

 

Exchange HA Sync between CP

 

 

 

-------------

 

OK let me see if I can console access for both.

 

I have tried hafailover but it seems second partition has problem.

 

dcx85108-fcs01:FID128:admin> firmwareshow

Slot Name       Appl     Primary/Secondary Versions               Status

--------------------------------------------------------------------------

  6  CP0        FOS      v8.1.1                                   STANDBY

                         v8.1.1

  7  CP1        FOS      v8.1.1                                   ACTIVE *

                         Unknown

*  Local CP

 

WARNING: Firmwaredownload is in progress.

dcx85108-fcs01:FID128:admin> fabricshow

Switch ID   Worldwide Name          Enet IP Addr    FC IP Addr      Name

-------------------------------------------------------------------------

  2: fffc02 10:00:c4:f5:7c:8c:20:60 10.228.184.171  0.0.0.0         "brcdg610-lfcs02"

  3: fffc03 50:00:53:35:65:98:4f:7e 0.0.0.0         0.0.0.0         "fcr_xd_3_2"

114: fffc72 10:00:00:05:1e:04:55:98 10.228.184.108  0.0.0.0        >"brcd5000-fcs02"

115: fffc73 10:00:00:05:1e:34:7e:9c 10.228.184.109  0.0.0.0         "brcd4100-fcs02"

116: fffc74 10:00:00:05:1e:44:32:00 10.228.184.110  0.0.0.0         "brcddcx-fcs02"

117: fffc75 10:00:00:05:1e:54:0b:44 10.228.184.113  0.0.0.0         "brcdmace-fcs01"

118: fffc76 10:00:00:05:1e:37:ac:30 10.228.184.114  0.0.0.0         "brcd7500-fcs02"

119: fffc77 10:00:00:05:1e:35:26:30 10.228.184.115  0.0.0.0         "brcd200e-fcs02"

120: fffc78 10:00:00:27:f8:62:b0:a0 10.228.184.231  0.0.0.0         "brcd6505-fcs03"

121: fffc79 10:00:00:05:33:56:9c:00 10.228.184.205  0.0.0.0         "dcx85108-fcs01"

122: fffc7a 10:00:00:05:33:56:59:00 10.228.184.208  0.0.0.0         "dcx85104-fcs01"

123: fffc7b 10:00:00:27:f8:3d:c7:f5 10.228.184.249  0.0.0.0         "brcd6520-fcs02"

125: fffc7d 10:00:50:eb:1a:f9:18:4c 10.228.185.167  0.0.0.0         "brcdG620-fcs02"

127: fffc7f 10:00:c4:f5:7c:2d:35:08 10.228.184.214  172.16.0.61     "brcdx6-4-fcs02"

160: fffca0 50:00:53:35:65:90:1e:7f 0.0.0.0         0.0.0.0         "fcr_fd_160"

 

The Fabric has 15 switches

 

 

dcx85108-fcs01:FID128:admin>

dcx85108-fcs01:FID128:admin> reboot

Warning: This command is being run on a control processor (CP)

based system and will cause the active CP to reboot.

This will cause disruption to all traffic in this chassis.

Are you sure you want to reboot the active CP [y/n]?y

2017/10/16-17:03:42, [RAS-1007], 17074, SLOT 7 CHASSIS, INFO, Brocade_DCX, System is about to reload.

Rebooting! Mon Oct 16 17:03:52 EDT 2017

 

Broadcast message from root (ttyS0) Mon Oct 16 17:03:52 2017...

 

The system is going down for reboot NOW !!

INIT: Switching to runlevel: 6

INIT: Sending processes the TERM signal

dcx85108-fcs01:FID128:admin> rsync: read errors mapping "/fabos/modules/pluto-zentron-module.ko": Input/output error (5)

Service instances out of sync

2017/10/16-17:03:55, [FSSM-1003], 17075, SLOT 7 CHASSIS, WARNING, Brocade_DCX, HA State out of sync.

eth0: Link is down at 31696881 jiffies

eth3: Link is down at 31696897 jiffies

bonding: bond0: released all slaves

eth1: Link is down at 31696920 jiffies

eth2: Link is down at 31696928 jiffies

##RSCMON exiting due to signal: 19, pending signals: 0x0, 0x0

##exiting due to signal: 19, pending signals: 0x0, 0x0

Stopping diagnostics

Unmounting all filesystems.

Please stand by while rebooting the system...

reboot_event: Calling bus0 flush

Restarting system.

.

 

The system is coming up, please wait...

 

 

U-Boot 1.1.3 (Jun  1 2017 - 11:38:13)

 

CPU:   8548_E, Version: 2.1, (0x80390021)

Core:  E500, Version: 2.2, (0x80210022)

Clock Configuration:

       CPU:1199 MHz, CCB: 399 MHz,

       DDR: 199 MHz, LBC:  49 MHz

L1:    D-cache 32 kB enabled

       I-cache 32 kB enabled

Board: Zentron

CPU Board Revision 255.198 (0xffc6)

    PCI2: disabled

I2C:   ready

DRAM:  initdram: Initializing DDRSDRAM

initdram: Normal boot in progress

dimm_spd_sdram 2001: memsize = 800

initdram: Call ddr_enable_ecc dram size -2147483648

    DDR: 2048 MB

POST RAM test disabled.

Adjusted memory 0x7d000000 2097152000

Now running in RAM - U-Boot at: 74f7d000

trap_init : 0x0

system inventory subsystem initialized

FLASH:  4 MB

L2 cache 512KB: enabled

CPLD: Init complete

ATA interface setup upm 256: ffcff00, ffcfc04, ffffc00, fffffc01

PCI: gd->brcd_flags = 0, PCI init

Skip our host bridge

        00  11  8086  b555  0680  1a

        00  12  8086  b555  0680  1a

        00  13  8086  b555  0680  1a

        00  14  1131  1561  0c03  3b

        00  14  1131  1562  0c03  3b

           CPLD rev: 0xd0

  CPU0 CPLD version: 0xba

In:    serial

Out:   serial

Err:   serial

Net:

ENET2: PHY is not applicable

ENET1: PHY is Broadcom BCM5461S 10/100/1000 BaseT PHY (2060c1)

ENET0: PHY is Broadcom BCM5461S 10/100/1000 BaseT PHY (2060c1)

ENET3: PHY is Broadcom BCM5461S 10/100/1000 BaseT PHY (2060c1)

 

board_init_r: Adjusted ram 2097152000 0x7d000000 2048000k

Checking system RAM - press any key to stop test

 

Checking memory address: 00100000

 

System RAM test using Default POST RAM Test succeeded.

 

set_bootstatus: BS_LOAD_OS, platform_idx = 2

Hit ESC to stop autoboot:  0

Map file at LBA sector 0x2b8100

do_bootm 171: load_addr 0x02000000 addr 0x00400000

do_bootm 174: ## Booting image at 00400000 load_addr 0x02000000 ...

   Image Name:   Linux-2.6.14.2

   Image Type:   PowerPC Linux Multi-File Image (gzip compressed)

   Data Size:    2675443 Bytes =  2.6 MB

   Load Address: 00000000

   Entry Point:  00000000

   Contents:

   Image 0:  1596767 Bytes =  1.5 MB

   Image 1:  1078663 Bytes =  1 MB

   Uncompressing Multi-File Image ... do_bootm_linux 565: ## Current stack ends at 0x74F5BB38 => set upper limit to 0x00800000

## initrd at 0x00585DAC ... 0x0068D332 (len=1078663=0x107587)

   Loading Ramdisk to 1fef8000, end 1ffff587 ... OK

initrd_start = 1fef8000, initrd_end = 1ffff587

## Transferring control to Linux (at address 00000000) ...

tlbcam_index=11

mpc85xx_setup: Doing Pcie bridge setup

cpld_init: Map PCI RAS device

 

Installing Linux 2.6 Kernel

Attempting to find a root file system on hda2...

INIT: version 2.78 booting

Bypassing firmware validation.

--- Partition /dev/hda1 is inconsistent.

--- Its content will be restored to be the same as that of /dev/hda2.

--- Please check the version and re-load firmware if necessary after the system boots up.

INIT: Entering runlevel: 3

loading bonding module

Enabling FOS failure detection feature

enable fabos log

Starting Fabric OS Services...

FIPS off, RRD...

 

 

Enter trace_initialize

fablog module loaded with mtracer!

 

sys_chip_init: class 0 max 24 Done

ethsw_ha_enabled = 1

main-cpu in Neptune...

HASM enable auto-reboot

 

Detected 2 CPs in Chassis

Probe for PCI timeout detect

.

 

 

Fabric OS (dcx85108-fcs01)

 

 

dcx85108-fcs01 console login: admin

Password:

*****************************************************************

Logging into STANDBY CP, not all commands are fully supported !!

*****************************************************************

unknown(0):FID128:admin> slotshow

rbash: slotshow: command not found

unknown(0):FID128:admin> firmwareshow

Slot Name       Appl     Primary/Secondary Versions               Status

--------------------------------------------------------------------------

 7  CP1        FOS      v8.1.1                                   STANDBY *

                         Unknown

*  Local CP

 

WARNING: Firmwaredownload is in progress.

unknown(0):FID128:admin> Proxy: ip2ipc started (run=1, rc=0), debug=0

Proxy: ipc2ip started (rc=0)

HAM: HAM:ham_open_htbt:Open heartbeat: 3e0006

HAM: HAM:ham_open_htbt:Open heartbeat: 3f0107

Oct 16 17:05:36  Heartbeat to 3e0006 Up!

2017/10/16-17:05:35, [HAM-1004], 17076, SLOT 7 CHASSIS, INFO, Brocade_DCX, Processor rebooted - Reboot.

 

Exchange HA Sync between CP

bootenv: Could not get requested variable bootargs.

2017/10/16-17:05:57, [FV-1001], 17077, SLOT 7 CHASSIS, INFO, dcx85108-fcs01, Flow Vision daemon initialized.

SNMP Research EMANATE/Lite Agent Version 16.2.0.9

Copyright 1989-2006 SNMP Research, Inc.

All service instances in sync

2017/10/16-17:06:31, [FSSM-1002], 17078, SLOT 7 CHASSIS, INFO, Brocade_DCX, HA State is in sync.

HAM: HAM:ham_notify:Firmware download failed.

Firmware commit operation has started to restore the secondary partition.

2017/10/16-17:06:31, [SULB-1003], 17079, SLOT 7 CHASSIS, INFO, Brocade_DCX, Firmwarecommit has started.

Validating the filesystem ...

 

unknown(0):FID128:admin>

unknown(0):FID128:admin>

unknown(0):FID128:admin> slotshow

rbash: slotshow: command not found

unknown(0):FID128:admin> exit

logout

 

 

Fabric OS (dcx85108-fcs01)

 

 

dcx85108-fcs01 console login: admin

Password:

*****************************************************************

Logging into STANDBY CP, not all commands are fully supported !!

*****************************************************************

dcx85108-fcs01:FID128:admin> firmwareshow

Slot Name       Appl     Primary/Secondary Versions               Status

--------------------------------------------------------------------------

  6  CP0        FOS      v8.1.1                                   ACTIVE

                         v8.1.1

  7  CP1        FOS      v8.1.1                                   STANDBY *

                         Unknown

*  Local CP

 

WARNING: Firmwaredownload is in progress.

dcx85108-fcs01:FID128:admin> firmwaredownloadstatus

[1]: Fri Jun 16 00:52:22 2017

Slot 6 (CP0, active): Firmware is being downloaded to standby CP. This step may take up to 30 minutes.

 

[2]: Fri Jun 16 00:59:17 2017

Slot 6 (CP0, active): Firmware has been downloaded successfully to Standby CP.

 

[3]: Fri Jun 16 00:59:21 2017

Slot 6 (CP0, active): Standby CP is going to reboot with new firmware.

 

[4]: Fri Jun 16 01:01:22 2017

Slot 6 (CP0, active): Standby CP booted successfully with new firmware.

 

[5]: Fri Jun 16 01:02:29 2017

Slot 7 (CP1, active): Forced failover succeeded. New Active CP is running new firmware

 

[6]: Fri Jun 16 01:03:13 2017

Slot 7 (CP1, active): Firmware is being downloaded to standby CP. This step may take up to 30 minutes.

 

[7]: Fri Jun 16 01:08:03 2017

Slot 7 (CP1, active): Firmware has been downloaded successfully on Standby CP.

 

[8]: Fri Jun 16 01:08:07 2017

Slot 7 (CP1, active): Standby CP reboots.

 

[9]: Fri Jun 16 01:10:04 2017

Slot 7 (CP1, active): Standby CP booted successfully with new firmware.

 

[10]: Fri Jun 16 01:10:04 2017

Slot 7 (CP1, active): Firmware commit operation has started on both active and standby CPs.

 

[11]: Fri Jun 16 01:10:05 2017

Slot 7 (CP1, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[12]: Fri Jun 16 01:13:53 2017

Slot 7 (CP1, active): The commit operation has completed successfully.

 

[13]: Fri Jun 16 01:13:53 2017

Slot 7 (CP1, active): Firmware commit operation has completed successfully on active CP.

 

[14]: Fri Jun 16 01:13:53 2017

Slot 7 (CP1, active): Firmwaredownload command has completed successfully. Use firmwareshow to verify the firmware versions.

 

[15]: Fri Jun 16 01:15:24 2017

Slot 6 (CP0, standby): The commit operation has completed successfully.

 

[16]: Wed Jun 21 02:40:48 2017

Slot 7 (CP1, standby): Firmware commit operation has started to restore the secondary partition.

 

[17]: Wed Jun 21 02:40:48 2017

Slot 7 (CP1, standby): The firmware commit operation has started. This may take up to 10 minutes.

 

[18]: Wed Jun 21 02:50:17 2017

Slot 7 (CP1, standby): Firmware commit failed - Firmwarecommit failed. (0x1a)

 

[19]: Thu Aug 10 08:52:36 2017

Slot 7 (CP1, active): Firmware commit operation has started to restore the secondary partition.

 

[20]: Thu Aug 10 08:52:37 2017

Slot 7 (CP1, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[21]: Thu Aug 10 09:57:13 2017

Slot 7 (CP1, standby): Firmware commit operation has started to restore the secondary partition.

 

[22]: Thu Aug 10 09:57:14 2017

Slot 7 (CP1, standby): The firmware commit operation has started. This may take up to 10 minutes.

 

[23]: Thu Aug 10 10:06:46 2017

Slot 7 (CP1, standby): Firmware commit failed - Firmwarecommit failed. (0x1a)

 

[24]: Thu Aug 10 10:58:02 2017

Slot 7 (CP1, active): Firmware commit operation has started to restore the secondary partition.

 

[25]: Thu Aug 10 10:58:03 2017

Slot 7 (CP1, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[26]: Thu Aug 10 12:02:10 2017

Slot 7 (CP1, standby): Firmware commit operation has started to restore the secondary partition.

 

[27]: Thu Aug 10 12:02:10 2017

Slot 7 (CP1, standby): The firmware commit operation has started. This may take up to 10 minutes.

 

[28]: Thu Aug 10 12:11:35 2017

Slot 7 (CP1, standby): Firmware commit failed - Firmwarecommit failed. (0x1a)

 

[29]: Thu Aug 10 13:03:41 2017

Slot 7 (CP1, active): Firmware commit operation has started to restore the secondary partition.

 

[30]: Thu Aug 10 13:03:41 2017

Slot 7 (CP1, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[31]: Thu Aug 10 13:14:19 2017

Slot 7 (CP1, active): Firmware commit failed - Firmwarecommit failed. (0x1a)

 

[32]: Fri Aug 11 04:18:46 2017

Slot 7 (CP1, standby): Firmware commit operation has started to restore the secondary partition.

 

[33]: Fri Aug 11 04:18:47 2017

Slot 7 (CP1, standby): The firmware commit operation has started. This may take up to 10 minutes.

 

[34]: Fri Aug 11 04:28:10 2017

Slot 7 (CP1, standby): Firmware commit failed - Firmwarecommit failed. (0x1a)

 

[35]: Fri Aug 11 05:19:43 2017

Slot 7 (CP1, active): Firmware commit operation has started to restore the secondary partition.

 

[36]: Fri Aug 11 05:19:45 2017

Slot 7 (CP1, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[37]: Fri Aug 11 05:30:16 2017

Slot 7 (CP1, active): Firmware commit failed - Firmwarecommit failed. (0x1a)

 

[38]: Fri Aug 11 06:24:25 2017

Slot 7 (CP1, standby): Firmware commit operation has started to restore the secondary partition.

 

[39]: Fri Aug 11 06:24:25 2017

Slot 7 (CP1, standby): The firmware commit operation has started. This may take up to 10 minutes.

 

[40]: Fri Aug 11 06:33:54 2017

Slot 7 (CP1, standby): Firmware commit failed - Firmwarecommit failed. (0x1a)

 

[41]: Fri Aug 11 07:24:20 2017

Slot 7 (CP1, active): Firmware commit operation has started to restore the secondary partition.

 

[42]: Fri Aug 11 07:24:21 2017

Slot 7 (CP1, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[43]: Fri Aug 11 07:34:55 2017

Slot 7 (CP1, active): Firmware commit failed - Firmwarecommit failed. (0x1a)

 

[44]: Fri Aug 11 08:29:20 2017

Slot 7 (CP1, standby): Firmware commit operation has started to restore the secondary partition.

 

[45]: Fri Aug 11 08:29:21 2017

Slot 7 (CP1, standby): The firmware commit operation has started. This may take up to 10 minutes.

 

[46]: Fri Aug 11 08:38:45 2017

Slot 7 (CP1, standby): Firmware commit failed - Firmwarecommit failed. (0x1a)

 

[47]: Wed Sep 20 07:57:40 2017

Slot 7 (CP1, active): Firmware commit operation has started to restore the secondary partition.

 

[48]: Wed Sep 20 07:57:40 2017

Slot 7 (CP1, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[49]: Wed Sep 20 08:08:10 2017

Slot 7 (CP1, active): Firmware commit failed - Firmwarecommit failed. (0x1a)

 

[50]: Wed Sep 20 09:28:14 2017

Slot 7 (CP1, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[51]: Wed Sep 20 09:37:23 2017

Slot 7 (CP1, active): Firmware commit failed - Firmwarecommit failed. (0x1a)

 

[52]: Wed Sep 20 16:22:59 2017

Slot 6 (CP0, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[53]: Wed Sep 20 16:22:59 2017

Slot 6 (CP0, active): Firmware commit failed - No need to commit firmware. (0x41)

 

[54]: Tue Sep 26 02:34:45 2017

Slot 6 (CP0, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[55]: Tue Sep 26 02:34:45 2017

Slot 6 (CP0, active): Firmware commit failed - No need to commit firmware. (0x41)

 

[56]: Tue Sep 26 02:36:40 2017

Slot 6 (CP0, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[57]: Tue Sep 26 02:36:40 2017

Slot 6 (CP0, active): Firmware commit failed - No need to commit firmware. (0x41)

 

[58]: Mon Oct 16 16:19:14 2017

Slot 6 (CP0, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[59]: Mon Oct 16 16:19:14 2017

Slot 6 (CP0, active): Firmware commit failed - No need to commit firmware. (0x41)

 

[60]: Mon Oct 16 16:48:48 2017

Slot 7 (CP1, active): Firmware commit operation has started to restore the secondary partition.

 

[61]: Mon Oct 16 16:48:49 2017

Slot 7 (CP1, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[62]: Mon Oct 16 16:58:12 2017

Slot 7 (CP1, active): Firmware commit failed - Firmwarecommit failed. (0x1a)

 

[63]: Mon Oct 16 17:01:55 2017

Slot 7 (CP1, active): Firmware commit operation has started to restore the secondary partition.

 

[64]: Mon Oct 16 17:01:55 2017

Slot 7 (CP1, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[65]: Mon Oct 16 17:06:31 2017

Slot 7 (CP1, standby): Firmware commit operation has started to restore the secondary partition.

 

[66]: Mon Oct 16 17:06:31 2017

Slot 7 (CP1, standby): The firmware commit operation has started. This may take up to 10 minutes.

 

dcx85108-fcs01:FID128:admin> Repairing the secondary partition now.

Please wait ...

rsync: read errors mapping "/boot/bootrom.bin": Input/output error (5)

 

 

 

-

------------

 

After reseated and powered on CP blade. Now it’s in the state below and firmware commit is keep failing.

 

WARNING: Firmwaredownload is in progress.

 

 

dcx85108-fcs01:FID128:root> firmwareshow

Slot Name       Appl     Primary/Secondary Versions               Status

--------------------------------------------------------------------------

  6  CP0        FOS      v8.1.1                                   ACTIVE *

                         v8.1.1

  7  CP1        FOS      v8.1.1                                   STANDBY

                         Unknown

*  Local CP

 

WARNING: Firmwaredownload is in progress.

dcx85108-fcs01:FID128:root>

 

 

 

[55]: Tue Sep 26 02:34:45 2017

Slot 6 (CP0, active): Firmware commit failed - No need to commit firmware. (0x41)

 

[56]: Tue Sep 26 02:36:40 2017

Slot 6 (CP0, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[57]: Tue Sep 26 02:36:40 2017

Slot 6 (CP0, active): Firmware commit failed - No need to commit firmware. (0x41)

 

[58]: Mon Oct 16 16:19:14 2017

Slot 6 (CP0, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[59]: Mon Oct 16 16:19:14 2017

Slot 6 (CP0, active): Firmware commit failed - No need to commit firmware. (0x41)

 

dcx85108-fcs01:FID128:root>

 

 

 

 

-

 

-----------------

 

Have you seen this? And any open issues?

I have requested for remove and reinsert to clear out any errors.

 

Provide any procedure or recovery steps.

 

 

we tried the suggested scenario but unable to disable the CP blade. It gives the below error message:

========================================

dcx85108-fcs01:FID128:admin> slotpoweroff 7
Not supported on this slot

dcx85108-fcs01:FID128:admin> bladedisable 7
Blade 7 is being disabled...Failed
Check if you typed a SW BLADE (Not CP) number and
it is within the range, executing 'slotshow' command on the switch.
Also, check if switch disabled or slot off or no blade,
or if the blade is running Diagnostics.

dcx85108-fcs01:FID128:admin> firmwareshow

Slot Name       Appl     Primary/Secondary Versions               Status

--------------------------------------------------------------------------

  6  CP0        FOS      v8.1.1                                   ACTIVE *

                         v8.1.1

  7  CP1        FOS      v8.1.1                                   STANDBY

                         Unknown

*  Local CP

 

WARNING: Firmwaredownload is in progress.

dcx85108-fcs01:FID128:admin>

 

dcx85108-fcs01:FID128:admin> slotshow

 

Slot   Blade Type     ID    Status

-----------------------------------

  1     SW BLADE     96     ENABLED

  2     UNKNOWN             VACANT

  3     SW BLADE    153     ENABLED

  4     UNKNOWN             VACANT

  5     CORE BLADE   98     ENABLED

  6     CP BLADE     50     ENABLED

  7     CP BLADE     50     ENABLED

  8     CORE BLADE   98     ENABLED

  9     UNKNOWN             VACANT

10     SW BLADE     97     ENABLED

11     UNKNOWN             VACANT

12     UNKNOWN             VACANT

 

dcx85108-fcs01:FID128:admin>

=======================================

 

I also tried doing firmwaresync but it also failed with below:

 

dcx85108-fcs01:FID128:admin> firmwaresync

 

This command will copy the firmware on the active CP blade to the

standby CP blade but will require that existing telnet, secure telnet or

SSH sessions to the standby CP blade to be restarted.

 

This command may take up to 20 minutes.

 

 

Do you want to continue (Y/N) [Y]: y

Another firmware operation is running now.

dcx85108-fcs01:FID128:admin>

 

I think to perform any update we will have to stop the ongoing firmware download process.

 

Can you look at this?

 

Here are the detail from switch:

======================================

dcx85108-fcs01:FID128:root> slotshow

 

Slot   Blade Type     ID    Status

-----------------------------------

  1     SW BLADE     96     ENABLED

  2     UNKNOWN             VACANT

  3     SW BLADE    153     ENABLED

  4     UNKNOWN             VACANT

  5     CORE BLADE   98     ENABLED

  6     CP BLADE     50     ENABLED

  7     CP BLADE     50     ENABLED

  8     CORE BLADE   98     ENABLED

  9     UNKNOWN             VACANT

10     SW BLADE     97     ENABLED

11     UNKNOWN             VACANT

12     UNKNOWN             VACANT

 

dcx85108-fcs01:FID128:root> firmwareshow

Slot Name       Appl     Primary/Secondary Versions               Status

--------------------------------------------------------------------------

  6  CP0        FOS      v8.1.1                                   STANDBY

                         v8.1.1

  7  CP1        FOS      v8.1.1                                   ACTIVE *

                         Unknown

*  Local CP

 

WARNING: Firmwaredownload is in progress.

 

dcx85108-fcs01:FID128:root> firmwarecheck

Validating integrity of firmware on root filesystem

Please wait...

..5....T   /fabos/modules/blaster-module.ko

..5....T   /fabos/lib/libtracedb.so.1.0

Firmware integrity check failed.

dcx85108-fcs01:FID128:root>

 

dcx85108-fcs01:FID128:admin> firmwaredownloadstatus

[47]: Wed Sep 20 07:57:40 2017

Slot 7 (CP1, active): Firmware commit operation has started to restore the secondary partition.

 

[48]: Wed Sep 20 07:57:40 2017

Slot 7 (CP1, active): The firmware commit operation has started. This may take up to 10 minutes.

 

[49]: Wed Sep 20 08:08:10 2017

Slot 7 (CP1, active): Firmware commit failed - Firmwarecommit failed. (0x1a)

 

dcx85108-fcs01:FID128:admin>

========================================

 

-Sunil

 

Brocade Moderator
Posts: 414
Registered: ‎03-29-2011

Re: Issues with DCX8510 CP blade's secondary partition reported after upgrade from 8.1.0 to 8.1.1

Hi Sunil,

 

look like firmware partitions on CP0 are corrupt - notice that from the below both read (of 8.1.1.) and writes are failing

 

rsync: read errors mapping "/tftpboot/SWBD88/kernel-debug-2.6.14.2-3282828861.ppc.rpm": Input/output error (5)

rsync: read errors mapping "/boot/bootrom.bin": Input/output error (5)

rsync: read errors mapping "/fabos/modules/pluto-zentron-module.ko": Input/output error (5)

rsync: read errors mapping "/tftpboot/SWBD88/kernel-debug-2.6.14.2-3282828861.ppc.rpm": Input/output error (5)

 

ERROR: boot/bootrom.bin failed verification -- update discarded.

ERROR: fabos/modules/pluto-zentron-module.ko failed verification -- update discarded.

ERROR: tftpboot/SWBD88/kernel-debug-2.6.14.2-3282828861.ppc.rpm failed verification -- update

 

during firmwarecommit.  As long as you have  a pending commit / firmware download (as determined by bootenv variables)

you will not be able to proceed.  The slotpoweroff works on core and port blades, but not on CP, BTW.  

Notice that firmwarecheck is also reporting errors - original install of 8.1.1 is probably not complete.

 

dcx85108-fcs01:FID128:root> firmwarecheck

Validating integrity of firmware on root filesystem

Please wait...

..5....T   /fabos/modules/blaster-module.ko

..5....T   /fabos/lib/libtracedb.so.1.0

Firmware integrity check failed.

dcx85108-fcs01:FID128:root>

 

I would open a support case for this to determine if the compact flash is really bad (swap out CP) or can be recovered. How full are the partitions, e.g. what does 'df' report as root?

 

 




If this provided you with a solution to this issue, please mark it with the button at the bottom "Accept as solution".


Any and all information provided by me is not reviewed, approved or endorsed by Brocade and is provided solely as a convenience for Brocade customers. All systems and all networks are different and unique. If you have a service affecting network problem, please open a TAC service request for service through Brocade, or through your OEM equipment provider. If this provided you with a solution to this issue, please mark it with the button at the bottom "Accept as solution"
Brocade Moderator
Posts: 61
Registered: ‎03-29-2010

Re: Issues with DCX8510 CP blade's secondary partition reported after upgrade from 8.1.0 to 8.1.1

Well, here's a last ditch try. You can use the firmwarecleaninstall but here's the rub. It is not as clean as the name implies.

 

First: Your dir WILL reboot. There is no way around it.

Next: Even if you do a configupload and then configdownload with the 'all' switch, it's unlikely that your logical switches will come back up right. You may have to disable VF, then enable it, then reload the config again.

Finally; Although this will clean up the file system, there is no guarantee that you aren't hitting a double bit error, or some other file system defect.

 

Proceed with caution if you choose to use firmwarecleaninstall.

doc

Any and all information provided by me is for entertainment value and should not be relied upon as a guaranteed solution or warranty of mechantability. All systems and all networks are different and unique. If you have a concern about data loss, or network disconnection, please open a TAC service request for service through Brocade, or through your OEM equipment provider. If this provided you with a solution to this issue, Please mark it with the button at the bottom "Accept as solution".

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.

vADC is now Pulse Secure
Download FREE NVMe eBook