Fibre Channel (SAN)

Reply
Occasional Contributor
Posts: 5
Registered: ‎12-14-2004

Lost connection with switch during fw-upgrade.

The switch was not able to load the fw, it tryed over and over again but the job was canceled. As this was an edge-switch and I had free ports in the core I took a chanse and booted the switch. Now I'm not able to connect to the switch, I am able to just about log in before I loose connection. The admin-port, (ethernet-port) is up for about 30 sec before I loose connection and the routine repeats itself. The switch is Brocade but labeld IBM B32, the current OS is 6.1.1a, I was trying to upgrade to 6.2.2 and then to 6.3.1a.

Is there a way to upgrade the code without the ethernet-port? Any other helpful hints on how to solve this?

Here is the output I'm able to get the seconds I'm logged in.

Fabric OS (IBM_2005_B32)
Fabos Version 6.1.1a


IBM_2005_B32 login: admin
Password:

Broadcast message from root Mon Nov 15 00:00:00 2010...

********************************************************************
Notice: System has changed state to active.
All active commands are available now.
********************************************************************

Super Contributor
Posts: 644
Registered: ‎03-01-2007

Re: Lost connection with switch during fw-upgrade.

Connectin lost, is most caused by Firewall behind between Switch and FTP Server, or wrong configured IP Address.

an you post here please the exact outptut from "firmwareshow" ?

Occasional Contributor
Posts: 5
Registered: ‎12-14-2004

Re: Lost connection with switch during fw-upgrade.

Hi

The switch was able to download the code from the ftp-server. The same as with the other switches I upgraded that night. The difference is that it had problems implementing the upgrade, thats when I booted the switch. Now I have problems accessing the switch for more than 30 sec before the ethernetport is going down, I do not have the time to run a firmwareshow command before I am thrown out. Something whent wrong during codeupgrade and booting the switch certainly did not help. So I am wondering if there is a way out of this mess? 

Occasional Contributor
Posts: 5
Registered: ‎12-14-2004

Re: Lost connection with switch during fw-upgrade.

Here is output from the switch connected via serial cabel. The switch loops with the same messages. What do I have to do to recover this switch?

Any ideas anyone?

Bjørnulf


loading kernel

loaded at:     01000000 01400364
board data at: 013FE324 013FE364
relocated to:  01005110 01005150
zimage at:     0100597D 01172C93
initrd at:     01173000 013FDC00
avail ram:     01401000 10000000

Linux/PPC load:
BootROM command line: quiet
Uncompressing Linux...done.
Now booting the kernel
PCI: Cannot allocate resource region 0 of device 0000:00:00.0
Installing Linux 2.6 Kernel
Attempting to find a root file system on hda1...
INIT: version 2.78 booting
Bypassing firmware validation.
--- Partition /dev/hda2 is inconsistent.
--- Its content will be restored to be the same as that of /dev/hda1.
--- Please check the version and re-load firmware if necessary after the system
boots up.
INIT: Entering runlevel: 3
rls_validate_file_3: 1024 indexes lost
uptime: 4293889640; sysc_qid: 0
2010/11/17-14:17:34, , 1,, INFO, IBM_2005_B32, Processor rebooted - Un
known

Fabric OS (IBM_2005_B32)


IBM_2005_B32 console login: SNMP Research EMANATE/Lite Agent Version 16.2.0.9
Copyright 1989-2006 SNMP Research, Inc.
sysctrld: all services Standby
2010/11/17-14:17:47, , 2,, INFO, FD1_SA_003, The effective configurat
ion has changed to NyGilde.
sec0: Security is initializing........
sysctrld: all services Active
kernel BUG in cache_grow at mm/slab.c:2217!
Oops: Exception in kernel mode, sig: 5
NIP: C0042840 LR: C0042118 SP: CB5D70B0 REGS: cb5d6ff0 TRAP: 0700  DBCR0:4000000
0 Tainted: P
simple_pd_save done!
PowerPC Book-E Watchdog Exception
MSR: 00021000 EE: 0 PR: 0 FP: 0 ME: 1 IR/DR: 00
TASK = cb539bd0 'emd0' THREAD: cb5d4000
Last syscall: 54
GPR00: FFE8800F CB5D70B0 CB539BD0 C031B6C0 C13FF4A0 00000000 0000000C CB5D712C
GPR08: C13FF4B0 00000001 C13FE220 3EC00B60 00200200 1073B86C 00000000 00000007
GPR16: CE8A15E0 CB5D7A98 CB5D7A88 D2CE0000 D2CBC6FC D2CB0000 D2CC0000 D2CC0000
GPR24: 0000000C 000000D1 00000001 C13FF4B0 C13FF4A8 00100100 C031B6C0 C13FE220
NIP cache_alloc_refill+0x244/0x510
LR __kmalloc+0xb4/0xb8
Call trace:
cb5d70e0 __kmalloc+0xb4/0xb8
cb5d70f0 kmalloc_wrapper_dbg+0x4c/0x26c
cb5d7110 condor_dma_instantiate+0xbc/0x288
cb5d7170 condor_chip_instantiate+0x1e8/0x448
cb5d7280 pulsar_chip_create+0x23c/0x554
cb5d7320 blade_hierarchy_instantiate+0xe00/0x172c
cb5d7410 pulsar_inst+0x22c/0x5a4
cb5d74f0 fabsys_blade_instantiate+0x48/0x64
cb5d7510 fabsys_blade_init+0x50/0x47c
cb5d7540 _rdy+0x500/0xf9c
cb5d7630 fsm_controller+0x238/0x2fc
cb5d7670 sys_fsm_dft_ctrl+0x1d4/0x5a4
cb5d7720 slot_dft_class_hndlr+0x70/0xe0
cb5d7770 slot_class_hndlr+0x10c/0x294
cb5d7800 sysScnProcState+0x70/0x14c
cb5d7840 sysCtrlProcCmd+0x4ccc/0x609c
cb5d7ec0 fabsys_ioctl+0x50/0x94
cb5d7ed0 do_ioctl+0x68/0x9c
cb5d7ee0 vfs_ioctl+0xb8/0x400
cb5d7f00 sys_ioctl+0x40/0x74
cb5d7f30 ret_from_syscall+0x0/0x48

hda: task_out_intr: status=0x50 { DriveReady SeekComplete }
ide: failed opcode was: unknown
PD Start
MTRACER
mtracer_panicdump
mtracer_panicdump write=0x3c3b
reboot_reason
set_reboot_reason reason=Software Fault:Kernel Panic
PD_MISC
CONSOLE_LOG
KERNEL_STACK_DUMP
PLATFORM
pl.0
PANIC_DUMP_LOG
PD Time: 14.25745 Seconds
PD Completed

The system is coming up, please wait...
Read board ID of 0x80 from addr 0x23
Read extended model ID of 0x16 from addr 0x22
Matched board/model ID to platform index 4

Read board ID of 0x80 from addr 0x23
Read extended model ID of 0x16 from addr 0x22
Matched board/model ID to platform index 4
Checking system RAM - press any key to stop test

Checking memory address: 00100000

System RAM test using Default POST RAM Test succeeded.

Press escape within 4 seconds to enter boot interface.
Booting "Fabric Operating System" image.

loading kernel

loaded at:     01000000 01400364
board data at: 013FE324 013FE364
relocated to:  01005110 01005150
zimage at:     0100597D 01172C93
initrd at:     01173000 013FDC00
avail ram:     01401000 10000000

Linux/PPC load:
BootROM command line: quiet
Uncompressing Linux...done.
Now booting the kernel
PCI: Cannot allocate resource region 0 of device 0000:00:00.0
Installing Linux 2.6 Kernel
Attempting to find a root file system on hda1...
INIT: version 2.78 booting
Bypassing firmware validation.
--- Partition /dev/hda2 is inconsistent.
--- Its content will be restored to be the same as that of /dev/hda1.
--- Please check the version and re-load firmware if necessary after the system
boots up.
INIT: Entering runlevel: 3
pdcheck: info: found new pd: mtd_ts=1290003472 DIE Wed Nov 17 14:17:52 2010, cf_
ts=1290003373
uptime: 4293889785; sysc_qid: 0
2010/11/17-14:18:36, , 3,, INFO, IBM_2005_B32, Processor rebooted - So
ftware Fault:Kernel Panic

SNMP Research EMANATE/Lite Agent Version 16.2.0.9
Copyright 1989-2006 SNMP Research, Inc.


Fabric OS (IBM_2005_B32)


IBM_2005_B32 console login: sysctrld: all services Standby
2010/11/17-14:18:47, , 4,, INFO, FD1_SA_003, The effective configurat
ion has changed to NyGilde.
sec0: Security is initializing........
sysctrld: all services Active
kernel BUG in cache_grow at mm/slab.c:2217!
Oops: Exception in kernel mode, sig: 5
NIP: C0042840 LR: C0042118 SP: CB09F0B0 REGS: cb09eff0 TRAP: 0700  DBCR0:4000000
0 Tainted: P
simple_pd_save done!
PowerPC Book-E Watchdog Exception
MSR: 00021000 EE: 0 PR: 0 FP: 0 ME: 1 IR/DR: 00
TASK = cb7c34b0 'emd0' THREAD: cb09c000
Last syscall: 54
GPR00: FFE8800F CB09F0B0 CB7C34B0 C031B6C0 C13FF4A0 00000000 0000000C CB09F12C
GPR08: C13FF4B0 00000001 C13FE220 3EC00B60 00200200 1073B86C 00000000 00000007
GPR16: CEF322E0 CB09FA98 CB09FA88 D2CE0000 D2CBC6FC D2CB0000 D2CC0000 D2CC0000
GPR24: 0000000C 000000D1 00000001 C13FF4B0 C13FF4A8 00100100 C031B6C0 C13FE220
NIP cache_alloc_refill+0x244/0x510
LR __kmalloc+0xb4/0xb8
Call trace:
cb09f0e0 __kmalloc+0xb4/0xb8
cb09f0f0 kmalloc_wrapper_dbg+0x4c/0x26c
cb09f110 condor_dma_instantiate+0xbc/0x288
cb09f170 condor_chip_instantiate+0x1e8/0x448
cb09f280 pulsar_chip_create+0x23c/0x554
cb09f320 blade_hierarchy_instantiate+0xe00/0x172c
cb09f410 pulsar_inst+0x22c/0x5a4
cb09f4f0 fabsys_blade_instantiate+0x48/0x64
cb09f510 fabsys_blade_init+0x50/0x47c
cb09f540 _rdy+0x500/0xf9c
cb09f630 fsm_controller+0x238/0x2fc
cb09f670 sys_fsm_dft_ctrl+0x1d4/0x5a4
cb09f720 slot_dft_class_hndlr+0x70/0xe0
cb09f770 slot_class_hndlr+0x10c/0x294
cb09f800 sysScnProcState+0x70/0x14c
cb09f840 sysCtrlProcCmd+0x4ccc/0x609c
cb09fec0 fabsys_ioctl+0x50/0x94
cb09fed0 do_ioctl+0x68/0x9c
cb09fee0 vfs_ioctl+0xb8/0x400
cb09ff00 sys_ioctl+0x40/0x74
cb09ff30 ret_from_syscall+0x0/0x48

hda: task_out_intr: status=0x50 { DriveReady SeekComplete }
ide: failed opcode was: unknown
PD Start
MTRACER
mtracer_panicdump
mtracer_panicdump write=0x3bcc
reboot_reason
set_reboot_reason reason=Software Fault:Kernel Panic
PD_MISC
CONSOLE_LOG
KERNEL_STACK_DUMP
PLATFORM
pl.0
PANIC_DUMP_LOG
PD Time: 14.27619 Seconds
PD Completed

The system is coming up, please wait...
Read board ID of 0x80 from addr 0x23
Read extended model ID of 0x16 from addr 0x22
Matched board/model ID to platform index 4

Read board ID of 0x80 from addr 0x23
Read extended model ID of 0x16 from addr 0x22
Matched board/model ID to platform index 4
Checking system RAM - press any key to stop test

Checking memory address: 00100000

System RAM test using Default POST RAM Test succeeded.

Press escape within 4 seconds to enter boot interface.

1) Start system.
2) Recover password.
3) Enter command shell.

Option? 1
Booting "Fabric Operating System" image.

loading kernel

loaded at:     01000000 01400364
board data at: 013FE324 013FE364
relocated to:  01005110 01005150
zimage at:     0100597D 01172C93
initrd at:     01173000 013FDC00
avail ram:     01401000 10000000

Linux/PPC load:
BootROM command line: quiet
Uncompressing Linux...done.
Now booting the kernel
PCI: Cannot allocate resource region 0 of device 0000:00:00.0
Installing Linux 2.6 Kernel
Attempting to find a root file system on hda1...
INIT: version 2.78 booting
Bypassing firmware validation.
--- Partition /dev/hda2 is inconsistent.
--- Its content will be restored to be the same as that of /dev/hda1.
--- Please check the version and re-load firmware if necessary after the system
boots up.
INIT: Entering runlevel: 3
pdcheck: info: found new pd: mtd_ts=1290003531 DIE Wed Nov 17 14:18:51 2010, cf_
ts=1290003472
rls_validate_file_3: 1024 indexes lost
uptime: 4293889921; sysc_qid: 0
2010/11/17-14:19:43, , 1,, INFO, IBM_2005_B32, Processor rebooted - Un
known

Fabric OS (IBM_2005_B32)


IBM_2005_B32 console login: SNMP Research EMANATE/Lite Agent Version 16.2.0.9
Copyright 1989-2006 SNMP Research, Inc.
sysctrld: all services Standby
2010/11/17-14:19:55, , 2,, INFO, FD1_SA_003, The effective configurat
ion has changed to NyGilde.
sec0: Security is initializing........
sysctrld: all services Active
kernel BUG in cache_grow at mm/slab.c:2217!
Oops: Exception in kernel mode, sig: 5
NIP: C0042840 LR: C0042118 SP: CB2CB0B0 REGS: cb2caff0 TRAP: 0700  DBCR0:4000000
0 Tainted: P
MSR: 00021000 EE: 0 PR: 0 FP: 0 ME: 1 IR/DR: 00
TASK = cb160150 'emd0' THREAD: cb2c8000
Last syscall: 54
GPR00: FFE8800F CB2CB0B0 CB160150 C031B6C0 C13FF4A0 00000000 0000000C CB2CB12C
GPR08: C13FF4B0 00000001 C13FE220 3EC00B60 00200200 1073B86C 00000000 00000007
GPR16: CEE443E0 CB2CBA98 CB2CBA88 D2CE0000 D2CBC6FC D2CB0000 D2CC0000 D2CC0000
GPR24: 0000000C 000000D1 00000001 C13FF4B0 C13FF4A8 00100100 C031B6C0 C13FE220
NIP cache_alloc_refill+0x244/0x510
LR __kmalloc+0xb4/0xb8
Call trace:
cb2cb0e0 __kmalloc+0xb4/0xb8
cb2cb0f0 kmalloc_wrapper_dbg+0x4c/0x26c
cb2cb110 condor_dma_instantiate+0xbc/0x288
cb2cb170 condor_chip_instantiate+0x1e8/0x448
cb2cb280 pulsar_chip_create+0x23c/0x554
cb2cb320 blade_hierarchy_instantiate+0xe00/0x172c
cb2cb410 pulsar_inst+0x22c/0x5a4
cb2cb4f0 fabsys_blade_instantiate+0x48/0x64
cb2cb510 fabsys_blade_init+0x50/0x47c
cb2cb540 _rdy+0x500/0xf9c
cb2cb630 fsm_controller+0x238/0x2fc
cb2cb670 sys_fsm_dft_ctrl+0x1d4/0x5a4
cb2cb720 slot_dft_class_hndlr+0x70/0xe0
cb2cb770 slot_class_hndlr+0x10c/0x294
cb2cb800 sysScnProcState+0x70/0x14c
cb2cb840 sysCtrlProcCmd+0x4ccc/0x609c
cb2cbec0 fabsys_ioctl+0x50/0x94
cb2cbed0 do_ioctl+0x68/0x9c
cb2cbee0 vfs_ioctl+0xb8/0x400
cb2cbf00 sys_ioctl+0x40/0x74
cb2cbf30 ret_from_syscall+0x0/0x48

PD Start
MTRACER
mtracer_panicdump
mtracer_panicdump write=0x3c3e
reboot_reason
set_reboot_reason reason=Software Fault:Kernel Panic
PD_MISC
CONSOLE_LOG
KERNEL_STACK_DUMP
PLATFORM
pl.0
PANIC_DUMP_LOG
PD Time: 14.26530 Seconds
PD Completed

The system is coming up, please wait...
Read board ID of 0x80 from addr 0x23
Read extended model ID of 0x16 from addr 0x22
Matched board/model ID to platform index 4

Read board ID of 0x80 from addr 0x23
Read extended model ID of 0x16 from addr 0x22
Matched board/model ID to platform index 4
Checking system RAM - press any key to stop test

Checking memory address: 00100000

System RAM test using Default POST RAM Test succeeded.

Press escape within 4 seconds to enter boot interface.
Booting "Fabric Operating System" image.

loading kernel

loaded at:     01000000 01400364
board data at: 013FE324 013FE364
relocated to:  01005110 01005150
zimage at:     0100597D 01172C93
initrd at:     01173000 013FDC00
avail ram:     01401000 10000000

Linux/PPC load:
BootROM command line: quiet
Uncompressing Linux...done.
Now booting the kernel
PCI: Cannot allocate resource region 0 of device 0000:00:00.0
Installing Linux 2.6 Kernel
Attempting to find a root file system on hda1...
INIT: version 2.78 booting
Bypassing firmware validation.
--- Partition /dev/hda2 is inconsistent.
--- Its content will be restored to be the same as that of /dev/hda1.
--- Please check the version and re-load firmware if necessary after the system
boots up.
INIT: Entering runlevel: 3

Contributor
Posts: 33
Registered: ‎04-24-2010

Re: Lost connection with switch during fw-upgrade.

kernel BUG in cache_grow at mm/slab.c:2217!
Oops: Exception in kernel mode, sig: 5

do you check compact flash usage for this switch before upgrade? you can try to perform supportshow to view this information(bin/df).

if compact flash usage is above 90%, please use savecore  command to clear excess core files.

Super Contributor
Posts: 644
Registered: ‎03-01-2007

Re: Lost connection with switch during fw-upgrade.

--->>>....please use savecore  command

this command is only available and supported by FOS till 5.2.x

In FOS 5.3; and all 6.x is this command not avalable

"supportftp" is supported.

Contributor
Posts: 33
Registered: ‎04-24-2010

Re: Lost connection with switch during fw-upgrade.

ABBA-SYSTEMS,you are right

bjornulf, i would suggest you try to run this command.

hope this helps.

N/A
Posts: 1
Registered: ‎06-13-2008

Re: Lost connection with switch during fw-upgrade.

Hi bjonulf ,

To examine a panic dump file you can run:

pdshow dump_file

...without arguments , it will display the latest dulp file

Based on the messages, you should have one:

....

PD_MISC
CONSOLE_LOG
KERNEL_STACK_DUMP
PLATFORM
pl.0
PANIC_DUMP_LOG
PD Time: 14.25745 Seconds
PD Completed

....

simple_pd_save done!

....

It might be interesting to see it.

During A firmware upgrade (firmwaredownload) the firmware is downloaded/installed on the secondary partition.

In docs is written(check the firmwareDownload command in FOS Command Reference Manual):

If firmwareDownload is interrupted due to an unexpected reboot as a result of a software error or
power failure, the command automatically recovers the corrupted secondary partition. Wait for the
recovery to complete before starting another firmwareDownload

In your logs is written:

Bypassing firmware validation.
--- Partition /dev/hda2 is inconsistent.
--- Its content will be restored to be the same as that of /dev/hda1.
--- Please check the version and re-load firmware if necessary after the system
boots up.

Can you run the command:

version

to see which firmware versions do you have on switch ?

If you still see FOS 6.1.1a, you can try to run firmwareRestore

radu.

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.