Fibre Channel (SAN)

Reply
New Contributor
Posts: 2
Registered: ‎01-10-2012

Setting bfa modprobe.conf

Hello,

I have Brocade 815 HBA on RHEL 5.6. I am seeing very slow IO performance in my Oracle Database. I see that everythign is set to 8GB  to match the switch and the DS8800 SAN with 160 disks.

I am trying to eet some parameters on the bfa modules. If I do:

modinfe bfa

<snip>

parm:           max_xfer_size:default=32MB, Range (int)

I see the parameter max_xfer_size.

$> cat /sys/module/bfa/parameters/max_xfer_size
32767

The value returned is not 32MB. Infact it is 1 byte short of the 32k default that I would iferr from the Range values offered.

Is this parameter modifyable in RHEL, of so what is the correct value? Are there other paramters that could help with my IO problem?

Thanks,

Casey

Contributor
Posts: 36
Registered: ‎11-06-2008

Re: Setting bfa modprobe.conf

Casey,

Please try this noop scheduler as stated below for other parameter:

=================================================================================================

Linux tuning

Linux disk I/O scheduling reorders, delays, and merges requests to achieve better throughput and lower latency than would happen if all the requests were sent straight to the disk. Linux 2.6 has four different disk I/O schedulers: noop, deadline, anticipatory and completely fair queuing. Enabling the “noop” scheduler avoids any delays in queuing of I/O commands. This helps in achieving higher I/O rates by queuing multiple outstanding I/O requests to each disk.

To enable the noop scheduler, run the following commands on your system.

for i in /sys/block/sd/queue/scheduler

do

echo noop > $i

done

NOTE

You must disable the default scheduler because it is not tuned for achieving the maximum I/O

performance.

=================================================================================================

I beleived the default is 32K and not 32MB since the max is 2048K=2MB

max_xfer_size:default=32MB, Range (int)

New Contributor
Posts: 2
Registered: ‎01-10-2012

Re: Setting bfa modprobe.conf

I changed the default io scheduler in grub.conf to evalator=noop and still an seeing poor io performance.

I checked that the io scheduler is changed for the SAN disks.

cat /sys/block/sdaa/queue/scheduler
anticipatory deadline cfq

I am trying to get the max_xfer_size. Are the values xxxK - 1? Should i have to rebuild the initrd to get the values to set?

Thanks
Casey

Contributor
Posts: 36
Registered: ‎11-06-2008

Re: Setting bfa modprobe.conf

Casey,

According to our engineering:

Max_xfer_size (basically define the size of one IO)  defaults to 32M in bfa driver linux, but the linux OS can only send down 512k IO, so changing it between 32M ~512k will have no effect on performance. Lowering max_xfer_size will typically lower down throughput.

pcie-max-read-reqsz:   Changes are typically not needed on majority of the server. We have only seen one Sparc machine in the past not providing optimal PCI settings.   This change will help increase I/O write performance if the value was previously not configured by the server properly.

You might need to reboot the server but I'm not certain that you need to rebuild your initrd.

Could you please provide me the following and send them to me via my email hvu@brocade.com:

  1. Throughput experienced
  2. Expected throughput
  3. OS version – Oracle RAC
  4. Supportsave logs for the HBAs - I can help you obtain this support save log file for the Brocade adapter
Frequent Contributor
Posts: 90
Registered: ‎12-26-2010

Re: Setting bfa modprobe.conf

changing the system paramater or modifying the grub  does not require running the initrd for the change you mentioned.  Check the messages file and if required it will report an error/warning there

Frequent Contributor
Posts: 90
Registered: ‎12-26-2010

Re: Setting bfa modprobe.conf

most of the array have prefetching and Write caching feature which is  typically   higher than 512K described there; So i am just wondering how exactly the OS limit 512K described is fit in this situation .  Can you also share your reference to this( i am sure you have one), i am just curious to look at it

Join the Community

Get quick and easy access to valuable resource designed to help you manage your Brocade Network.

Click to Register
Download FREE NVMe eBook