01-10-2012 03:37 PM
I have Brocade 815 HBA on RHEL 5.6. I am seeing very slow IO performance in my Oracle Database. I see that everythign is set to 8GB to match the switch and the DS8800 SAN with 160 disks.
I am trying to eet some parameters on the bfa modules. If I do:
parm: max_xfer_size:default=32MB, Range (int)
I see the parameter max_xfer_size.
$> cat /sys/module/bfa/parameters/max_xfer_size
The value returned is not 32MB. Infact it is 1 byte short of the 32k default that I would iferr from the Range values offered.
Is this parameter modifyable in RHEL, of so what is the correct value? Are there other paramters that could help with my IO problem?
01-10-2012 04:26 PM
Please try this noop scheduler as stated below for other parameter:
Linux disk I/O scheduling reorders, delays, and merges requests to achieve better throughput and lower latency than would happen if all the requests were sent straight to the disk. Linux 2.6 has four different disk I/O schedulers: noop, deadline, anticipatory and completely fair queuing. Enabling the “noop” scheduler avoids any delays in queuing of I/O commands. This helps in achieving higher I/O rates by queuing multiple outstanding I/O requests to each disk.
To enable the noop scheduler, run the following commands on your system.
for i in /sys/block/sd/queue/scheduler
echo noop > $i
You must disable the default scheduler because it is not tuned for achieving the maximum I/O
I beleived the default is 32K and not 32MB since the max is 2048K=2MB
max_xfer_size:default=32MB, Range (int)
01-11-2012 12:41 PM
I changed the default io scheduler in grub.conf to evalator=noop and still an seeing poor io performance.
I checked that the io scheduler is changed for the SAN disks.
anticipatory deadline cfq
I am trying to get the max_xfer_size. Are the values xxxK - 1? Should i have to rebuild the initrd to get the values to set?
01-11-2012 01:12 PM
According to our engineering:
Max_xfer_size (basically define the size of one IO) defaults to 32M in bfa driver linux, but the linux OS can only send down 512k IO, so changing it between 32M ~512k will have no effect on performance. Lowering max_xfer_size will typically lower down throughput.
pcie-max-read-reqsz: Changes are typically not needed on majority of the server. We have only seen one Sparc machine in the past not providing optimal PCI settings. This change will help increase I/O write performance if the value was previously not configured by the server properly.
You might need to reboot the server but I'm not certain that you need to rebuild your initrd.
Could you please provide me the following and send them to me via my email email@example.com:
02-12-2012 06:07 PM
changing the system paramater or modifying the grub does not require running the initrd for the change you mentioned. Check the messages file and if required it will report an error/warning there
02-12-2012 06:17 PM
most of the array have prefetching and Write caching feature which is typically higher than 512K described there; So i am just wondering how exactly the OS limit 512K described is fit in this situation . Can you also share your reference to this( i am sure you have one), i am just curious to look at it