Previously I wrote an article on How to install and run Openfiler on a USB key. I thought that everything was working fine but eventually found that NFS and CIFS performance was too slow. Upon reading a few forums and stumbling across this thread in particular, the reason was down to Openfiler requiring an update.
I have since tried to update the installation by running conary updateall at the CLI. Unfortunately, this installs an updated kernel (184.108.40.206-0.24.smp.gcc3.4.x86_64 (SMP)) and also a new ramdisk which makes all the hard work from the previous post defunct. This article shows you how to perform the update and then make a new initrd-usb-update.img to work with the new kernel.
So assuming you’ve made a successful USB key using the previous article, continue with the following to update your Openfiler installation and also make the updated Openfiler installation USB key bootable.
Let’s first update Openfiler.
- Log into the CLI as root
- This will take a while as it downloads around 26 packages and installs them.
- Once complete insert the Openfiler CD into your drive and restart your system, making sure that it boots from CD.
Creating a new ramdisk that works with the new kernel
This part is more or less very much similar to the steps in the previous post, there are some minor additions that we need to make, but for completeness I’ve included all the steps here.
Once Openfiler finishes booting from the CD type.
- Go through the menus and select your region and keyboard and skip the automatic mounting of your installed OS. We will do this manually.
When the prompt appears, create a directory to mount the USB key.
Now mount the / of the USB key onto /mnt/sysimage.
mount /dev/sda2 /mnt/sysimage
Note: your / partition may be /dev/sda3 instead, depending on how you setup your partitioning during the installation of Openfiler.
Now mount the boot partition of the USB key onto /mnt/sysimage/boot.
mount /dev/sda1 /mnt/sysimage
Note: your / partition may be /dev/sda1 instead, depending on how you setup your partitioning during the installation of Openfiler.
Make the /mnt/sysimage your working environment by changing your root location so you are working on the file system on the usb key.
Copy the current initrd file to a temporary location where we can work on it.
cp /boot/initrd-1 /tmp/initrd.gz
Note1: now’s a good time to press TAB, there will now be two kernels, use 220.127.116.11-0.24.smp.gcc3.4.x86_64 as this is the updated kernel that was installed during the update.
Gunzip the initrd.gz file
Make a temporary working directory
We are using /tmp/b because /tmp/a already exists as the temporary working directory from the previous article.
Go into the new working directory
Extract the contents of the initrd file into /tmp/b.
cpio –i < /tmp/initrd
Now we edit the init file to load the USB and SCSI drivers for the new initrd-usb-update.img ramdisk.
Do a search for “mount –t proc /proc /proc” and add the following underneath this line. We want to load these USB and storage modules before any other modules that’s why these entries need to be at the top of the file. Note that there is a new module crc-t10dif.ko which is required by the new kernel to boot from USB and as such must be launch during init time.
echo “Starting Openfiler on USB”
echo “Loading scsi_mod.ko module”
echo “Starting crc-t10dif.ko module”
echo “Loading sd_mod.ko module”
echo “Loading sr_mod.ko module”
echo “Loading ehci-hcd.ko module”
echo “Loading uhci-hcd.ko module”
echo “Loading ohci-hcd.ko module”
echo “Loading usb-storage.ko module”
- Do a search for insmod /lib/scsi_mod.ko, insmod /lib/sd_mod.ko, insmod /lib/ehci-hcd.ko, insmod /lib/uhci-hcd.ko and /lib/crc-t10dif.ko and remove these duplicate entries from the rest of the file. We do not want these loaded again.
- Save the file and exit with CTRL X, then Y, or if you used vi then :wq!
- Now we need to copy all of the modules in Step 13 into our working directory.
Go to the drivers directory
Note2: just press tab to fill in this bit, there will now be two kernels, use 18.104.22.168-0.24.smp.gcc3.4.x86_64 as this is the updated kernel that was installed during the update.
Copy all of the modules in Step 13 to /tmp/b/lib.
cp scsi/scsi_mod.ko /tmp/b/lib
cp scsi/sr_mod.ko /tmp/b/lib
cp scsi/sd_mod.ko /tmp/b/lib
cp usb/host/ehci-hcd.ko /tmp/b/lib
cp usb/host/uhci-hcd.ko /tmp/b/lib
cp usb/host/ohci-hcd.ko /tmp/b/lib
cp usb/storage/usb-storage.ko /tmp/b/lib
cp /lib/modules/22.214.171.124-0.24.smp.gcc3.4.x86_64/kernel/lib/crc-t10dif.ko /tmp/b/lib
Now let’s package the contents of the working directory /tmp/b into our new initrd-usb-update.img.
find . | cpio –c –o | gzip -9 > /boot/initrd-usb-update.img
Now all we need to do is edit the /boot/grub/menu.1st file to tell the kernel to use the new ramdisk that we just created. Remember that this new ramdisk is currently located in /boot/ (aka /dev/sda1) and is called initrd-usb-update.img.
Find the line starting with initrd /vmlinux-…………………… and replace with
- Save the file and reboot the computer, remove the CD and allowing it to boot from the USB key. You now have your new updated Openfiler installation booting from the USB key directly.
Turn off flow control for SMB Clients
For better CIFS performance turn off your network adapter flow control. I can achieve a sustained 60 mb/s transfer between my Macbook and Openfiler once flow control is turned off. I was only achieving around 30 mb/s previously.
Turn off flow control for ESXi hosts using NFS/iSCSI to Openfiler
First understand what flow control is before performing the follow actions, the following articles provide good cases for either enabling or disabling flow control and auto-negotiation for flow control.
http://www.telecom.otago.ac.nz/tele301/student_html/ethernet-autonegotiation-flow-control.html – not to be confused with auto-negotiation of flow control.
Since this is my lab I’m going to disable flow control completely.
To do this on ESXi hosts follow these instructions or use VMware KB 1013413.
- Enable Remote SSH for the ESXi host first.
- Use your favourite SSH client and log in as root (assuming you can, disable lock-down mode etc).
Run the following command to list all your vmnic interfaces, make a note of the vmnic that is used to connect to the Openfiler Server.
In my case it’s just vmnic0 (I’m using a HP Microserver), type the following command to see the current flow control status of that adapter.
ethtool –show-pause vmnic0
- Run the following commands to set auto-negotiation or RX flow control or TX flow control, any combination is possible.
To disable flow control for sent and received traffic, use the command:
ethtool –pause tx off rx off
To disable auto-negotiation of flow control, use the command:
ethtool –pause autoneg off
- Open the /etc/rc.local file using a text editor and append the same commands used in Step 6, placing each on its own line. Then save the file.
For an ESXi host, save the configuration change using the command:
The commands added to the /etc/rc.local file will be executed at startup, persisting the configuration changes across reboots. As they are executed in Step 6, no reboot is required for them to take effect.
5 thoughts on “How to update your Openfiler USB installation for better performance”
Can I be the worst freetard ever and ask you to put your “openfiler on a stick” iso available ?
I find myself without a cd drive, but perfectly able to put an iso on a usb… and you just made a perfectly working one with the latest version 8)
Thanks for reading my blog.
Do you mean make an image of my openfiler box usb key? It will be around 512mb compressed zip file, when extracted it will be a 2Gb .ima file and you will need a 2Gb USB stick in order to write the image to. Not sure how useful this will be for you as it was built on a Dell PE SC440. Let me know if you still want it and I’ll make it available.
I would be glad to have it and try.
At worst it is lost time and bandwidth, at best an OF on a key solution 8)
I’ll make it available this weekend.
You sir are a life saver. My openfiler would not load after i did the conary update all and I wasnt able to load my old config for 2.3.26. Your step by step was so well written that my crisis was averted in under 15 mins. Thanks for posting such a great article.