Networking

How to update your Openfiler USB installation for better performance

Previously I wrote an article on How to install and run Openfiler on a USB key. I thought that everything was working fine but eventually found that NFS and CIFS performance was too slow. Upon reading a few forums and stumbling across this thread in particular, the reason was down to Openfiler requiring an update.

I have since tried to update the installation by running conary updateall at the CLI. Unfortunately, this installs an updated kernel (2.6.29.6-0.24.smp.gcc3.4.x86_64 (SMP)) and also a new ramdisk which makes all the hard work from the previous post defunct. This article shows you how to perform the update and then make a new initrd-usb-update.img to work with the new kernel.

So assuming you’ve made a successful USB key using the previous article, continue with the following to update your Openfiler installation and also make the updated Openfiler installation USB key bootable.

Update Openfiler

Let’s first update Openfiler.

1. Log into the CLI as root
2. Run

   conary updateall

3. This will take a while as it downloads around 26 packages and installs them.
4. Once complete insert the Openfiler CD into your drive and restart your system, making sure that it boots from CD.

Creating a new ramdisk that works with the new kernel

This part is more or less very much similar to the steps in the previous post, there are some minor additions that we need to make, but for completeness I’ve included all the steps here.

1. Once Openfiler finishes booting from the CD type.

   linux rescue

2. Go through the menus and select your region and keyboard and skip the automatic mounting of your installed OS. We will do this manually.
3. When the prompt appears, create a directory to mount the USB key.

   mkdir /mnt/sysimage

4. Now mount the / of the USB key onto /mnt/sysimage.

   mount /dev/sda2 /mnt/sysimage

   Note: your / partition may be /dev/sda3 instead, depending on how you setup your partitioning during the installation of Openfiler.
   

5. Now mount the boot partition of the USB key onto /mnt/sysimage/boot.

   mount /dev/sda1 /mnt/sysimage

   Note: your / partition may be /dev/sda1 instead, depending on how you setup your partitioning during the installation of Openfiler.
   

6. Make the /mnt/sysimage your working environment by changing your root location so you are working on the file system on the usb key.

   chroot /mnt/sysimage

7. Copy the current initrd file to a temporary location where we can work on it.

   cp /boot/initrd-¹ /tmp/initrd.gz

   Note¹: now’s a good time to press TAB, there will now be two kernels, use 2.6.29.6-0.24.smp.gcc3.4.x86_64 as this is the updated kernel that was installed during the update.
   

8. Gunzip the initrd.gz file

   gunzip /tmp/initrd.gz

9. Make a temporary working directory

   mkdir /tmp/b

   We are using /tmp/b because /tmp/a already exists as the temporary working directory from the previous article.
   

10. Go into the new working directory

    cd /tmp/b

11. Extract the contents of the initrd file into /tmp/b.

    cpio –i < /tmp/initrd

12. Now we edit the init file to load the USB and SCSI drivers for the new initrd-usb-update.img ramdisk.

    nano init

13. Do a search for “mount –t proc /proc /proc” and add the following underneath this line. We want to load these USB and storage modules before any other modules that’s why these entries need to be at the top of the file. Note that there is a new module crc-t10dif.ko which is required by the new kernel to boot from USB and as such must be launch during init time.

    echo “Starting Openfiler on USB”

    echo “Loading scsi_mod.ko module”

    insmod /lib/scsi_mod.ko

    echo “Starting crc-t10dif.ko module”

    insmod /lib/crc-t10dif.ko

    echo “Loading sd_mod.ko module”

    insmod /lib/sd_mod.ko

    echo “Loading sr_mod.ko module”

    insmod /lib/sr_mod.ko

    echo “Loading ehci-hcd.ko module”

    insmod /lib/ehci-hcd.ko

    echo “Loading uhci-hcd.ko module”

    insmod /lib/uhci-hcd.ko

    echo “Loading ohci-hcd.ko module”

    insmod /lib/ohci-hcd.ko

    sleep 5

    echo “Loading usb-storage.ko module”

    insmod /lib/usb-storage.ko

    sleep 5

14. Do a search for insmod /lib/scsi_mod.ko, insmod /lib/sd_mod.ko, insmod /lib/ehci-hcd.ko, insmod /lib/uhci-hcd.ko and /lib/crc-t10dif.ko and remove these duplicate entries from the rest of the file. We do not want these loaded again.
15. Save the file and exit with CTRL X, then Y, or if you used vi then :wq!
16. Now we need to copy all of the modules in Step 13 into our working directory.
17. Go to the drivers directory

    cd /lib/modules/²/kernel/drivers

    Note²: just press tab to fill in this bit, there will now be two kernels, use 2.6.29.6-0.24.smp.gcc3.4.x86_64 as this is the updated kernel that was installed during the update.
    

18. Copy all of the modules in Step 13 to /tmp/b/lib.

    cp scsi/scsi_mod.ko /tmp/b/lib

    cp scsi/sr_mod.ko /tmp/b/lib

    cp scsi/sd_mod.ko /tmp/b/lib

    cp usb/host/ehci-hcd.ko /tmp/b/lib

    cp usb/host/uhci-hcd.ko /tmp/b/lib

    cp usb/host/ohci-hcd.ko /tmp/b/lib

    cp usb/storage/usb-storage.ko /tmp/b/lib

    cp /lib/modules/2.6.29.6-0.24.smp.gcc3.4.x86_64/kernel/lib/crc-t10dif.ko /tmp/b/lib

19. Now let’s package the contents of the working directory /tmp/b into our new initrd-usb-update.img.

    cd /tmp/b

    find . | cpio –c –o | gzip -9 > /boot/initrd-usb-update.img

20. Now all we need to do is edit the /boot/grub/menu.1st file to tell the kernel to use the new ramdisk that we just created. Remember that this new ramdisk is currently located in /boot/ (aka /dev/sda1) and is called initrd-usb-update.img.

    nano /boot/grub/menu.1st

21. Find the line starting with initrd /vmlinux-…………………… and replace with

    initrd /initrd-usb-update.img

22. Save the file and reboot the computer, remove the CD and allowing it to boot from the USB key. You now have your new updated Openfiler installation booting from the USB key directly.

Turn off flow control for SMB Clients

For better CIFS performance turn off your network adapter flow control. I can achieve a sustained 60 mb/s transfer between my Macbook and Openfiler once flow control is turned off. I was only achieving around 30 mb/s previously.

Turn off flow control for ESXi hosts using NFS/iSCSI to Openfiler

First understand what flow control is before performing the follow actions, the following articles provide good cases for either enabling or disabling flow control and auto-negotiation for flow control.

http://www.telecom.otago.ac.nz/tele301/student_html/ethernet-autonegotiation-flow-control.html – not to be confused with auto-negotiation of flow control.

http://virtualthreads.blogspot.com/2006/02/beware-ethernet-flow-control.html

Since this is my lab I’m going to disable flow control completely.

To do this on ESXi hosts follow these instructions or use VMware KB 1013413.

1. Enable Remote SSH for the ESXi host first.
2. Use your favourite SSH client and log in as root (assuming you can, disable lock-down mode etc).
3. Run the following command to list all your vmnic interfaces, make a note of the vmnic that is used to connect to the Openfiler Server.

   esxcfg-nics –l

4. In my case it’s just vmnic0 (I’m using a HP Microserver), type the following command to see the current flow control status of that adapter.

   ethtool –show-pause vmnic0

5. Run the following commands to set auto-negotiation or RX flow control or TX flow control, any combination is possible.
6. To disable flow control for sent and received traffic, use the command:

   ethtool –pause tx off rx off

7. To disable auto-negotiation of flow control, use the command:

   ethtool –pause autoneg off

1. Open the /etc/rc.local file using a text editor and append the same commands used in Step 6, placing each on its own line. Then save the file.
2. For an ESXi host, save the configuration change using the command:

   /sbin/auto-backup.sh

   The commands added to the /etc/rc.local file will be executed at startup, persisting the configuration changes across reboots. As they are executed in Step 6, no reboot is required for them to take effect.