//
you're reading...
Networking, Openfiler, VMwire Lab

How to update your Openfiler USB installation for better performance

Previously I wrote an article on How to install and run Openfiler on a USB key. I thought that everything was working fine but eventually found that NFS and CIFS performance was too slow. Upon reading a few forums and stumbling across this thread in particular, the reason was down to Openfiler requiring an update.

I have since tried to update the installation by running conary updateall at the CLI. Unfortunately, this installs an updated kernel (2.6.29.6-0.24.smp.gcc3.4.x86_64 (SMP)) and also a new ramdisk which makes all the hard work from the previous post defunct. This article shows you how to perform the update and then make a new initrd-usb-update.img to work with the new kernel.

So assuming you’ve made a successful USB key using the previous article, continue with the following to update your Openfiler installation and also make the updated Openfiler installation USB key bootable.

Update Openfiler

Let’s first update Openfiler.

  1. Log into the CLI as root
  2. Run

    conary updateall

  3. This will take a while as it downloads around 26 packages and installs them.
  4. Once complete insert the Openfiler CD into your drive and restart your system, making sure that it boots from CD.

Creating a new ramdisk that works with the new kernel

This part is more or less very much similar to the steps in the previous post, there are some minor additions that we need to make, but for completeness I’ve included all the steps here.

  1. Once Openfiler finishes booting from the CD type.

    linux rescue

  2. Go through the menus and select your region and keyboard and skip the automatic mounting of your installed OS. We will do this manually.
  3. When the prompt appears, create a directory to mount the USB key.

    mkdir /mnt/sysimage

  4. Now mount the / of the USB key onto /mnt/sysimage.

    mount /dev/sda2 /mnt/sysimage

    Note: your / partition may be /dev/sda3 instead, depending on how you setup your partitioning during the installation of Openfiler.

  5. Now mount the boot partition of the USB key onto /mnt/sysimage/boot.

    mount /dev/sda1 /mnt/sysimage

    Note: your / partition may be /dev/sda1 instead, depending on how you setup your partitioning during the installation of Openfiler.

  6. Make the /mnt/sysimage your working environment by changing your root location so you are working on the file system on the usb key.

    chroot /mnt/sysimage

  7. Copy the current initrd file to a temporary location where we can work on it.

    cp /boot/initrd-1 /tmp/initrd.gz

    Note1: now’s a good time to press TAB, there will now be two kernels, use 2.6.29.6-0.24.smp.gcc3.4.x86_64 as this is the updated kernel that was installed during the update.

  8. Gunzip the initrd.gz file

    gunzip /tmp/initrd.gz

  9. Make a temporary working directory

    mkdir /tmp/b

    We are using /tmp/b because /tmp/a already exists as the temporary working directory from the previous article.

  10. Go into the new working directory

    cd /tmp/b

  11. Extract the contents of the initrd file into /tmp/b.

    cpio –i < /tmp/initrd

  12. Now we edit the init file to load the USB and SCSI drivers for the new initrd-usb-update.img ramdisk.

    nano init

  13. Do a search for “mount –t proc /proc /proc” and add the following underneath this line. We want to load these USB and storage modules before any other modules that’s why these entries need to be at the top of the file. Note that there is a new module crc-t10dif.ko which is required by the new kernel to boot from USB and as such must be launch during init time.

    echo “Starting Openfiler on USB”

    echo “Loading scsi_mod.ko module”

    insmod /lib/scsi_mod.ko

    echo “Starting crc-t10dif.ko module”

    insmod /lib/crc-t10dif.ko

    echo “Loading sd_mod.ko module”

    insmod /lib/sd_mod.ko

    echo “Loading sr_mod.ko module”

    insmod /lib/sr_mod.ko

    echo “Loading ehci-hcd.ko module”

    insmod /lib/ehci-hcd.ko

    echo “Loading uhci-hcd.ko module”

    insmod /lib/uhci-hcd.ko

    echo “Loading ohci-hcd.ko module”

    insmod /lib/ohci-hcd.ko

    sleep 5

    echo “Loading usb-storage.ko module”

    insmod /lib/usb-storage.ko

    sleep 5

  14. Do a search for insmod /lib/scsi_mod.ko, insmod /lib/sd_mod.ko, insmod /lib/ehci-hcd.ko, insmod /lib/uhci-hcd.ko and /lib/crc-t10dif.ko and remove these duplicate entries from the rest of the file. We do not want these loaded again.
  15. Save the file and exit with CTRL X, then Y, or if you used vi then :wq!
  16. Now we need to copy all of the modules in Step 13 into our working directory.
  17. Go to the drivers directory

    cd /lib/modules/2/kernel/drivers

    Note2: just press tab to fill in this bit, there will now be two kernels, use 2.6.29.6-0.24.smp.gcc3.4.x86_64 as this is the updated kernel that was installed during the update.

  18. Copy all of the modules in Step 13 to /tmp/b/lib.

    cp scsi/scsi_mod.ko /tmp/b/lib

    cp scsi/sr_mod.ko /tmp/b/lib

    cp scsi/sd_mod.ko /tmp/b/lib

    cp usb/host/ehci-hcd.ko /tmp/b/lib

    cp usb/host/uhci-hcd.ko /tmp/b/lib

    cp usb/host/ohci-hcd.ko /tmp/b/lib

    cp usb/storage/usb-storage.ko /tmp/b/lib

    cp /lib/modules/2.6.29.6-0.24.smp.gcc3.4.x86_64/kernel/lib/crc-t10dif.ko /tmp/b/lib

  19. Now let’s package the contents of the working directory /tmp/b into our new initrd-usb-update.img.

    cd /tmp/b

    find . | cpio –c –o | gzip -9 > /boot/initrd-usb-update.img

  20. Now all we need to do is edit the /boot/grub/menu.1st file to tell the kernel to use the new ramdisk that we just created. Remember that this new ramdisk is currently located in /boot/ (aka /dev/sda1) and is called initrd-usb-update.img.

    nano /boot/grub/menu.1st

  21. Find the line starting with initrd /vmlinux-…………………… and replace with

    initrd /initrd-usb-update.img

  22. Save the file and reboot the computer, remove the CD and allowing it to boot from the USB key. You now have your new updated Openfiler installation booting from the USB key directly.

Turn off flow control for SMB Clients

For better CIFS performance turn off your network adapter flow control. I can achieve a sustained 60 mb/s transfer between my Macbook and Openfiler once flow control is turned off. I was only achieving around 30 mb/s previously.

Turn off flow control for ESXi hosts using NFS/iSCSI to Openfiler

First understand what flow control is before performing the follow actions, the following articles provide good cases for either enabling or disabling flow control and auto-negotiation for flow control.

http://www.telecom.otago.ac.nz/tele301/student_html/ethernet-autonegotiation-flow-control.html – not to be confused with auto-negotiation of flow control.

http://virtualthreads.blogspot.com/2006/02/beware-ethernet-flow-control.html

Since this is my lab I’m going to disable flow control completely.

To do this on ESXi hosts follow these instructions or use VMware KB 1013413.

  1. Enable Remote SSH for the ESXi host first.
  2. Use your favourite SSH client and log in as root (assuming you can, disable lock-down mode etc).
  3. Run the following command to list all your vmnic interfaces, make a note of the vmnic that is used to connect to the Openfiler Server.

    esxcfg-nics –l

  4. In my case it’s just vmnic0 (I’m using a HP Microserver), type the following command to see the current flow control status of that adapter.

    ethtool –show-pause vmnic0

  5. Run the following commands to set auto-negotiation or RX flow control or TX flow control, any combination is possible.
  6. To disable flow control for sent and received traffic, use the command:

    ethtool –pause tx off rx off

  7. To disable auto-negotiation of flow control, use the command:

    ethtool –pause autoneg off

  1. Open the /etc/rc.local file using a text editor and append the same commands used in Step 6, placing each on its own line. Then save the file.
  2. For an ESXi host, save the configuration change using the command:

    /sbin/auto-backup.sh

    The commands added to the /etc/rc.local file will be executed at startup, persisting the configuration changes across reboots. As they are executed in Step 6, no reboot is required for them to take effect.

Advertisements

About Hugo Phan

@hugophan

Discussion

5 thoughts on “How to update your Openfiler USB installation for better performance

  1. Hello

    Can I be the worst freetard ever and ask you to put your “openfiler on a stick” iso available ?
    I find myself without a cd drive, but perfectly able to put an iso on a usb… and you just made a perfectly working one with the latest version 8)

    Cheers
    Knewbie

    Posted by Laurent | June 18, 2011, 01:02
    • Hey Laurent,

      Thanks for reading my blog.

      Do you mean make an image of my openfiler box usb key? It will be around 512mb compressed zip file, when extracted it will be a 2Gb .ima file and you will need a 2Gb USB stick in order to write the image to. Not sure how useful this will be for you as it was built on a Dell PE SC440. Let me know if you still want it and I’ll make it available.

      Hugo

      Posted by Hugo Phan | June 18, 2011, 11:41
  2. Hello

    I would be glad to have it and try.
    At worst it is lost time and bandwidth, at best an OF on a key solution 8)

    Laurent

    Posted by Laurent | June 20, 2011, 19:19
  3. You sir are a life saver. My openfiler would not load after i did the conary update all and I wasnt able to load my old config for 2.3.26. Your step by step was so well written that my crisis was averted in under 15 mins. Thanks for posting such a great article.

    Posted by Matthew Jones | December 21, 2011, 03:35

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: