Category: ESXi

Creating (a better) vSphere 5 ESXi embedded USB Stick (HP)

In a previous post I blogged about creating a vanilla vSphere 5 ESXi USB drive using the VMware .iso file from VMware. This post shows how to create one using the HP version of vSphere ESXi (5.0_Oct_2011_ESXi_HD-USB-SDImgeInstlr_Z7550-00253.iso).

Note: (You can use any vendor customized vSphere ESXi .iso file: VMware, Dell and IBM).

The HP version comes pre-installed with all the HP CIM providers which work very well with HP servers, including the HP MicroServer. Using the HP version gives you the more details in the Hardware Status tab.

I’m going to be using a different method, recommended by Will Rodbard (thanks Will), who is a colleague of mine at VMware, you can see his comments from the previous post. In summary the steps are:

  1. Find and download the following tools:

    HPUSBFW & UNETBOOTIN

  2. Run the HPUSBFW tool, click on the USB drive, select ‘Fat32′ and click Format
  3. Run UNETBOOTIN, select Diskimage and browse to the ESXi 5 ISO file
  4. Select the USB drive you have just formatted and click OK
  5. If you want to make more USB keys for more servers, then now is the time to create .IMG files using WinImage, then you can basically clone the image of the USB key to more USB keys. Or if you don’t wish to use WinImage then just perform steps 1 to 4 again.

Once completed your USB drive will boot into the ESXi 5 installer. Once booted, install the ESXi 5 Hypervisor to the USB drive (overwriting the installer). This will then leave you with the installed ESXi Hypervisor on the USB.

Note that using this method creates a brand new bootable USB key for use in a new installation of vSphere ESXi. You will have to go through the process of installing ESXi onto the USB key, or another disk or LUN on the target server. If you want a USB key that is already installed with ESXi which saves you from going through the installation wizard, you can use the other method in this post.

[Aside]

I coincidently left an older USB key in my laptop and booted. Here’s a picture of my Macbook Pro running vSphere ESXi, and it all works by the way, including networking!

Advertisement

Creating vSphere 5 ESXi embedded USB Stick

A very quick post on how to create an image that contains vSphere 5 ESXi Embedded with which you can use to quickly create USB sticks that have the ESXi hypervisor installed.  This is not the same as creating a bootable USB key that contains the installation files to install ESXi from the USB stick.  For this method please refer to this post.

Use this in your lab environment, I wouldn’t recommend doing this in production environments.

In previous versions of vSphere ESXi, it was relatively straight forward to create a bootable USB key which already contained the ESXi hypervizor.  This was done by extracting the files from the ISO and then using ‘dd’ to image the directory structure to the USB stick.  With vSphere ESXi 5 however, this technique is no longer possible.  There is a workaround however.  ESXi is installed and configured in two steps, the installation is done to a disk with a vanilla installation of ESXi without configuration.  The server is then rebooted and the configuration of ESXi continues with the creation of the management network vmk0 or vmk1 (depending on your setup), hostname, DNS etc.

For this to work, we do not perform the second part, which is the configuration, but take an image of the USB key directly after the installation of the vanilla installation of ESXi without configuration.  This enables us to image this vanilla installation onto as many USB sticks, i.e., servers as we like without clashes in virtual MAC addresses and the like.

What you will need: VMware Workstation, 1 USB stick, the ESXi Installable ISO file VMware-VMvisor-Installer-5.0.0-469512.x86_64.iso, WinImage.

Quick steps

  1. Create a new ESX virtual machine in VMware Workstation with CD-ROM drive, USB adapter, 2Gb RAM and 2vCPUs.
  2. Mount the ESXi Installable ISO file to the CD Drive.
  3. Insert the USB stick to your workstation (the same one that runs VMware Workstation).
  4. Boot the VM and connect the USB stick to the VM.
  5. Install ESXi as normal, making sure that you install onto the USB stick, when installation is complete, disconnect the USB stick from the VM and do not reboot the VM, just turn it off.  You no longer need this VM.
  6. With the USB stick still connected to your workstation, open up Winimage.
  7. Go to Disk | Creating Virtual Hard Disk image from physical drive and select the USB stick that you installed ESXi on.
  8. Select a location where to save your image and change the file type to Image file (*.ima).
  9. WinImage will now make a backup on your newly installed USB stick.

Creating vSphere 5 ESXi embedded bootable USB sticks

  1. Now that you have an ESXi image, you can use this to build lots of USB sticks which are ready for ESXi deployment.
  2. Insert a new USB stick into a spare USB port.
  3. Launch WinImage and navigate to Disk | Restore Virtual Hard Disk image on physical drive.
  4. Select the USB stick and click on OK.
  5. Navigate to the image file that you created previously.  WinImage will now restore the backed up image to your new USB stick.
  6. Repeat as necessary.

Configure ESXi

Once the stick is ready, just insert into a spare USB port on your server and ESXi will boot into the configuration screen ready for you to configure management network details.

You may need to log onto the local console once ESXi has finished booting and launch the ‘Restore Network Settings’.  This will reset the vmk0 or vmk1 (depending on your setup) interface.

VMware Tools RPM Installation with PXEBOOT and Kickstart

Purpose

This article explains how to use the Operating System Specific Packages (OSPs) to install VMware Tools during the PXEBOOT and Kickstart of a Linux guest OS running on vSphere 4.1 and above.

Background

As of vSphere 4.1, the VMware Tools RPMs are no longer available on the CD image linux.iso. To install you must use the tar.gz package and install it manually.

To quote the vSphere 4.0 U2 release notes:

“The VMware Tools RPM installer, which is available on the VMware Tools ISO image for Linux guest operating systems, has been deprecated and will be removed in a future ESX release. VMware recommends using the tar.gz installer to install VMware Tools on virtual machines with Linux guest operating systems.”

This future ESX release is indeed vSphere 4.1. The reason that the RPM packages are no longer available on the VMware Tools CD image for Linux guest operating systems ISO is to reduce the footprint of ESXi. Therefore the linux.iso no longer contains the RPM installer.

Not only is this a real pain for customers who have always deployed packages onto their Linux guests with RPMs but also removing the VMware Tools RPM also causes issues with the deployment of Linux guests through PXEBOOT and Kickstart methods.

Yes, you could argue that there is the option of using vCenter Templates along with Customization Specifications (CS) for Linux, and I for one love the fact that the CS works for Linux very well. This is not always an option for a customer who has already configured their entire environment to deploy both ESXi servers and virtual machine guests with PXEBOOT and Kickstart.

The only option here is to perform manual installations of VMware Tools using the tar.gz file which is included with vSphere 4.1. This of course poses a few issues.

One: this is a repetitive task which is both time consuming and problematic if the number of new VMs to provision is high.

Two: some security conscious organisations do not allow the gcc Compiler and/or the Linux Kernel sources to be installed on the VM guests. Both the gcc Compiler and the Linux Kernel sources are mandatory for the successful installation of VMware Tools using the tar.gz file.

Three: if the guest VM is using a VMXNET2 or VMXNET3 ethernet adapter, then the guest VM will not have compatible drivers if VMware Tools is not installed.

These are just three of the reasons that I’ve seen in the wild, there are more but I won’t go into much detail here.

So how do we go about solving this?

Have a read of KB article 1024047: ESXi 4.x does not include RPM format for VMware Tools.

The solution here is to use the Operating System Specific Packages (OSPs). VMware Tools OSPs allow you to use your operating system’s native update mechanisms to automatically download, install, and manage VMware Tools for the supported operating systems. For more information regarding OSPs, see http://www.vmware.com/download/packages.html.

For this guide, we are referring to RHEL’s yum update mechanism.

This guide details how you can use the http://www.vmware.com/pdf/osp_install_guide.pdf to prepare your Build Server to enable the automated deployment of VMware Tools during a guest RHEL in a PXEBOOT environment.

Resolution

For this guide, I will be using the Ultimate Deployment Appliance 2.0 (uda20.build17) to deploy a RHEL 5.5 x64 virtual machine.

First we need to prepare to install the operating specific packages (OSPs) for RHEL 5 guest operating systems on the Build Server. We do this by preparing the directory structure for the rpms, and placing these in the correct location on the Build Server for the guest VM to download during a Kickstart installation.

Perform the following on your workstation.

The OSPs are located on the VMware Tools packages Web site at http://packages.vmware.com/tools.

  1. First download the entire directory that contains the package relevant to your operating system, for this guide I will be using the RHEL x86_64 packages located at http://packages.vmware.com/tools/esx/4.1u1/rhel5/x86_64/index.html.

     

  2. Clicking on this link will give you the following list of files.

  3. Do a right-click Save-As and save these files to a location of your choice. I settled for a folder called vmwaretools on my Desktop.

     

  4. Also create a folder within vmwaretools called repodata, and also download the files from http://packages.vmware.com/tools/esx/4.1u1/rhel5/x86_64/repodata into your vmwaretools/repodata directory.

     

  5. Once these two tasks have been complete you will see the following directory contents of vmwaretools.

  6. And the contents of vmwaretools/repodata.

  7. Now you will need to download the VMware Packaging Public Keys.

     

  8. Create a directory called keys within vmwaretools/

     

  9. Point your web browser to http://packages.vmware.com/tools/keys and download all of the files into your vmwaretools/keys directory.

     

  10. The contents of the keys directory should look like this

     

  11. You should now have a vmwaretools folder structure that looks like this

     

  12. Now copy the entire vmwaretools directory to your Build Server using your favourite SCP application. I’m using the UDA 2.0, so I will be placing vmwaretools in /var/public/www/kickstart/rhel/. “rhel” is the ‘Flavor’ name that I called my RHEL 5 OS configuration.

Perform the following on the Build Server.

  1. On the UDA 2.0 the contents of the /var/public/www/kickstart/rhel/vmwaretools directory should now look like this.

     

  2. If you navigated to http://<ip_address_of_uda/kickstart/rhel/vmwaretools/, you should be able to see the same folder structure. If you can’t then you may need to do a chmod 755 on the vmwaretools directory and its contents.

     

  3. Now edit your kickstart file to look something like this.

    %post

    # Import the VMware Packaging Public Keys from the Build Server.

    rpm –import http://%5BUDA_IPADDR%5D/kickstart/%5BTEMPLATE%5D/vmwaretools/key/VMWARE-PACKAGING-GPG-DSA-KEY.pub

    rpm –import http://%5BUDA_IPADDR%5D/kickstart/%5BTEMPLATE%5D/vmwaretools/key/VMWARE-PACKAGING-GPG-RSA-KEY.pub

    # Create and edit the VMware repository directory and file, note that this points to the Build Server during Kickstart build. Once VMware Tools is installed we shall change the baseurl to point to the VMware OSP URL.

    # Add the following contents to the repository file and save

    cat > /etc/yum.repos.d/vmware-tools.repo <<\EOF1

    [vmware-tools]

    name=VMware Tools

    baseurl=http://192.168.200.30/kickstart/rhel/vmwaretools

    enabled=1

    gpgcheck=1

    EOF1

    # Install VMware Tools by accepting all defaults

    yum install -y vmware-tools

    # Delete the customised vmware-tools.repo file and reconfigure with the baseurl to point to the VMware OSP URL for RHEL5 64-bit.

    rm –f /etc/yum.repos.d/vmware-tools.repo

    cat > /etc/yum.repos.d/vmware-tools.repo <<\EOF2

    [vmware-tools]

    name=VMware Tools

    baseurl= http://packages.vmware.com/tools/esx/4.1u1/rhel5/x86_64

    enabled=1

    gpgcheck=1

    EOF2

  4. I have attempted to use the VMware Packaging Public Keys from the VMware OSP repository URL but this does not work, anyone care to enlighten me in the comments below?

    I tried this:

    # Import the VMware Packaging Public Keys

    rpm –import http://packages.vmware.com/tools/keys/VMWARE-PACKAGING-GPG-DSA-KEY.pub

    rpm –import http://packages.vmware.com/tools/keys/VMWARE-PACKAGING-GPG-RSA-KEY.pub

     

  5. I also attempted to use the VMware OSP URL directly in the vmware-tools.repo file without luck either. If you can get this to work then please comment below and I’ll update the post.

You should now be able to deploy a guest Linux VM running on vSphere ESXi 4.1 through PXEBOOT and Kickstart and have the VM automatically install VMware Tools. Plus, all future updates of VMware Tools can just be done by invoking a yum install -y vmware-tools.

Enabling VMXNET 3 for PXEBOOT and KICKSTART of RHEL Virtual Machines

Purpose

This guide shows you how to create a new initrd.img with integration for the VMXNET3 driver to allow guest RHEL virtual machines equipped with the VMXNET 3 driver to Kickstart build RHEL using PXEBOOT.

Background

VMware’s VMXNET 3 network adapter supports PXE booting but RHEL 5 does not have a driver that supports network installations using the default initrd.img.

If you tried to perform automated installation using kickstart with the standard initrd.img you will see the following screen:

 

This is because Anaconda does not recognise the VMXNET 3 device and therefore is not able to load a driver for it.

This guide shows you how to create a new initrd.img with integration for the VMXNET3 driver.

For the impatient few, I’ve made the resulting initrd.img.vmxnet file available for download, it is a clean ramdisk image that was made using the steps below.

It is the PXEBOOT RAMDISK with the VMXNET3 driver for RHEL5 (created from the rhel-server-5.5-x86_64-dvd) [2.6.18-194.el5].

Tested and working to support VMXNET3 in Anaconda.  You can download it from here initrd.img.vmxnet and then jump all the way to Step 18 to place it on your Build Server.

Prerequisites

Prepare a Reference Virtual Machine

First create a new reference virtual machine with the following hardware specifications:

Configuration Value
VM Hardware Version Hardware Version 7
Network Adapter VMXNET 3
SCSI Controller LSI Logic SAS
SYSTEM .vmdk Device SCSI 0:0 15Gb
Remove Floppy Device Yes

Install RHEL (rhel-server-5.5-x86_64-dvd) by mounting the ISO to the VM and then perform a manual installation of VMware Tools. This will give you the reference virtual machine with which you will then use to copy the vmxnet.ko and vmxnet3.ko from.

Enable sshd services on the Reference VM by typing:

/etc/init.d/sshd start

This will make it a lot easier to copy files to your Build Server.

Prepare your Build Server

Create your own PXEBOOT and Kickstart installation or use one that you already have. For my example I will be using the Ultimate Deployment Appliance 2.0 (uda20.build17).

Most of the configuration is done on Build Server so by all means enable SSHD to make things a lot easier for you.

My Build Server IP is 192.168.200.30.

Integrating VMXNET 3 into initrd.img

At this point you should have SSH access to both your Build Server and your Reference VM.

Perform the following on the Build Server.

1.    Make some working directories to work in

mkdir /tmp/workingdir

mkdir /tmp/workingdir/initrd

mkdir /tmp/workingdir/modules

Perform the following on the Reference VM.

2.    Obtain the initial ramdisk initrd.img from the pxeboot directory, this file can be obtained from the rhel-server-5.5-x86_64-dvd ISO file which should still be connected to the Reference VM. The initrd.img file can be found in the images/pxeboot directory.

3.    Mount the ISO image

mount /dev/cdrom /media

4.    Copy the initrd.img to the Build Server

scp /media/images/pxeboot/initrd.img root@192.168.200.30:/tmp/workingdir/

5.    We now need to ascertain the PCI and Device ID of the VMXNET 3 network adapter by first running

Lspci

        Note that our VMware VMXNET3 Ethernet Controller lives on 0b:00:0

6.    With this information we can obtain the HEX number for the device by running

lspci –n

Note the HEX value for device 0b:00.0 is 15ad:07b0.

Perform the following on the Build Server.

7.    Unpack the initrd.img file to allow us to amend the ramdisk, you should be in /tmp/workingdir/initrd/

zcat ../initrd.img | cpio –id

 

8.    Extract the modules.cgz archive from within the initrd subdirectory

cd /tmp/workingdir/modules

zcat ../initrd/modules/modules.cgz | cpio –id

Perform the following on the Reference VM.

9.    Copy the vmxnet*.ko modules from the Reference VM over to the Build Server. The vmxnet*.ko files are located in /lib/modules/2.6.18-194.el5/misc

scp /lib/modules/2.6.18-194.el5/misc/vmxnet*.ko root@192.168.200.30:/tmp/workingdir/modules/2.6.18-194.el5/x86_64/

10.    Copy the modules.alias file from the Reference VM to the Build Server for use later on. This file contains the vmxnet entries and is located at /lib/modules/2.6.18-194.el5/

scp /lib/modules/2.6.18-194.el5/modules.alias root@192.168.200.30:/tmp/workingdir/initrd/modules/modules.alias.reference

Perform the following on the Build Server.

11.    Change permissions for the two new vmxnet*.ko files, you should be in /tmp/workingdir/modules/2.6.18-194.el5/x86_64/

chmod 744 vmxnet*

12.    Pack up the new modules.cgz which now includes the vmxnet*.ko modules and create a new cpio archive to replace the old modules.cgz.

cd /tmp/workingdir/modules

find . | cpio -o -H crc | gzip -9 > /tmp/work/initrd/modules/modules.cgz

    After a few seconds the operation will complete.

13.    Modify the pci.ids file with an entry for the VMXNET 3 adapter.

cd /tmp/workingdir/initrd/modules

nano pci.ids

14.    Search for VMware and add the following line under the Abstract SVGA Adapter

07b0    VMware Adapter

    The 07b0 number here is whatever was obtained from Step 5 above.

15.    Edit the module-info file and add the following entries for the VMXNET and VMXNET 3 Adapters, put it in under ‘v’ to keep it in alphabetical descending order. You should still be in /tmp/workingdir/initrd/modules/

nano /tmp/workingdir/initrd/modules/module-info

vmxnet

    eth

    “VMware vmxnet Ethernet driver”

 

vmxnet3

    eth

    “VMware vmxnet3 Ethernet driver”

16.    Import the vmxnet entries from the Reference VM’s module.alias file (now called module.alias.reference) into the Build Server’s module.alias file.

grep vmxnet /tmp/workingdir/initrd/modules/modules.alias.reference >> /tmp/workingdir/initrd/modules/modules.alias

The contents of the new module.alias file should look like this.

17.    Package the new initrd.img ramdisk up with all the changes done above.

cd /tmp/workingdir/initrd

find . | cpio -o -H newc | gzip -9 > /tmp/workingdir/initrd.img.vmxnet

18.    Copy the new initrd.img.vmxnet into the PXEBOOT environment. On UDA2.0 this location is /var/public/tftproot/

cp /tmp/workingdir/initrd.img.vmxnet /var/public/tftproot/

19.    Edit your PXEBOOT configuration to use the new initrd.img.vmxnet file instead of the standard initrd.img file. My example uses the UDA.

cd /var/public/conf/templates/

nano rhel.dat

20.    On the line CMDLINE=, edit the initrd= entry to point to the new initrd.img.vmxnet instead.

CMDLINE=ks=http://[UDA_IPADDR]/kickstart/[TEMPLATE]/[SUBTEMPLATE].cfg initrd=initrd.img.vmxnet ramdrive_size=8192

21.    That’s it, now PXEBOOT a VM and it will now be able to Kickstart using the VMXNET3 network adapter.

Uninstalling vCD agent on ESXi host

To unistall the vCD agent (vslad) on an ESXi host:

  • Enable Remote Tech Support (SSH) in Configuration | Security Profile | Properties
Enable Remote Tech Support (SSH)
  • Log into the ESXi host using your favourite SSH client
  • Navigate to /opt/vmware/uninstallers
  • Now run the script named vslad-uninstall.sh, or you could just do the below after logging into the ESXi host

/opt/vmware/unistallers/vslad-uninstall.sh

  • Disable Remote Tech Support (SSH)
  • Restart your ESXi host.