you're reading...

Changing the HBA queue depths on multiple dual-port adapters

Following on from optimising the storage for a customer, I decided to change the queue depths for the Emulex HBAs. The ESX hosts, each have two dual-port Emulex HBAs, the diagram below shows the setup..

Only two ports are in use at the moment, vmhba2 and vmhba5. To determine the instance numbers that are in use by the Emulex ESX driver – lpfc (use qla2300 or similar for QLogic), the output of the ls command includes a number for each active HBA in the system. We can then use the instance numbers to find the active adapters.

Emulex example

# ls /proc/scsi/lpfc

You should get an output similar to

Because of the way that the host is connected and from the picture above, I already know that 2 and 5 are the active adapters. Running the following command will confirm

# cat /proc/scsi/lpfc/2

this shows that vmhba2 is currently active and has 4-paths to the SAN

Running the same command on vmhba3 gives the following as expected

Running the command on vmhba5 is also as expected.

Now that we’ve found out which vmhbas are active, we can use the output to find out which lpfc# options we should add to the lpfc_740.o module to configure the queue length.

The outputs of # cat /proc/scsi/lpfc/2 and # cat /proc/scsi/lpfc/5, give us lpfc numbers of 0 and 3 respectively. So to configure a queue depth of 64 for lpfc2 and lpfc5 we run the following command

# esxcfg-module -s “lpfc0_lun_queue_depth=64 lpfc3_lun_queue_depth=64” lpfc_740


# esxcfg-boot -b

The -q option shows configured options for a module.

Now we reboot for the changes to take effect.

In this case, both HBAs lpfc0 (vmhba2) and lpfc3 (vmhba5) will have their queue depths set to 64.

With this post and the previous one, we have set manual load balancing for the LUNs over eight different paths and also changed the queue depth to 64, this should keep the ESX optimised for now, maybe I’ll change the VMFS3.MaxHeapSizeMB to 64 for good measure!


About Hugo Phan



No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: