Following on from optimising the storage for a customer, I decided to change the queue depths for the Emulex HBAs. The ESX hosts, each have two dual-port Emulex HBAs, the diagram below shows the setup..
Only two ports are in use at the moment, vmhba2 and vmhba5. To determine the instance numbers that are in use by the Emulex ESX driver – lpfc (use qla2300 or similar for QLogic), the output of the ls command includes a number for each active HBA in the system. We can then use the instance numbers to find the active adapters.
# ls /proc/scsi/lpfc
You should get an output similar to
Because of the way that the host is connected and from the picture above, I already know that 2 and 5 are the active adapters. Running the following command will confirm
# cat /proc/scsi/lpfc/2
Running the same command on vmhba3 gives the following as expected
Running the command on vmhba5 is also as expected.
Now that we’ve found out which vmhbas are active, we can use the output to find out which lpfc# options we should add to the lpfc_740.o module to configure the queue length.
The outputs of # cat /proc/scsi/lpfc/2 and # cat /proc/scsi/lpfc/5, give us lpfc numbers of 0 and 3 respectively. So to configure a queue depth of 64 for lpfc2 and lpfc5 we run the following command
# esxcfg-module -s “lpfc0_lun_queue_depth=64 lpfc3_lun_queue_depth=64” lpfc_740
# esxcfg-boot -b
Now we reboot for the changes to take effect.
In this case, both HBAs lpfc0 (vmhba2) and lpfc3 (vmhba5) will have their queue depths set to 64.
With this post and the previous one, we have set manual load balancing for the LUNs over eight different paths and also changed the queue depth to 64, this should keep the ESX optimised for now, maybe I’ll change the VMFS3.MaxHeapSizeMB to 64 for good measure!