How to Enable VMware VAAI and Install NetApp Plugin on ESXi

VAAI stands for VMware APIs Array Integration (for vSphere Storage).

VAAI is also referred to as hardware acceleration (or hardware offload APIs), which are a set of APIs to enable communication between VMware vSphere ESXi hosts and storage devices.

This APIs are enabled on the ESXi host to offload certain storage operations to the storage array instead of all the I/O operations being handled by the VM kernel.

The tasks that can be offloaded by these APIs are: VM cloning, migrating from one datastore to another, migrating datastore with in the same storage array, etc.

Hardware-assisted locking allows vCenter to offload SCSI commands from the ESX server to the storage system so the array can control the locking mechanism while the system does data updates.

VMware does not support VAAI on VMFS with multiple LUNs/extents if they all are on different arrays even if all arrays support offloading. In this case the vmkernel will handle I/O operations instead of the array.

This tutorial provides instructions on how to enable VAAI and install plugin on your VMware ESXi 5.x servers.

1. Enable VAAI from Command Line

You can verify whether VAAI is enabled ornot from command line using esxcfg-advcfg command.

If the following output displays “0” (as shown below), then VAAI is not enabled.

# esxcfg-advcfg -g /DataMover/HardwareAcceleratedMove
Value of HardwareAcceleratedMove is 0

# esxcfg-advcfg -g /DataMover/HardwareAcceleratedInit
Value of HardwareAcceleratedInit is 0

Use the option “-s 1” to enable VAAI as shown below. You have to set both “Move” and “Init” to 1 as shown below.

#  esxcfg-advcfg -s 1 /DataMover/HardwareAcceleratedMove
Value of HardwareAcceleratedMove is 1

# esxcfg-advcfg -s 1 /DataMover/HardwareAcceleratedInit
Value of HardwareAcceleratedInit is 1

Please note that in VMware vSphere v5.0 and later, this is enabled by default.

2. Enable VAAI from vSphere Client GUI

• Open the VMware vSphere Client.
• In the Inventory pane, select the ESXi/ESX host.
• Click the Configuration tab.
• Under Software, click Advanced Settings.
• Click DataMover.
• Change the DataMover.HardwareAcceleratedMove setting to 1.
• Change the DataMover.HardwareAcceleratedInit setting to 1.

• Click VMFS3.
• Change the VMFS3.HardwareAcceleratedLocking setting to 1.

• Click OK to save your changes.
• Repeat this process for the all ESXi/ESX hosts connected to the storage.

3. Verify the Plugin

You can use the esxcli command to verify whether the particular plugin that you are interested in installing is already present. In this example, I’m looking for NetApp plugin

esxcli software vib list | grep -i NetApp

For this example, I am installing the NetApp plug-in for NAS datastores. Download the latest plugin from the vendor website and save it locally. You can also download the plug-in for EMC storage and other vendors from their website.

4. Install the Plugin

Use the following esxcli command to Install the Plugin. As you notice the plugin will have .vib extension.

# esxcli software vib install -v /NetAppNasPlugin.v20.vib
Installation Result
   Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
   Reboot Required: true
   VIBs Installed: NetApp_bootbank_NetAppNasPlugin_1.0-020
   VIBs Removed:
   VIBs Skipped:

After this, reboot the ESX machine for the changes to take effect.

Now, verify that the plugin is installed succesfully.

# esxcli software vib list | grep -i netapp
NetAppNasPlugin  1.0-020     NetApp     VMwareAccepted    2014-07-30

5. Check Datastore’s VAAI Support

Verify the NAS datastore already mounted supports VAAI using the vmkfstools command as shown below.

# vmkfstools -P -h /vmfs/volumes/46db973f-cca15877
NFS-1.00 file system spanning 1 partitions.
File system label (if any): TEMP NAS
Mode: public
Capacity 2 TB, 791.9 GB available, file block size 4 KB
UUID: 46db973f-cca15877-0000-000000000000
Partitions spanned (on "notDCS"):
 nfs:TEMP NAS
NAS VAAI Supported: YES
Is Native Snapshot Capable: YES

6. VAAI Performance Tuning and Monitoring

The default XCOPY (Clone operations) size is 4MB.

With a 32MB I/O, the expectation would be to see this counter in esxtop increment in batches of eight, the number of work items that will be created to deliver a 32MB I/O.

The default XCOPY size can be incremented to a maximum value of 16MB if required, but that should be done only on the advice of your storage array vendor.

While a storage array vendor might suggest making a change to this parameter for improved performance on their particular array, it may lead to issues on other arrays which do not work well with the new setting, including degraded performance.

You can observe the clone operations (XCOPY) in esxtop output. To see this output in esxtop, first select u for device view and then f to change which fields are displayed. Options o and p display VAAI statistics. Each primitive also has a column for failure statistics, that is, CLONE_F, ATSF and ZERO_F. These should be monitored for any failed operations.

Execute the following esxcfg-advcfg command to query the current transfer size:

# esxcfg-advcfg -g /DataMover/MaxHWTransferSize
Value of MaxHWTransferSize is 4096

Execute the following esxcfg-advcfg command to change the transfer size:

# esxcfg-advcfg -s 16384 /DataMover/MaxHWTransferSize
Value of MaxHWTransferSize is 16384

7. Check the status of VAAI

Use the following esxcli command to display the status of VAAI along with all installed plugins.

# esxcli storage core device vaai status get -d naa.xxxxxxxxxxxxxxxx
naa.xxxxxxxxxxxxxxxx
 VAAI Plugin Name: VMW_VAAIP_NETAPP
ATS Status: supported
Clone Status: supported
Zero Status: supported
Delete Status: supported