VAAI Compatibility and Displaying VAAI Statistics

Most VMware admins are familiar with VAAI (vStorage API for Array Integration), a key API to make storage faster, more efficient, and easier to manage in vSphere environments. In this blog, I explain how VAAI affects storage within vSphere environments and what each primitive does and its inherent benefit. I also describe how Tegile flash arrays are certified by VMware for VAAI.

VAAI Primitives:

The below table lists VAAI primitives, their brief descriptions, and their benefit:

Type VAAI Primitive name (per VMware HCL) Also known as Definition Benefit
BLOCK (FC & iSCSI) Thin provisioning Allows over-provisioning of space in a datastore Efficiency
Quota Soft threshold Will alert before running out of space and pause VMs in Datastore if running out of space. Alerting
Hardware assisted locking Atomic Test & Set (ATS) Enables locking at storage device level. If this feature is not available hypervisor will handle locking Performance
Full copy Extended Copy (XCopy) Used for cloning at an array level during VM Cloning and VMotion. Reduced SAN traffic
Block Zero Space Reclamation (SCSI UNMAP) Kindly refer to the Tegile VAAI space Reclamation blog for additional details on block zero primitive Efficiency
NFS NFS full File Clone Hardware Snap Full Copy for NFS Reduced NAS Traffic
NFS Space Reservation Zero-thick files Creates thick files (non-thin) Performance
NFS Extended Stat Extended Stat VSphere can query file size Operations
NFS Space Reclaim Space Reclamation Similar to block reclamation – unlinks files Efficiency
NFS Fast File Clone Linked Clone Offload Offloads Linked Clone operations to array Performance

For more detailed descriptions, please refer to this VMware KB article #1021976

Storage Certification: is required by VMware to ensure these APIs are working correctly with the storage array, and recertification is required every time a vendor releases a major firmware update. Here’s a link to the searchable page to the VMware Compatibility Matrix, which specifies storage models, protocols, and the VAAI primitives that are officially supported. Tegile supports all the VAAI primitives for NFS, FC and iSCSI protocols.

VMware Compatibility Matrix: VMware publishes the compatibility matrix based on submitted and accepted certifications from hardware vendors. As an example, the following screenshot shows a partial listing for a Tegile IntelliFlash™ T3800 array as shown in the VMware compatibility matrix for search criteria: Partner Name: Tegile Systems and Array Type=FC.

The following screenshot (based on a search for Tegile and FC) shows how the VMware compatibility list shows the VAAI block primitives certified for Tegile T3800 array with IntelliFlash 3.0 firmware.

Tegile LUNS and VAAI: To check which VAAI primitives are supported from the command-line, use the following command after logging in to an ESX host. Ironically, the naming for each of these is different in different places, command outputs, compatibility list, and documentation.

(partial listing for a LUN)
[root@esx930-2:~] esxcli storage core device vaai status get
VAAI Plugin Name:
ATS Status: supported
Clone Status: supported
Zero Status: supported
Delete Status: supported

VAAI Plugin Name:
ATS Status: supported
Clone Status: supported
Zero Status: supported
Delete Status: supported

Displaying VAAI Extended LUN Statistics

It is useful to monitor VAAI operations if you encounter any performance issues performing certain operations. For example, you may want to monitor VAAI operations if it’s taking a long time to clone or run SCSI-UNMAP. In fact, it is always recommended to monitor SCSI-UNMAP when reclaiming space.

VMware vSphere shell has a performance-monitoring tool called ESXTOP. ESXTOP has the ability to display extended statistics for LUN(s), which can be selectively enabled to monitor if a particular VAAI counter is incremented or not on a VAAI operation. The following sections show how extended statistics for VAAI performance can be monitored using ESXTOP.

Monitoring VAAI primitive performance:

  1. Start ESXTOP on the ESXi host
  2. Press “u” on the keyboard to look at Disk VAAI statistics (without extended stats)
  3. Press “f” to add new fields as below and press “O” and “P” to enable them. You could disable others if you only want VAAI statistics. Pressing a parameter again will disable it (toggle)
    (partial listing – * means that column is displayed in ESXtop)
    * O: VAAISTATS= VAAI Stats
    * P: VAAILATSTATS/cmd = VAAI Latency Stats (ms)
  4. Press any key to return to main (disk) screen. You will see additional fields as below for VAAI Stats.

Clone Operations:

CLONE_RD – number of successfully completed CLONE commands where disk device was a source
CLONE_WR – number of successfully completed CLONE commands where disk device was a destination
CLONE_F – number of failed CLONE commands
MBC_RD/s – CLONE data read per second (in MB)
MBC_WR/s – CLONE data written per second (in MB)

ATS Operations (atomic test & copy):

ATS – number of successful ATS operations
ATSF – number of failed ATS operations

Block Zero Operations:

ZERO – number of successful block zero operations
ZERO-F – number of failed block zero operations
MBZero/s – Megabytes reclaimed

DELETE Operations:

DELETE – no of successful delete operations
DELETE-F – no of failed delete operations
MBDEL/S – Megabytes deleted

Failure of a VAAI operation indicates that:

  • Either the array doesn’t no support the VAAI primitive; in this case it is better to disable the primitive for better performance;
  • Or it’s a bug with the VAAI implementation; if that’s the case, contact your storage vendor to get this issue fixed via firmware patch and then enable it.

Keep in mind though, when a VAAI primitive is disabled, it gets disabled for the entire host that is connected to the SAN and all storage enclosure. Therefore, selectively disabling a VAAI primitive by array vendor is not an option. There is a VMware KB article #1033665 that shows you how to disable one or more VAAI primitives if required.

As of IntelliFlash version 3.5, Tegile supports VAAI for NAS for the following additional primitives:

  1. Full Copy
  2. NFS Fast File Clone (will enable VCAI [View Composer Array Integration] for linked clones)

For more info on VAAI for NAS, refer to the following document for more details


A storage array that supports VMware VAAI primitives provides significant advantages in performance, manageability and efficiency. VMware’s certification program ensures that customers can reap these benefits when they’re using VMware-certified storage.

VMware storage compatibility matrix is an easy and searchable page that provides confirmation of certified storage with vSphere to a customer before purchasing and deploying it.

It is also possible to determine if VAAI primitives are supported from ESXI shell command-line.

ESXtop, a native ESXi performance utility provides an easy way to monitor and measure VAAI performance. VAAI primitives can be disabled as a stop-gap solution, if there are too many failed VAAI operations that are causing performance issues.

One comment on “VAAI Compatibility and Displaying VAAI Statistics

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>