How to use iperf on ESXi host

How to use iperf on ESXi host in an article that explains how to use iperf to test network performance on an ESXi host.

Using iperf on an ESXi host is a practical way to measure network performance and validate bandwidth between virtualization components. In this guide, we will demonstrate how to use iperf and iperf3 for network testing scenarios involving ESXi hosts and virtual machines.

Basically, what is iperf/iperf3?

Based on the website https://iperf.fr/, iPerf3 is a tool for active measurements of the maximum achievable bandwidth on IP networks. It supports tuning various parameters related to timing, buffers, and protocols (TCP, UDP, SCTP with IPv4 and IPv6). For each network test, it reports the bandwidth, loss, and other parameters.

The tool iperf or iperf3 can be used in a lot of operational systems. In this case, the Vmware ESXi has support for the iperf by default. In other words, we do not need to download and install iperf on the ESXi host.

How to use iperf on ESXi host?

As we said, with iperf tool we can measure of the maximum achievable bandwidth between two devices. We can do this test between two ESXi hosts, between an ESXi host and a VM, or between both VMs, for example.

The iperf tool has a client/server architecture. So, in this case, one of both devices involved in the test has a Server and another device is a Client.

****** If you are running ESXi 7.x. the below syntax can be used to run iperf:

In this example, our goal is to measure the achievable bandwidth between two ESXi hosts (HOST-01 and HOST-02). In this case, “HOST-01” will be as a Server, and “HOST-02” will be as a Client.

Commands from HOST-01 (Iperf Server):

[root@host01:~] cp /usr/lib/vmware/vsan/bin/iperf3 /usr/lib/vmware/vsan/bin/iperf3.copy
[root@host01:~] /usr/lib/vmware/vsan/bin/iperf3.copy -s

Commands from HOST-02 (Iperf Client):

[root@host02:~] /usr/lib/vmware/vsan/bin/iperf3 -n 1G -c 192.168.200.11

Where:
-n 1G = amount of data that is sending to the iperf server
-c = using iperf as a client
192.168.200.11 = IP address of the iperf server

Note: In this example, we received a timeout message. This connection timeout can be due to the hosts’ firewalls being active. We can temporarily disable the host’s firewall on both ESXi hosts with the below command:

esxcli network firewall set --enabled false

After that, we can run iperf again on both ESXi hosts:

So, in this example:

• We transferred 1 GB of data from HOST-02 to HOST-01
• The achievable bandwidth was 941 Mbits/sec

After the test, if you want to enable the firewall again:

esxcli network firewall set --enabled true

Note: If you need to get the IP address of a specific vmkernel of the ESXi host, you can execute the below command to achieve it:

esxcli network IP interface ipv4 get

****** If you are running ESXi 8.x., we have some considerations:

On VMware ESXi 8.0.0 build-20513097, if we need to use the tool Iperf3 to perform some network tests, we can receive the below error:

[root@esxi8-01:/usr/lib/vmware/vsan/bin] /usr/lib/vmware/vsan/bin/iperf3 -s
iperf3: error - unable to start listener for connections: Operation not permitted
iperf3: exiting

On the same ESXi host, the vmkernel.log logged an entry about this event:

2023-04-25T15:08:36.061Z In(182) vmkernel: cpu2:6469331)VmkAccess: 194: iperf3: running in appDom(27): ipAddr = ::, port = 5201: Access denied by vmkernel access control policy

This behavior is probably related to the enforcement level configured on Security Domains on the ESXi host. The below VMware KB explains a little about that:

• https://kb.vmware.com/s/article/87510

WE ONLY RECOMMEND DOING THAT DURING THE IPERF TEST. AFTER THAT, RETURN THE CONFIGURATION TO DEFAULT, AS WE WILL SHOW:

1- Access the ESXi host by SSH, and check the enforcement level of all Domains:
localcli system secpolicy domain list

2- Based on the vmkernel.log, the domain that handles the iperf3 execution is called “appDom”. So, we can check the enforcement level for this Domain with the below command:

[root@esxi8-01:/usr/lib/vmware/vsan/bin] localcli system secpolicy domain list | grep appDom
appDom                   enforcing

In this case, the enforcement level is “enforcing”.

3- Change the enforcement level for the Domain “appDom” to “disable”:

localcli system secpolicy domain set -n appDom -l disabled

4- After that, we can run the iperf3 as a server normally:

[root@esxi8-01:/usr/lib/vmware/vsan/bin] ./iperf3 -s -B 10.237.180.156
-----------------------------------------------------------
Server listening on 5201 (test #1)
-----------------------------------------------------------

5- After executed the iperf tests, back the “appDom” enforcement level to the default value (enforcing):

localcli system secpolicy domain set -n appDom -l enforcing

How to use iperf to test the network between the ESXi host and a specific VM?

As we mentioned before, we can test the network between an ESXi host and a specific VM. In this example, we will download and run iperf on a Windows Server VM.

To download the iperf package for Windows, click here.

After that, we can start the iperf as a Client on a Windows Server VM:

cd C:\Users\Administrator\Desktop\iperf-3.1.3-win64\iperf-3.1.3-win64
iperf3.exe -n 500M -c 192.168.200.11

So, in this example:

• We transferred 500 MB of data from the Windows Server VM to HOST-01
• The achievable bandwidth was 193 Mbits/sec