Implementing vSAN HCI Mesh is an article that explains how to mount a remote vSAN datastore provided by a remote vSAN Cluster.
An Introduction About vSAN HCI Mesh
The vSAN HCI Mesh is a service that enables administrators to mount the datastore of another vSAN cluster. We have the client cluster that can be a vSAN Cluster or a vSphere cluster and we have the remote vSAN cluster. This remote vSAN cluster has a vSAN datastore and the client cluster can mount this datastore and using to create your VMs (Virtual Machines).
The client cluster only is used to provide computing resources (like CPU, Memory, and Network), and all Storage resources are provided by the vSAN datastore (provided by the remote vSAN cluster).
The client cluster can be a vSAN cluster or a vSphere Cluster (traditional cluster with no vSAN features enabled).
Important: The hosts in a client cluster do not require any vSAN license to connect to a server cluster, but they need to reside in a vSphere Cluster to allow a connection (in other words, a Standalone ESXi host cannot mount a vSAN remote datastore). As of vSAN 8.0 U1, for the ESA, the client and server clusters must be managed by the same vCenter Server.
Why vSAN HCI Mesh?
We can have several answers to this question. But I will provide my opinion about it, based on my research on this subject.
So, vSAN has a flexible architecture in which it is possible to scale-up and scale-out with no disruption. Scale-up is to grow the storage capacity by adding more disks on an existing host and Scale-out is to add more hosts into an existing vSAN cluster.
As applications workloads organically grow, this enables performance to be right-sized at each expansion interval. Also, M&A (Mergers and Acquisitions) may result in new business units bringing unforeseen storage requirements to a cluster.
These are examples of scale events that can prove challenging to any architecture and to address these challenges, we can use the vSAN HCI Mesh.
How does VMware vSAN HCI Mesh work?
Firstly, we need to know that the vSAN HCI Mesh is a unique, software-based approach for the disaggregation of compute and storage resources.
Therefore, vSAN allows one or more vSAN clusters to remotely mount datastores from other vSAN clusters (as Server clusters) within the vCenter Inventory.
Important: Further, setup is easy, with built-in health checks to verify the configuration is healthy and supported. So, to read more about it, I would like to recommend the below link:
https://blogs.vmware.com/virtualblocks/2020/09/16/introducing-vmware-vsan-hci-mesh/
In the picture below (from core.vmware.com), we can see a vSAN HCI Mesh Topology example.
The vSAN as a Server Cluster has your Local vSAN Datastore and this local vSAN Datastore is available for remote vSAN clusters (vSAN Client Cluster):
About Our Lab Environment
In this example, we have both clusters with 3 ESXi hosts in each cluster. Below, are the software versions that we are running on each cluster:
— vCenter Server: 8.0.1 build-21560480
— ESXi: 8.0.1 build-21495797
Clusters’ details:
— Cluster-A: vSAN 8 ESA (This is a Server vSAN Cluster)
— Cluster-B: Client Cluster (We will mount a remote vSAN datastore)
Both clusters “Cluster-A” and “Cluster-B” are managed by the same vCenter Server and are in the same data center in the vCenter Server Inventory.
In addition, our “Cluster-A” is the vSAN cluster “server” and our “Cluster-B” is our vSAN cluster “client”. In this example, our vSAN cluster “Cluster-A” is running the vSAN 8 with ESA (Express Storage Architecture) and the vSAN datastore name is “vsanDatastore-ClusterA”.
With vSAN 8 U1, for instance, the ability to mount a remote datastore depends on the fact that the datastore resides in the same vCenter Server Inventory.
In the below picture, we can see more details about our lab environment. We can see both vSphere Cluster and all ESXi hosts under the tab “Hosts & Clusters”:
Mount a remote vSAN datastore
So, the first step is to validate if the vSAN Datastore is configured correctly in the “server cluster”.
After selecting the vSAN cluster “Cluster-A” –> Configure –> vSAN –> Services we can see the vSAN service is enabled and the Express Storage Architecture (ESA) is being used:
Under Fault Domains, we can confirm all hosts that are present in the vSAN Cluster “Cluster-A”:
Under the Datastores tab, we can see all datastores and, of course, the name of the vSAN datastore (in this case, for instance, “vsanDatastore-ClusterA”:
To mount the remote vSAN datastore, go to the cluster-B. We are selecting the “Cluster-B” –> Configure –> vSAN –> Services. Note that in this example, we do not have the vSAN service enabled in this cluster:
Mark the option “I don’t need a local vSAN datastore” and click on CONFIGURE:
Click on APPLY:
After that, “Cluster-B” is ready to mount the remote datastore. Click on “MOUNT REMOTE DATASTORES”. You will be redirected to the Remote Datastores menu. Click on “MOUNT REMOTE DATASTORE”:
The available data store will be listed here. In this case, we are selecting the remote vSAN datastore. Click on NEXT to continue:
A lot of tests will be executed to check the compatibility. Click on FINISH:
Here, we can see the remote vSAN datastore mounted 🙂
The column name “Server Cluster” shows the name of the server cluster and the column name “VM Count” shows the number of VMs stored in this datastore owned by this cluster (in this example, by “Cluster-B”):
When we select the server cluster (from Cluster-A) –> Configure –> vSAN –> Remote Datastores, we have the same view found in the picture above. Under the “Client Clusters” column we can see details about the client clusters, in other words, we can see the names of all client clusters that have been mounted on the vSAN Datastore. In this example, we can see “Cluster-B” as a client cluster:
Creating a Virtual Machine and using the remote datastore
Now, we will create a simple VM and choose the remote vSAN datastore to store this virtual machine Objects:
Type the name of the virtual machine. In this example, the VM name is “slax-Cluster-B”. Click on NEXT to continue:
Select “Cluster-B” as a compute resource cluster for this VM. Click on NEXT to continue:
Select the vSAN datastore as storage for this VM. In this example, the datastore “vsanDatastore-ClusterA” is the remote vSAN datastore provided by the “server” cluster. Click on NEXT to continue:
And click on FINISH to finish the Virtual Machine creation wizard:
Now, under the Remote Datastores menu, we can see that we have one VM in the “VM Count” column:
Selecting the VM –> Configure –> Policies, we can see that this VM is using the vSAN Default Store Policy (inherited by the vSAN datastore default policy):
Under the same VM –> Monitor –> vSAN –> Physical disk placement we have the information that this virtual machine is placed on a remote datastore managed by the “Cluster-A”:
To see placement details for the VM’s objects, click on the “Remote object details” tab:
Look that the ESXi hosts that are storing the VM’s objects are the ESXi hosts from “Cluster-A” and not ESXi hosts from “Cluster-B” (owner of the VM):
Important: if you need to shut down the server cluster for any reason, all clusters using the vSAN datastore provided by this cluster will be impacted. Be careful with that.
vSAN HCI Mesh Maximuns and Requirements
Maximums for vSAN HCI Mesh include:
- A client cluster can mount a max of 10 remote vSAN datastores;
- A server cluster can serve its data store to a maximum of 5 client clusters
- Total of 128 hosts can access the server cluster (client cluster hosts and server cluster hosts).
Requirements for vSAN HCI Mesh include:
- Hybrid or all-flash clusters;
- Latency less than 5 ms (ideally less than 1 ms);
- 25 Gbps or higher is recommended for the network);
- L2/L3 networking is supported.
Unsupported Configurations
The following features are configurable but not supported by VMware:
- vSAN Stretched Clusters;
- Two-node Clusters;
- Cloud Native Storage (CNS);
- vSAN Data-in-Transit Encryption;
- RDMA-enabled clusters;
- VMs spanning multiple data stores (vmdk 1 on VMFS and vmdk 2 on a remote vSAN datastore).
I want to share both links when it is possible to see more details about the vSAN HCI Mesh, Design Considerations, Limitations, etc:
https://core.vmware.com/resource/vmware-vsan-hci-mesh#section1
https://www.yellow-bricks.com/2020/10/07/vsan-hci-mesh-considerations/