The maximum number of vCPUs per VM is the configuration for a single VM. You can assign more vCPUs than a host has for different VMs (CPU overcommitment). This configuration might have performance issues.
When migrating from on-premises, make sure that you are within the specified limit. You must adjust the number of vCPUs to match the maximum number available on the host to be able to start a VM.
- Cores per socket setting
From vSphere 5.0 onward, you can control how vCPUs are presented to a VM as virtual sockets or virtual cores. This configuration might affect some applications (such as SQL Server and Oracle) that have their own CPU schedulers. As a best practice, always match the number of virtual sockets to the number of physical sockets in your system. On VMware Cloud on AWS, all host types currently have a two-socket configuration. Configure your VM to have no more than two virtual sockets. If the total number of vCPUs fits within a single physical socket, expose just a single virtual socket.
Figure 11.1 – Virtual Hardware settings – validate Cores per Socket
Note
This best practice is still applicable to the new vNUMA calculation algorithm.
Let’s review the following examples:
- A VM with 16 vCPUs running on any host type should be configured with 1 vSocket/16 cores, as shown in Figure 11.1
- A VM with 24 vCPUs running on an i3 host should be configured with 2 vSockets/12 cores; when running on i3en or i4i, there is 1 vSocket/24 cores
- A VM with 72 vCPUs running on any supported host type should be configured with 2 vSockets/36 cores (you cannot configure 72 vCPUs on i3)
- vNUMA
It’s a best practice to recheck the vNUMA configuration after you migrate VMs to a new hardware platform because the underlying physical NUMA configuration of the host types might be different.
The following table outlines the physical NUMA configuration of VMware Cloud on AWS host types. As a best practice, a VM with a number of vCPUs lower or equal to the number of cores in a single physical NUMA node should be configured as a single NUMA VM.
| Instance Type | NUMA Configuration | Max vCPUs per VM to Fit into a Single Physical NUMA Node |
| i3.metal | 2 nodes, 18 cores, 256 GB RAM | 18 |
| i3en.metal | 2 nodes, 48 cores, 384 GB RAM | 48 |
| I4i.metal | 2 nodes, 64 cores, 512 GB RAM | 64 |
Table 11.2 – VMware Cloud on AWS host physical NUMA CPU configuration
Let’s review the following examples:
- A VM with 16 vCPUs running on an i3 host should have only a single virtual NUMA node
- A VM with 64 vCPUs running on an i3en host should have two virtual NUMA nodes
- A VM with 64 vCPUs running on an i4i host should have only a single virtual NUMA node
For a complex configuration, where a VM requires more memory than a single physical NUMA node has, but the number of vCPUs fits into a single NUMA node, refer to the detailed NUMA configuration discussed in the VMware documentation (https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/solutions/sql-server-on-vmware-best-practices-guide.pdf).
To validate the current virtual NUMA configuration, you can use in-guest tools or use a PowerShell script (https://github.com/vbondzio/sowasvonunsupported/blob/master/Get-vNUMA-config.ps1). VMware support might be able to assist in retrieving the vNUMA configurations for VMs running on VMware Cloud on AWS.
If you need to change the vNUMA, simply add or remove vCPUs and reboot the virtual machine – the underlying physical NUMA configuration of the host will be used to recreate the vNUMA configuration for the VM.
- CPU Hot Plug (Hot Add)
The CPU Hot Add feature allows us to quickly adjust the number of vCPUs without needing to reboot a VM.
Figure 11.2 – CPU Hot Plug configuration
While this feature might be useful from an automation or test and dev perspective, it also has a known performance impact. A VM with CPU Hot Add enabled does not have vNUMA configuration, which might lead to unbalanced memory access across NUMA nodes. If you are looking for the best performance for your applications, we recommend disabling CPU hot-add for all VMs.