Virtual memory reclamation techniques
Virtual machines perform memory allocation in the same way, as an operating system handles memory allocation and deallocation. The guest operating system frees itself from a piece of physical memory by adding the memory page numbers to the guest free list.
The guest operating system free list is not accessible to the hypervisor and thus it is difficult for the hypervisor to know when to free host physical memory and when the guest physical memory is freed. The hypervisor is completely unaware of which pages are free or allocated in the guest operating system, and because of this the hypervisor cannot reclaim host physical memory when the guest operating system frees guest physical memory.
So VMware hypervisor relies on memory reclamation techniques to reclaim the host physical memory that is freed by the guest operating system. The memory reclamation techniques are:
- Transparent page sharing
- Memory ballooning
- Host-level (or hypervisor) swapping
Getting ready
To step through this recipe, you will need a running ESXi Server, a couple of running virtual machines, and a working installation of vSphere Client. No other prerequisites are required.
How to do it...
Perhaps you don't need to do anything to enable Transparent Memory Page Sharing, as it is by default enabled on your ESXi Hypervisor.
Memory ballooning is driven by the VMware tools. So you need to install the latest version of VMware tools on all of your virtual machines. It will load the vmmemctl driver, which is responsible for memory ballooning.
Perform the following steps:
- Log in to the VMware vSphere Client.
- In the virtual machine inventory, right-click on the virtual machine and click on Install or Upgrade VMware Tools.
- Go to the VM Console and follow the on-screen instruction to install it.
Similarly, Host-level or Hypervisor Swapping is also enabled by default. You don't need to perform any additional steps to enable it.
How it works...
Let us now look at how these techniques work.
Transparent page sharing
When there are multiple virtual machines running on the same hypervisor, most of the time some of them might have identical sets of memory content (known as memory pages). This creates opportunities for sharing memory across virtual machines. ESXi Hypervisor can reclaim the redundant copies and keep only one copy using transparent page sharing. You can think of it as Memory De-duplication.
Traditionally in x86 systems, memory is split into 4 kilobytes pages and that happens only if you are using small pages with shadow pages tables. The TPS process runs every 60 minutes. It scans all memory pages and calculates a hash value for each of them. Those hashes are saved in a separate table and compared to each other by the kernel. Every time the ESXi kernel finds two identical hashes it starts a bit-by-bit comparison of the corresponding memory pages. If these pages are absolutely the same, the kernel leaves only one copy of page in memory and removes the second one. When one of your VM requests to write to this page, the kernel creates a new page because the change made by one VM must not affect the memory contents of another VM.
VMware ESXi scans the guest physical pages randomly, with a base scan rate specified by Mem.ShareScanTime. The maximum number of scanned pages per second in the host and the maximum number of per-virtual machine scanned pages, (that is, Mem.ShareScanGHz and Mem.ShareRateMax respectively) can also be specified in ESXi advanced settings.
Memory ballooning
Memory ballooning makes the guest operating system aware that it does not have enough memory from the host, so that the guest operating system frees some of its memory. When there is a memory crisis, hypervisor tells the balloon driver to request some amount of megabytes from the guest operating system. The hypervisor knows that pages occupied by the balloon driver will never store data, so the pages of pRAM backing the balloon driver can then be reallocated safely to other VMs. It is the guest operating system's call to decide which pages of vRAM it should allocate to the balloon driver, and it will start with free pages. If it has a plenty of free or idle guest physical memory, inflating the balloon will induce no guest-level paging and thus it will not affect guest performance. However, in a case of memory contention within the guest, the VM guest decides which guest physical pages are to be paged out to the virtual swap device in order to satisfy the balloon driver's allocation requests.
The balloon driver reclaims guest operating systems allocated memory using the Idle Memory Tax (IMT). IMT may reclaim up to 75 percent of idle memory. A guest operating system page file is necessary in order to prevent guest operating system kernel starvation. The memctl should aggressively reclaim memory due to severe host contention (make sure that the guest operating system page file is at least 65 percent of the configured vRAM). Even here, the guest operating system can make intelligent guesses about which pages of data are least likely to be requested in the future. (You'll see this in contrast with hypervisor-level swapping, which is discussed next.) Look at the following pictorial representation of memory page mapping to host memory:
Host-level swapping stars in ESXi are not sufficient to reclaim memory during transparent page sharing and ballooning. To support this, when starting a virtual machine, the hypervisor creates a separate swap file for the virtual machine. This is primarily because if it frees pRAM for other virtual machines, the hypervisor can directly swap out vRAM to the swap file.
Swapping is a guaranteed technique to reclaim a specific amount of memory within a specific amount of time. However, you should be concerned about host-level swapping because it can severely penalize guest performance. This occurs when the hypervisor has no knowledge about which guest physical pages should be swapped, and the swapping might cause unintended interactions with the native memory management policies in the guest operating system. The following is a pictorial representation of host-level memory page swapping:
