For each customer requiring some work to be done on a Windows system, I am running a virtual machine. This ensures that I am not leaking information between my customers and allows me to have just the right software stack installed for each of them.
Last year, I upgraded from Ubuntu 18.04 to 20.04. I am very conservative in my base operating system, this is my work horse, I cannot allow myself to lose data or work with a broken system. This is why I take my time before I upgrade.
At the end, the upgrade was a good thing, except for one thing.
Even if I upgraded VMWare Workstation from 15 to 16, it was slow like hell.
To solve this issue, I had to try so many things that I cannot be sure which one really helped or if this is a combination of all of them which at the end restored correct performance for the system.
3D Acceleration
It looks like VMWare Workstation 16 has issues with the 3D acceleration (I have an Intel integrated GPU on my laptop). I disabled it in the options of the VM. Note that the latest point release of VMware Workstation solved this issue.
Removing Mitigations
At the end, I remembered that with the newer kernels, the mitigations against all the security issues in the Intel CPU were enabled by default. I disabled them to regain a normally efficient laptop. My next laptop will not run with an Intel CPU.
How to disable the mitigations?
Very simple, edit as administrator the file /etc/default/grub.
Find the line with:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
and replace it with:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash mitigations=off"
Then run in the console/terminal:
$ sudo update-grub
And reboot your system.
You now have an unsecure system with respect to the Spectre class of attacks (and some more), but in my case a bit more usable system.
This is not recommended if you run untrusted code on your system. In my case, the less trusted code is running in a virtual machine, it means the code could use side channel attacks to gain knowledge of the main host software, but this is acceptable for my workload.
Switching to Wayland
The last action was to start Gnome not with xorg but with Wayland. Do not ask me to explain the details between Wayland and xorg, but using it (you select the one you want on the login screen) resulted in a generally very snappy desktop. I wonder if this change only could be the only one needed. Once I have the time, I may experiment a bit.
Update: After many trials, Wayland has too many disadvantages, like missing screensharing in MS Teams or special dialogs in some of my applications not showing correctly. I went back to Xorg. I am still not sure how what the best way is.
Latest update: It looks like VMWare 16.2 has now a correct support of 3D acceleration on Intel GPU, this should solve the issues.
Asynchronous IO
As it did not really solved the issue, I started looking at the vmware.log in more details.
If found this line:
2021-11-16T09:43:19.544Z Wa(03) vmx Couldn't initialize aioKernel: dlopen failed for \
libaio.so.1: libaio.so.1: cannot open shared object file: No such file or directory
I removed it by installing libaio:
sudo apt-get install libaio1
We will see if it improves the situation.
Type of Disk
The issue was still not solved. The last trial is to change the type of disk from SCSI to NVMe. For the moment, I am trying only on one VM. I had first to clone the full VM (not a linked clone, a full copy) and then create the custom VM with the source disk being the disk of the clone. It does not create again another disk for the conversion, so you end up with 3 copies of the data, only two.
Kernel Memory Management
It looks like I was able to find something effectively improving the situation, by adding these two lines to my /etc/sysctl.conf file:
vm.compaction_proactiveness=0
vm.swappiness = 5
For reference, the kernel is a 5.13:
$ uname -a
Linux host 5.13.0-28-generic