Zero to Hero: Setting Up Nested Virtualization for Accelerated Android Emulator Development

Introduction: Unlocking Peak Android Emulator Performance with Nested Virtualization

Developing for Android often involves extensive use of emulators. While powerful, traditional emulators can be resource-intensive, leading to slow build times and sluggish app testing. This bottleneck becomes even more pronounced when using solutions like Anbox or Waydroid, which aim to run Android in a containerized Linux environment, frequently leveraging Kernel-based Virtual Machine (KVM) for hardware acceleration. The real power, however, is unleashed when you can run these KVM-accelerated environments inside a virtual machine (VM) itself—a concept known as nested virtualization. This guide will walk you through setting up nested virtualization to supercharge your Android emulator development, focusing on enabling a KVM-accelerated Waydroid instance within a guest VM.

Understanding Nested Virtualization

Nested virtualization allows a hypervisor to run inside a virtual machine, effectively letting a guest VM act as a host to its own set of VMs or containers that require hardware virtualization. For Android emulator development, this means your development environment (a VM) can run a KVM-accelerated Android instance (like Waydroid or Anbox), benefiting from near-native performance even though it’s two layers deep in virtualization. This setup is particularly useful for:

Cloud-based development environments where your primary dev machine is a VM.
Isolated testing environments for Android-in-Linux solutions.
Maximizing resource utilization on powerful bare-metal servers by segmenting them into VMs, each capable of high-performance Android emulation.

The core technology enabling this is Intel VT-x or AMD-V, exposed by the host hypervisor to the guest VM.

Prerequisites and System Checks

Before diving in, ensure your host system meets the following requirements:

CPU Support: Your host CPU must support Intel VT-x (VMX) or AMD-V (SVM) extensions.
BIOS/UEFI Enabled: Virtualization features must be enabled in your host system’s BIOS/UEFI settings.
Host Operating System: A Linux distribution with KVM, QEMU, and libvirt installed (e.g., Ubuntu, Fedora, Debian).
Guest Operating System: A Linux distribution installed within your VM, also with KVM support.

Verifying Host CPU Virtualization Support

On your Linux host, open a terminal and run:

lscpu | grep -E 'vmx|svm'

If you see output containing `vmx` (for Intel) or `svm` (for AMD), your CPU supports virtualization.

Checking KVM Module on Host

Ensure the KVM modules are loaded:

lsmod | grep kvm

You should see `kvm_intel` or `kvm_amd`, and `kvm`.

Step 1: Enabling Nested Virtualization on the Host System

This is the critical first step. You need to instruct your host’s KVM module to expose virtualization capabilities to guest VMs. The process differs slightly for Intel and AMD CPUs.

For Intel CPUs (kvm_intel)

Temporarily enable nested virtualization:

sudo modprobe -r kvm_intel sudo modprobe kvm_intel nested=1

Verify the setting:

cat /sys/module/kvm_intel/parameters/nested

The output should be `Y` or `1`.

For AMD CPUs (kvm_amd)

Temporarily enable nested virtualization:

sudo modprobe -r kvm_amd sudo modprobe kvm_amd nested=1

Verify the setting:

cat /sys/module/kvm_amd/parameters/nested

The output should be `Y` or `1`.

Making Nested Virtualization Persistent

To ensure nested virtualization is enabled after a reboot, create a configuration file:

sudo nano /etc/modprobe.d/kvm-nested.conf

Add the following line based on your CPU:

Intel: options kvm_intel nested=1
AMD: options kvm_amd nested=1

Save and close the file. You might need to regenerate your initramfs for changes to take full effect, though often a reboot is sufficient:

sudo update-initramfs -u

Step 2: Configuring the Guest VM for Nested KVM

Now, you need to create or modify your guest VM to expose the nested virtualization capabilities. The key is to properly configure the CPU type and features in the VM’s XML definition.

Creating a New VM with Nested KVM Support

When creating a new VM using `virt-install`, you can specify CPU details:

sudo virt-install 	--name my-nested-android-vm 	--ram 8192 	--vcpus 4,maxvcpus=8 	--os-variant ubuntu22.04 	--disk path=/var/lib/libvirt/images/my-nested-android-vm.qcow2,size=60 	--location 'http://us.archive.ubuntu.com/ubuntu/dists/jammy/main/installer-amd64/' 	--network bridge=virbr0 	--graphics spice 	--cpu host-passthrough,feature=pmu=off 	--wait -1

The crucial part is `–cpu host-passthrough,feature=pmu=off`. `host-passthrough` attempts to expose the host CPU’s capabilities directly to the guest. `feature=pmu=off` is often necessary to avoid issues with performance monitoring unit (PMU) emulation inside nested environments.

Modifying an Existing VM for Nested KVM Support

If you have an existing VM, you’ll need to edit its XML configuration using `virsh edit`:

sudo virsh edit my-nested-android-vm

Locate the “ section and modify it to look like this:

Here’s what each part means:

`mode=’host-passthrough’`: Tells QEMU to pass through the host CPU features as much as possible. `check=’partial’` allows for some discrepancies.
“: Disables PMU emulation, which can be problematic in nested setups.
“ (or `svm` for AMD): Explicitly requires the guest VM to see the CPU’s hardware virtualization extensions.
“: Ensures the guest believes it’s running on a hypervisor.

Save the XML and then restart your VM for the changes to take effect:

sudo virsh shutdown my-nested-android-vm sudo virsh start my-nested-android-vm

Step 3: Verifying Nested KVM within the Guest VM

Once your guest VM is running with the updated configuration, log in to the guest OS and perform these checks:

Check for KVM Modules

lsmod | grep kvm

You should see `kvm_intel` or `kvm_amd`, and `kvm`.

Verify CPU Flags

grep -E 'vmx|svm' /proc/cpuinfo

Crucially, you should see `vmx` or `svm` flags listed in the output, indicating that the guest VM perceives hardware virtualization support.

Run `kvm-ok` (If Available)

If `qemu-kvm` or `cpu-checker` package is installed in your guest, you can run `kvm-ok`:

kvm-ok

It should report: `INFO: /dev/kvm exists` and `INFO: KVM acceleration can be used`.

Step 4: Setting Up Waydroid for Accelerated Development

Now that your guest VM has nested KVM enabled, you can install and run Waydroid, benefiting from hardware acceleration.

Install Waydroid in the Guest VM

For Ubuntu/Debian-based guests:

sudo apt update sudo apt install curl ca-certificates -y curl https://repo.waydro.id/waydroid.gpg --output /usr/share/keyrings/waydroid.gpg sudo echo
        
        
        
            
                
            
            
                Android Mobile Specs & Compare Directory
                Are you researching mobile hardware properties, processor SoCs, GPU chipsets, or RAM configurations? Access our complete specs catalog to compare up to 5 devices side-by-side!
                Compare Devices Specs →