Speeding Up Virtual Android: Performance Tuning Your Custom AOSP Kernel for QEMU

Introduction: The Quest for Performant Virtual Android

Running Android in a virtualized environment like QEMU, whether for development, testing, or solutions like Anbox and Waydroid, often presents significant performance challenges. The default generic kernels shipped with AOSP or various distributions are rarely optimized for virtual hardware, leading to sluggish UI, slow boot times, and overall poor responsiveness. This article delves into the expert-level process of compiling and tuning a custom Android Open Source Project (AOSP) kernel specifically for QEMU, unlocking substantial performance gains for your virtual Android instances.

Understanding kernel compilation for AOSP on QEMU is a critical skill for anyone pushing the boundaries of Android virtualization. We’ll cover everything from setting up the build environment to deep-diving into kernel configurations and integrating your optimized kernel.

The Case for a Custom AOSP Kernel on QEMU

Why bother with a custom kernel? The default kernels are designed for broad compatibility, not peak performance in a virtualized setting. A custom kernel allows us to:

Enable VirtIO Drivers: Crucial for efficient communication between the guest OS (Android) and the host’s hardware, offering near-native performance for disk I/O, networking, and graphics.
Optimize for KVM: If your host supports Kernel-based Virtual Machine (KVM), a custom kernel can be configured to take full advantage of hardware virtualization extensions.
Tailor CPU Governors & I/O Schedulers: Fine-tune how the kernel manages CPU frequency and disk access to prioritize performance over power saving, which is often irrelevant in a desktop VM scenario.
Strip Unnecessary Modules: Reduce kernel size and attack surface by removing drivers and features not needed for a virtualized environment, improving boot speed and memory footprint.

Setting Up Your Build Environment

Before diving into kernel compilation, you need a robust build environment. A Linux distribution like Ubuntu LTS is highly recommended.

Prerequisites:

Ensure you have essential development tools and libraries installed:

sudo apt update && sudo apt upgrade -y
sudo apt install -y git-core gnupg flex bison build-essential zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 libncurses5 lib32ncurses5-dev x11proto-core-dev libx11-dev libgl1-mesa-dev libxml2-utils xsltproc rsync bc ccache make openssl libssl-dev libelf-dev clang lld device-tree-compiler python3-pip android-sdk-platform-tools
sudo apt install -y openjdk-11-jdk # Or openjdk-17-jdk for newer AOSP versions

For AOSP source management, install the `repo` tool:

mkdir ~/bin
PATH=~/bin:$PATH
curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo

Obtaining the AOSP Kernel Source

AOSP kernels are typically maintained in separate Git repositories. You’ll use `repo` to fetch the specific kernel tree relevant to your AOSP version or target device.

First, initialize your AOSP workspace (if you haven’t already):

mkdir ~/aosp && cd ~/aosp
repo init -u https://android.googlesource.com/platform/manifest -b android-13.0.0_rXX # Replace with your desired AOSP branch
repo sync -j8

Next, identify the kernel source. For generic virtual devices or GSI (Generic System Image) builds, the `common` kernel project is a good starting point. For QEMU, specifically, you might look at `kernel/configs` for `qemu_defconfig` or similar.

cd ~/aosp
# Fetching a common kernel like the 'android-mainline' or specific 'qemu' kernel source
# AOSP's QEMU kernels are often prebuilts, but we can grab a generic one and configure.
# Example for common kernel: android-msm-pixel-4.4 for older versions, or generic/goldfish for emulation.
# Let's assume we're using a newer common kernel for building a GSI-like image.
# You might need to add a local manifest for the specific kernel source.
# For modern AOSP, kernels are often built with GKI (Generic Kernel Image) principles.

# Let's clone a recent 'common' kernel (e.g., from Android 13/14 branch) for demonstration
# It's often located in a 'kernel' directory relative to AOSP root or fetched via a dedicated manifest.
# Example: git clone https://android.googlesource.com/kernel/common.git -b android-mainline kernel/common

# For simplicity, let's assume a pre-existing kernel source at 'kernel/common'
# Or, for virtual devices, often 'kernel/goldfish' or kernels designed for GSI like 'kernel/configs'.
# Let's target 'kernel/configs' for GSI kernels, which often have good QEMU support.

mkdir -p kernel/configs
cd kernel/configs
repo init -u https://android.googlesource.com/kernel/manifest -b android-13-gsi
repo sync -j8 # This will fetch kernel sources relevant for GSI, which includes qemu support.
cd ..

Configuring the Kernel for QEMU/Virtualization

The heart of performance tuning lies in kernel configuration. We’ll start with a base configuration and then customize it.

Navigate to Kernel Source: Assuming your kernel source is in `kernel/common` or `kernel/configs/common`.

cd ~/aosp/kernel/configs/common # Adjust path if different

Set Up Environment Variables: Define your architecture and cross-compiler path.

export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-android-
# Assuming you have the Android NDK prebuilts for the toolchain
# Locate your NDK: e.g., NDK_ROOT=~/aosp/prebuilts/ndk/current
export PATH=~/aosp/prebuilts/clang/host/linux-x86/clang-r416183b/bin:$PATH # Adjust clang path
# Or, for a full AOSP build, the build environment usually sets this up.
# If building standalone, you might need to download a standalone toolchain.
# For AOSP, clang is preferred. Example for NDK toolchain:
# export CROSS_COMPILE_ARM64=~/aosp/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/aarch64-linux-android-

Load a Base Configuration: Start with a QEMU-specific or generic virtual configuration.

make qemu_defconfig # Or 'gsi_defconfig' if available and suitable for virtual targets

Customize with `menuconfig` (The Core Tuning Step):

make menuconfig

Inside `menuconfig`, navigate and enable/disable the following (using spacebar to select/deselect, Enter to enter sub-menus):

General setup —>
- `Optimize for size` (N) – Prioritize performance.
- `Local version – append to kernel release` (e.g., `-qemu-perf-v1`) – Helps identify your kernel.
Processor type and features —>
- `KVM guest support` (Y) – Essential for KVM acceleration.
- `Preemption Model` (Preemptible Kernel (Low-Latency Desktop)) – Improves UI responsiveness.
- `Timer frequency` (1000 Hz) – Higher frequency can reduce latency.
Device Drivers —>
- Virtio drivers —> (Enable all relevant ones as modules “ or built-in “)
  - `Virtio block driver` (Y)
  - `Virtio network driver` (Y)
  - `Virtio console driver` (Y)
  - `Virtio GPU driver` (Y) – Crucial for graphics performance.
  - `Virtio input driver` (Y)
- Block devices —>
  - Ensure `Virtio block driver` is enabled here as well.
Filesystems —>
- `Ext4 filesystem support` (Y)
- `F2FS filesystem support` (Y) – Common for Android.
Kernel Hacking —>
- `Magic SysRq key` (Y) – Useful for debugging.
Power management and ACPI options —>
- CPU Frequency scaling —>
  - `’ondemand’ governor` (M/Y)
  - `’performance’ governor` (M/Y) – Often ideal for VMs to prevent CPU downscaling. Set as default if possible.
Block layer —>
- IO Schedulers —>
  - `CFQ I/O scheduler` (Y) – Older default, good general purpose.
  - `Deadline I/O scheduler` (Y) – Good for databases, sometimes better for VMs.
  - `MQ-deadline I/O scheduler` (Y) – Modern, often best for NVMe/SSDs and virtualized setups.
  - Set `Default I/O scheduler` to `mq-deadline` or `deadline`.

Save your new configuration as `.config` and exit `menuconfig`.

Compiling Your Custom Kernel

With your optimized `.config` in place, it’s time to compile. This process can take a significant amount of time depending on your hardware.

# Go back to the root of your kernel source tree (e.g., ~/aosp/kernel/configs/common)

# Clean previous builds
make clean
make mrproper

# Compile the kernel image and device tree blobs (DTBs)
# Use -jN where N is the number of CPU threads + 1 or 2 for faster compilation.
make -j$(nproc) Image.gz dtbs
# For newer kernels, it might be just 'make -j$(nproc) Image dtbs' or 'make -j$(nproc) bzImage dtbs'
# The output will typically be in arch/arm64/boot/Image.gz and arch/arm64/boot/dts/qcom/YOUR_BOARD.dtb (or generic-arm64.dtb)

# If you need modules (e.g., virtio as modules, though built-in is often better for QEMU)
# make -j$(nproc) modules
# make modules_install INSTALL_MOD_PATH=./out/modules_install

Upon successful compilation, your kernel image (`Image.gz` or `Image`) and device tree blobs (`.dtb` files) will be located in `arch/arm64/boot/` and `arch/arm64/boot/dts/`, respectively.

Integrating the Custom Kernel with QEMU

You have two main ways to use your custom kernel with QEMU:

1. Direct QEMU Boot:

This is simpler for testing. You’ll specify your kernel and a ramdisk (e.g., `ramdisk.img` from an AOSP build or GSI).

# Example QEMU command. Adjust paths and parameters as needed.
# Assuming your compiled kernel is at ~/aosp/kernel/configs/common/arch/arm64/boot/Image.gz
# And you have a ramdisk.img from an AOSP build (e.g., out/target/product/generic_arm64/ramdisk.img)
# And a system.img, product.img, vendor.img

QEMU_PATH=~/aosp/prebuilts/qemu/linux-x86_64/qemu-system-aarch64
KERNEL_IMAGE=~/aosp/kernel/configs/common/arch/arm64/boot/Image.gz
RAMDISK_IMAGE=~/aosp/out/target/product/generic_arm64/ramdisk.img
SYSTEM_IMAGE=~/aosp/out/target/product/generic_arm64/system.img
VENDOR_IMAGE=~/aosp/out/target/product/generic_arm64/vendor.img
PRODUCT_IMAGE=~/aosp/out/target/product/generic_arm64/product.img

$QEMU_PATH 
    -M virt 
    -cpu host 
    -enable-kvm 
    -smp 4 
    -m 4G 
    -kernel $KERNEL_IMAGE 
    -initrd $RAMDISK_IMAGE 
    -append "console=ttyAMA0,115200 root=/dev/vda androidboot.console=ttyAMA0 androidboot.hardware=qemu_arm64" 
    -device virtio-blk-pci,drive=system 
    -drive if=none,id=system,file=$SYSTEM_IMAGE,format=raw 
    -device virtio-blk-pci,drive=vendor 
    -drive if=none,id=vendor,file=$VENDOR_IMAGE,format=raw 
    -device virtio-blk-pci,drive=product 
    -drive if=none,id=product,file=$PRODUCT_IMAGE,format=raw 
    -device virtio-net-pci,netdev=user0 
    -netdev user,id=user0,hostfwd=tcp::5555-:5555 
    -device virtio-gpu-pci 
    -usb -device usb-mouse -device usb-kbd

2. Integrating into AOSP Build (for `emulator` command):

If you want `lunch` and `emulator` commands to pick up your kernel, you’ll need to replace the prebuilt QEMU kernel in your AOSP tree.

Copy your compiled `Image.gz` to the relevant prebuilt location:

cp ~/aosp/kernel/configs/common/arch/arm64/boot/Image.gz ~/aosp/prebuilts/qemu-kernel/arm64/Image.gz

Rebuild AOSP system images:

cd ~/aosp
source build/envsetup.sh
lunch aosp_arm64-userdebug # Or your specific AOSP target
m_img # Or `make` if you want to rebuild everything. This specifically rebuilds system images with the new kernel.

Now, when you run `emulator`, it should use your custom kernel.

emulator -avd avd_name # Or just `emulator` if you've set up your AVD

Testing and Benchmarking Performance

Once your custom kernel is running, observe the changes. Pay attention to:

Boot Time: Compare with the stock kernel.
UI Responsiveness: Navigate through menus, open apps.
App Launch Times: How quickly do common apps (Browser, Settings) open?
Benchmark Scores: Run Android benchmarks like AnTuTu, Geekbench, or PCMark inside the VM and compare scores.
`adb shell top` or `htop` on host: Monitor CPU usage.
`adb shell iostat` or `vmstat`: Check I/O performance.

Experiment with different CPU governors (`performance` vs `ondemand`) and I/O schedulers to find the optimal balance for your host machine and workload.

Further Optimizations

KVM: Always ensure KVM is enabled on your host and QEMU is started with `-enable-kvm`.
QEMU CPU Type: Use `-cpu host` to pass through your host CPU features.
Memory Allocation: Allocate sufficient RAM (e.g., `-m 4G`) to the VM.
Disk Image Format: Use `raw` format for your QEMU disk images for best performance, especially with `virtio-blk-pci`.
Shared Folders: Consider using `virtio-9p` for efficient host-guest file sharing.

Conclusion

Compiling and tuning a custom AOSP kernel for QEMU is an advanced but highly rewarding endeavor. It empowers you to overcome the inherent performance limitations of generic virtual environments, transforming a sluggish virtual Android experience into a highly responsive and efficient one. By carefully configuring kernel options like VirtIO drivers, CPU governors, and I/O schedulers, you gain fine-grained control over your virtual machine’s performance characteristics. This deep dive into kernel internals not only speeds up your virtual Android but also provides invaluable insight into the Linux kernel and Android’s underlying architecture, making you a more capable developer or system integrator in the realm of Android virtualization.

Android Mobile Specs & Compare Directory

Are you researching mobile hardware properties, processor SoCs, GPU chipsets, or RAM configurations? Access our complete specs catalog to compare up to 5 devices side-by-side!

Compare Devices Specs →