Introduction

Running Android in a virtualized environment on Linux, whether through full-fledged emulators or container solutions like Anbox and Waydroid, often presents performance challenges. While Kernel-based Virtual Machine (KVM) offers near-native performance for CPU and memory, the guest operating system’s kernel still plays a crucial role in overall responsiveness, especially concerning graphics, I/O, and specialized hardware interactions. This article delves into establishing a robust benchmarking lab to systematically validate custom KVM guest kernel modifications aimed at enhancing Android’s performance.

We will explore common bottlenecks, discuss potential kernel-level optimizations, and provide a step-by-step guide to setting up a testing environment, modifying the Android guest kernel, deploying it, and rigorously benchmarking the changes. The goal is to provide a reproducible methodology for engineers and enthusiasts seeking to squeeze every bit of performance out of virtualized Android.

Understanding KVM and Android Virtualization Architectures

KVM, a virtualization infrastructure built into the Linux kernel, allows a host machine to run multiple virtual machines (VMs) with minimal overhead. It leverages hardware virtualization extensions (Intel VT-x or AMD-V) to provide direct access to the CPU for the guest OS, making it exceptionally fast for CPU-bound tasks. However, non-CPU interactions, such as disk I/O, network, and especially graphics, rely on paravirtualized devices (e.g., VirtIO) or emulation, which can introduce latency.

Android virtualization layers like Anbox and Waydroid typically run a full Android system or a subset of its userspace on a Linux container (LXC) or a dedicated KVM VM, often sharing the host’s kernel or utilizing a purpose-built guest kernel. Optimizing this guest kernel is paramount for achieving desktop-like fluidity. Performance is highly dependent on efficient VirtIO drivers, optimized memory management, and responsive scheduling within the guest kernel.

Identifying Performance Bottlenecks in Virtualized Android

Before diving into modifications, it’s essential to pinpoint where performance is lacking. Common areas include:

• Graphics Rendering: Frame drops, stuttering in UI, and low FPS in games are often due to inefficient GPU virtualization or suboptimal VirtIO-GPU drivers.
• Disk I/O: Slow app launches, sluggish file operations, and general system unresponsiveness can stem from I/O scheduler choices, virtio-blk configurations, or underlying host storage performance.
• CPU Scheduling: Latency-sensitive Android applications require prompt CPU access. Inefficient guest scheduling can lead to UI jank even with ample host CPU resources.
• Memory Management: Excessive swapping or inefficient memory reclamation within the guest kernel can degrade performance.

Initial profiling can be done using Android’s built-in developer options (e.g., Profile GPU rendering, systrace), adb shell top, and host-side tools like perf or htop to monitor KVM processes.

Proposed Kernel Modifications for Performance

VirtIO-GPU Enhancements

The virtio-gpu driver in the guest kernel is critical. Optimizations might involve:

• Faster Context Switching: Reducing overhead when switching between host and guest rendering contexts.
• Direct Rendering Interface (DRI) Improvements: Ensuring efficient communication for 3D acceleration.
• Buffer Management: Tuning buffer allocation and deallocation to minimize copies and latency.

I/O Scheduler Tuning

For virtualized block devices (e.g., /dev/vda), the I/O scheduler can significantly impact performance. Common choices:

• noop: A simple FIFO queue, often best for SSDs and virtual environments where the host scheduler handles complex optimizations.
• deadline: Prioritizes requests by their expiration deadline, good for latency-sensitive applications.
• mq-deadline: A multi-queue version of deadline, suitable for modern NVMe devices.

You can change the I/O scheduler for a device on the guest with:

echo noop > /sys/block/vda/queue/scheduler

Memory Management Optimizations

• KSM (Kernel Samepage Merging): While good for memory utilization, KSM can introduce CPU overhead. Tuning its parameters or disabling it might be beneficial in performance-critical scenarios.
• Swappiness: Adjusting vm.swappiness can control how aggressively the kernel uses swap space. A lower value (e.g., 10) can keep more data in RAM.

sysctl -w vm.swappiness=10

CPU Scheduler Tweaks

The Completely Fair Scheduler (CFS) can be tuned. For a guest, ensuring sufficient CPU bandwidth and reducing latency for Android’s UI thread is crucial. While direct scheduler patches are complex, monitoring scheduler latency can reveal issues.

Setting Up the Benchmarking Lab

Host Environment Setup (Ubuntu 22.04 LTS Example)

Install KVM/QEMU and related tools:

sudo apt update sudo apt install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virtinst virt-manager

Add your user to the libvirt and kvm groups:

sudo usermod -aG libvirt $(whoami) sudo usermod -aG kvm $(whoami)

Log out and back in for group changes to take effect.

Android Guest Kernel Source and Build Environment

To modify the kernel, you need the Android Common Kernel source or the specific kernel used by Anbox/Waydroid. For AOSP:

mkdir android-kernel cd android-kernel repo init -u https://android.googlesource.com/kernel/manifest -b android-5.15-lts repo sync -j$(nproc)

Install cross-compilation tools:

sudo apt install gcc-aarch64-linux-gnu make bison flex libssl-dev libelf-dev build-essential

Step-by-Step Kernel Modification & Deployment

1. Apply Your Kernel Changes

Navigate to your kernel source directory. For example, to make a simple change for demonstration, you might edit a file like drivers/block/virtio_blk.c (though real performance changes are more involved). Let’s assume you’ve identified a patch or a configuration change. Apply it:

# Example: Hypothetical patch apply patch -p1 < your_optimization.patch

Or, directly modify source files, e.g., to adjust an I/O scheduler default or add debug prints.

2. Configure and Compile the Kernel

Choose an architecture (e.g., arm64 for most Android guests):

ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make defconfig ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make menuconfig # Apply desired config changes, e.g., enabling KSM or specific virtio options ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make -j$(nproc)

This will generate arch/arm64/boot/Image (or Image.gz) and potentially a dtb (Device Tree Blob) if needed.

3. Prepare a Ramdisk and Launch the Guest

You’ll need an Android ramdisk (ramdisk.img) matching your Android version. If building AOSP, it’s generated during the AOSP build. Otherwise, extract it from an existing Android boot image.

Launch QEMU with your custom kernel:

qemu-system-aarch64 -enable-kvm -m 4G -smp 4 -cpu host -M virt -kernel /path/to/your/Image -initrd /path/to/your/ramdisk.img -append "console=ttyS0 root=/dev/vda androidboot.selinux=permissive" -drive file=/path/to/android.img,if=virtio,format=raw -device virtio-gpu -device virtio-keyboard -device virtio-mouse -serial stdio -usb -device usb-host,hostbus=1,hostaddr=2 # Optional: for USB passthrough

Replace `/path/to/your/Image`, `/path/to/your/ramdisk.img`, and `/path/to/android.img` with your actual file paths. The android.img would be your Android root filesystem or data partition.

Benchmarking Methodology

For consistent and reliable results:

1. Baseline Measurement: Always benchmark your unmodified guest kernel first.
2. Multiple Runs: Perform each benchmark multiple times (e.g., 5-10 runs) and calculate averages, discarding outliers.
3. System State: Ensure the guest is in a consistent state (e.g., fresh boot, no background apps) before each run.

Key Benchmarks:

1. Graphics Performance

• glmark2-es2: A cross-platform benchmark for OpenGL ES 2.0. Install via adb shell after pushing the binary.
• gfxbench: Comprehensive graphics benchmark for Android (available on Google Play Store).
• Manual FPS Measurement: For specific apps, use Android Developer Options’ “Profile GPU rendering” or external tools like Gamebench.

2. Disk I/O Performance

• fio: The Flexible I/O Tester. Push the ARM binary to the guest and run for various read/write patterns (sequential, random, block sizes).

adb push fio /data/local/tmp/ adb shell "cd /data/local/tmp && ./fio --name=test --ioengine=libaio --rw=randrw --bs=4k --size=1G --numjobs=4 --iodepth=16 --group_reporting"

• AndroBench / PCMark for Android: Offer synthetic I/O scores.

3. CPU Performance

• Geekbench 6: Industry-standard CPU benchmark for single-core and multi-core performance.
• AnTuTu Benchmark: Provides an overall system score, including CPU.
• sysbench: Push binary to guest and run CPU tests.

adb push sysbench /data/local/tmp/ adb shell "cd /data/local/tmp && ./sysbench cpu --cpu-max-prime=20000 run"

4. Network Performance

• iperf3: Measure throughput between host and guest. Run iperf3 -s on the host and iperf3 -c <host_ip> on the guest.

Analyzing Results and Iteration

Once you have benchmark data for both the baseline and modified kernels, compare the metrics. Look for statistically significant improvements or regressions.

• Graphical Data: Plot FPS, latency, and scores to visualize differences.
• Host-Side Monitoring: Use perf top -p <qemu_pid> on the host to see if your kernel changes reduced CPU usage related to specific virtio drivers or KVM itself.
• Guest Logs: Monitor dmesg and logcat for any errors or warnings introduced by your kernel modifications.

Kernel optimization is an iterative process. Small, targeted changes, followed by thorough testing, are more effective than large, untracked modifications. If a change doesn’t yield the expected performance gain, revert it or try a different approach.

Conclusion

Establishing a systematic benchmarking lab for KVM guest kernel modifications is essential for truly optimizing Android’s performance in virtualized environments. By understanding bottlenecks, applying targeted kernel changes, and rigorously validating them with appropriate benchmarks, developers can unlock significant performance gains. This methodology empowers you to transform a standard virtualized Android instance into a highly responsive, near-native experience, ideal for development, testing, or daily use in solutions like Anbox and Waydroid.