Introduction to Custom AOSP Kernel Boot Failures on QEMU

Developing custom Android Open Source Project (AOSP) builds for virtual devices like those powered by QEMU, Anbox, or Waydroid offers unparalleled flexibility for testing and development. However, one of the most persistent and frustrating challenges encountered by developers is a kernel that fails to boot. This can manifest in various ways, from a cryptic “Kernel Panic” to a system that simply hangs without any discernible output. This expert-level guide will systematically walk you through diagnosing and resolving common kernel boot failures when running your custom AOSP kernel on QEMU.

Prerequisites for Custom Kernel Development

Before diving into troubleshooting, ensure your environment is correctly set up. A robust AOSP build environment is crucial.

• AOSP Source Tree: A fully synchronized AOSP source tree.
• Build Tools: A functional build environment, typically Ubuntu LTS with necessary packages (Java, Python, Git, etc.).
• QEMU Installation: A recent version of QEMU installed on your host system, preferably compiled with Android-specific virtual hardware support if you’re using older AOSP versions, or at least a standard version that supports generic ARM/x86 emulation.
• Toolchain: The correct cross-compilation toolchain for your target architecture (e.g., aarch64-linux-android- for ARM64).

Understanding the QEMU Boot Process for Android

To effectively troubleshoot, it’s essential to understand how QEMU boots an Android system.

Kernel, Ramdisk, and Kernel Command Line

• Kernel Image: The compiled Linux kernel (Image or zImage) is the core. QEMU loads this binary directly.
• Ramdisk (initramfs): A compressed CPIO archive containing the initial root filesystem. This includes the crucial /init executable (Android’s init process) and init.rc scripts, which are responsible for mounting the real filesystem and launching Android services.
• Kernel Command Line: Parameters passed to the kernel at boot time via QEMU’s -append option. These dictate critical boot behaviors, such as the console to use, the root filesystem location, and various Android-specific properties (e.g., androidboot.console, androidboot.hardware).

Diagnosing Kernel Boot Failures: A Systematic Approach

Capturing and Analyzing Serial Console Output

The most critical first step is to capture the kernel’s boot messages. QEMU allows redirecting the serial console to standard output or a file.

qemu-system-aarch64 
  -machine virt,highmem=off 
  -cpu host 
  -smp 4 
  -m 4G 
  -kernel /path/to/your/aosp/out/target/product/generic_arm64/kernel 
  -initrd /path/to/your/aosp/out/target/product/generic_arm64/ramdisk.img 
  -append "console=ttyAMA0,115200 androidboot.console=ttyAMA0 root=/dev/ram0 rw init=/init" 
  -serial stdio 
  -no-reboot

Look for error messages like “Kernel Panic,” “VFS: Unable to mount root fs,” or messages indicating missing drivers or executables. The `stdio` output will be your primary debugging interface.

Validating Your Kernel Image

Ensure the kernel image you’re providing to QEMU is valid for the target architecture and not corrupted.

file /path/to/your/aosp/out/target/product/generic_arm64/kernel

The output should confirm it’s a Linux kernel image (e.g., “Linux kernel ARM64 EFI executable Image”). If it’s not recognized, your kernel compilation might have failed or produced an incorrect output format.

Inspecting the Ramdisk (initramfs)

A common source of boot failures is an incorrect or corrupted ramdisk. The kernel needs a valid /init executable and an init.rc script to proceed with the boot process.

To inspect the ramdisk, you can extract its contents:

mkdir ramdisk_contents 
cd ramdisk_contents 
cpio -idmv < /path/to/your/aosp/out/target/product/generic_arm64/ramdisk.img

Once extracted, verify:

• Presence of `/init`:

  ls -la init

  This should be a valid executable.

• Correct `init.rc`: Open and check init.rc for proper syntax and that it correctly sets up paths and mounts, especially if you’ve customized it.
• Required Libraries/Binaries: Ensure any custom binaries or libraries needed by /init or init.rc are present.

Scrutinizing the Kernel Command Line (`-append`)

The kernel command line is critical. Incorrect parameters can prevent the kernel from finding the ramdisk, setting up the console, or even launching /init.

• `console=`: Specifies the serial console device (e.g., ttyAMA0 for ARM Virt, ttyS0 for x86). Must match what your kernel expects and what QEMU provides.
• `root=/dev/ram0`: Tells the kernel to use the ramdisk as the root filesystem. Essential for Android’s initial boot.
• `init=/init`: Explicitly points to the /init executable within the ramdisk.
• `androidboot.console=`: Android-specific property for the primary console.

Ensure your `kernel_qemu_common.mk` (or similar for your target) correctly defines these parameters for your AOSP build.

Kernel Configuration Pitfalls for Virtualization

A custom kernel might lack essential drivers or configurations needed for QEMU’s virtual hardware.

• `CONFIG_VIRTIO_*`: Crucial for virtualized I/O (network, block devices). Ensure these are enabled if your `device.mk` or `BoardConfig.mk` uses VirtIO-based devices.
• `CONFIG_ASHMEM`, `CONFIG_BINDER`: Fundamental Android IPC and shared memory mechanisms. Must be enabled.
• `CONFIG_SERIAL_*`: The specific serial driver for your QEMU machine type (e.g., `CONFIG_SERIAL_AMBA_PL011` for ARM `virt` machine, `CONFIG_SERIAL_8250` for x86).
• `CONFIG_DEVTMPFS`, `CONFIG_TMPFS`: Essential for temporary filesystems.

To check your kernel configuration:

grep -E "CONFIG_VIRTIO_BLK|CONFIG_ASHMEM|CONFIG_BINDER" /path/to/your/kernel/build/source/.config

If any crucial configuration is missing, you’ll need to enable it via `make menuconfig` (or equivalent) and recompile your kernel.

Common Failure Scenarios and Solutions

“Kernel Panic – not syncing: VFS: Unable to mount root fs on unknown-block(1,0)”

Cause: The kernel failed to find or mount the ramdisk. This is often due to a malformed ramdisk, a missing `root=/dev/ram0` in the command line, or kernel configuration issues that prevent ramdisk support.

Solution:

• Verify Ramdisk: Re-extract the ramdisk and check its integrity, especially `init` and `init.rc`.
• Command Line: Double-check your `-append` parameters, ensuring `root=/dev/ram0` and `init=/init` are present and correct.
• Kernel Config: Ensure `CONFIG_BLK_DEV_RAM` and `CONFIG_DEVTMPFS` are enabled in your kernel.

“init: ‘/init’ failed to execute: No such file or directory”

Cause: The kernel successfully loaded the ramdisk but couldn’t find or execute the `/init` program. This can be due to a missing `/init`, incorrect permissions, or `/init` being compiled for the wrong architecture.

Solution:

• Check `init` presence: Extract ramdisk and confirm `init` exists at the root.
• Permissions: Ensure `init` has executable permissions (`chmod +x init`).
• Architecture Mismatch: Verify `file init` output matches your kernel’s architecture (e.g., AArch64 executable). If not, your ramdisk generation or `init` compilation is flawed.

Stuck at “Booting Linux…” or “Uncompressing Linux…”

Cause: The kernel itself is having trouble initializing. This can point to an incorrect kernel image for the machine type, memory issues, or very early kernel configuration problems that prevent even basic console output.

Solution:

• Kernel Image: Ensure you’re using the correct `Image` or `zImage` for your target.
• QEMU Machine Type: Verify the `-machine` parameter is appropriate for your AOSP target (e.g., `virt` for generic ARM AOSP).
• Memory: Try adjusting QEMU’s `-m` parameter; sometimes insufficient memory can cause early boot issues.
• Basic Kernel Config: Ensure a minimal set of necessary drivers are compiled directly into the kernel, not as modules, if you’re suspecting very early boot failures.

Advanced Debugging Techniques

Using GDB with QEMU

For deep-seated kernel issues, QEMU can be configured to wait for a GDB connection, allowing you to step through kernel code.

qemu-system-aarch64 ... -s -S

The `-s` (or `-gdb tcp::1234`) option opens a GDB server, and `-S` freezes the CPU at startup. You can then connect GDB to QEMU from another terminal:

aarch64-linux-android-gdb 
  -ex "target remote :1234" 
  -ex "add-symbol-file /path/to/your/kernel/build/source/vmlinux 0xaddress"

Where `0xaddress` is the base address of the kernel image (often available from QEMU logs or kernel build output). This allows inspecting registers, memory, and stepping through kernel startup code.

Conclusion

Troubleshooting QEMU kernel boot failures with custom AOSP builds requires a systematic approach, starting from the QEMU command line and serial output, and meticulously inspecting the kernel image, ramdisk, and kernel configuration. By understanding the interplay between these components and employing tools like CPIO for ramdisk inspection and GDB for deep kernel debugging, you can effectively pinpoint and resolve even the most elusive boot issues, paving the way for successful Android virtualization.