Introduction to eBPF on Android

The Extended Berkeley Packet Filter (eBPF) has revolutionized kernel-level tracing, networking, and security across various Linux distributions. Its ability to run sandboxed programs directly within the kernel, triggered by various events, offers unparalleled introspection and performance. While eBPF’s adoption is widespread in server environments, its integration and utilization within the Android ecosystem present unique challenges and opportunities. This hands-on tutorial guides you through building a custom Android kernel with eBPF support and deploying a simple event monitor.

Android, being a heavily customized Linux distribution, requires specific considerations for eBPF development. Key challenges include dealing with a diverse range of kernel versions, often outdated eBPF toolchains, and strict security policies like SELinux. However, the insights gained from direct kernel monitoring using eBPF can be invaluable for performance optimization, security analysis, and debugging complex system behaviors on Android devices.

Setting Up the Android Build Environment

Before we can build our custom kernel, we need a complete Android Open Source Project (AOSP) build environment. This ensures we have the correct toolchains, headers, and build scripts. For this tutorial, we’ll assume a standard Linux development machine (Ubuntu 20.04+ recommended).

1. Syncing AOSP Source

First, set up your AOSP environment. Choose a stable branch (e.g., Android 12 or 13) that typically offers better eBPF support. This process can take several hours depending on your internet connection.

mkdir -p ~/android/aosp
cd ~/android/aosp
repo init -u https://android.googlesource.com/platform/manifest -b android-13.0.0_r40
repo sync -j8

2. Identifying Your Device’s Kernel Source

Android kernels are often prebuilt and device-specific. You’ll need the kernel source that matches your target device (or a Generic Kernel Image, GKI, if applicable). The kernel source for AOSP builds is typically located under `aosp/kernel/common` or `aosp/device/manufacturer/device-name/kernel`. For simplicity, we’ll work with `kernel/common` and assume a `gki_arm64` configuration.

cd ~/android/aosp/kernel/common

Enabling eBPF in the Kernel Configuration

Most modern Android kernels support eBPF, but it might not be fully enabled or configured for your specific needs. We need to modify the kernel `.config` file.

1. Locating the Kernel Configuration

The kernel configuration files are usually found in `arch/arm64/configs/` (for ARM64 devices). We’ll modify a common GKI configuration. For example:

cp arch/arm64/configs/gki_arm64_defconfig .config
make menuconfig

During `make menuconfig`, navigate and enable the following options. Use the search function (`/`) to find them quickly:

• CONFIG_BPF_SYSCALL: Enable eBPF system call.
• CONFIG_BPF_JIT: Enable eBPF JIT compiler (for performance).
• CONFIG_BPF_EVENTS: Enable kprobes/uprobes, tracepoints for eBPF.
• CONFIG_DEBUG_INFO_BTF: Enable BTF (BPF Type Format) for richer debugging info.
• CONFIG_KPROBE_EVENTS, CONFIG_UPROBE_EVENTS, CONFIG_FTRACE, CONFIG_FUNCTION_GRAPH_TRACER: Essential tracing infrastructure.

After saving your changes, the `.config` file will be updated.

Compiling a Custom Android Kernel

With the eBPF options enabled, we can now compile our custom kernel. Ensure you have the AOSP prebuilts toolchain in your PATH.

1. Setting Up Build Variables

export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-android-
export PATH=~/android/aosp/prebuilts/clang/host/linux-x86/clang-r450784d/bin/:$PATH # Adjust path to your AOSP clang toolchain
export LLVM=1 # Use LLVM for kernel build

2. Building the Kernel

Now, build the kernel image and modules:

make -j$(nproc)

Upon successful compilation, you should find your kernel image (e.g., `Image.gz`) in `arch/arm64/boot/` and potentially a `boot.img` or `init_boot.img` if you follow device-specific build instructions. This tutorial assumes you’ll integrate this kernel into a full AOSP build or flash it separately.

Developing an eBPF Program: Monitoring `sys_enter`

Let’s create a simple eBPF program that monitors every system call entry (`sys_enter`) and records the system call ID and process ID into a BPF map. We’ll write this in C and compile it using `clang` with eBPF targets.

1. eBPF C Program (`syscall_monitor.bpf.c`)

#include <vmlinux.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>

// Define a BPF map to store syscall counts
struct {
    __uint(type, BPF_MAP_TYPE_HASH);
    __uint(max_entries, 1024);
    __type(key, u32);
    __type(value, u64);
} syscall_counts SEC(".maps");

// Define a BPF map to store syscall event logs
struct syscall_event {
    u32 pid;
    u32 syscall_id;
};

struct {
    __uint(type, BPF_MAP_TYPE_RINGBUF);
    __uint(max_entries, 256 * 1024); // 256 KB buffer
} events SEC(".maps");

// Tracepoint handler for sys_enter
SEC("tp/raw_syscalls/sys_enter")
int handle_sys_enter(struct bpf_raw_tracepoint_args *ctx)
{
    u32 syscall_id = ctx->args[1];
    u64 pid_tgid = bpf_get_current_pid_tgid();
    u32 pid = pid_tgid >> 32; // Get PID from pid_tgid

    // Increment syscall count
    u64 *count = bpf_map_lookup_elem(&syscall_counts, &syscall_id);
    if (count) {
        *count += 1;
    } else {
        u64 init_count = 1;
        bpf_map_update_elem(&syscall_counts, &syscall_id, &init_count, BPF_NOEXIST);
    }

    // Log event to ring buffer
    struct syscall_event *event;
    event = bpf_ringbuf_reserve(&events, sizeof(struct syscall_event), 0);
    if (event) {
        event->pid = pid;
        event->syscall_id = syscall_id;
        bpf_ringbuf_submit(event, 0);
    }

    return 0;
}

char LICENSE[] SEC("license") = "GPL";

2. Compiling the eBPF Program

You’ll need `clang` with eBPF support and `libbpf` headers. Use the AOSP clang toolchain you set up earlier.

clang -target bpf -O2 -emit-llvm -c syscall_monitor.bpf.c -o - | llc -march=bpf -filetype=obj -o syscall_monitor.bpf.o
# Alternatively, if bpf-clang supports it directly:
clang -target bpf -g -O2 -c syscall_monitor.bpf.c -o syscall_monitor.bpf.o

Loading and Attaching the eBPF Program

On the Android device (after flashing your custom kernel), you’ll need a userspace loader to load the `syscall_monitor.bpf.o` file, create the BPF maps, and attach the program to the `sys_enter` tracepoint.

You can use `libbpf` from a custom Android application or leverage `bpftool` if it’s available or compiled for your Android userspace. For a quick test, we’ll demonstrate using `bpftool` commands (assuming it’s available).

1. Pushing the eBPF Object File to Device

adb push syscall_monitor.bpf.o /data/local/tmp/

2. Loading and Attaching with `bpftool` (on device)

Ensure `bpftool` (compiled for Android’s userspace) is available on your device, perhaps by pushing it to `/data/local/tmp/`.

adb shell
cd /data/local/tmp/
# Load the program and maps
bpftool prog load syscall_monitor.bpf.o /sys/fs/bpf/syscall_monitor

# Attach to the tracepoint
bpftool net attach tp raw_syscalls/sys_enter prog_id 
# To find PROG_ID: bpftool prog show -> look for ID of syscall_monitor

Note: Attaching to tracepoints usually requires root privileges. SELinux policies on Android might also prevent this. You might need to adjust SELinux or temporarily put it in permissive mode during development (`setenforce 0`).

Reading eBPF Map Data from Userspace

Once the eBPF program is running, we can read the data from the maps. The `syscall_counts` map holds aggregated data, and the `events` ring buffer provides a stream of individual events.

1. Reading `syscall_counts` Map

# On device (root required)
bpftool map dump id 

This will output key-value pairs of syscall IDs and their counts.

2. Reading `events` Ring Buffer

Reading from the ring buffer is slightly more complex. You’d typically use a `libbpf` userspace application to poll the ring buffer for new events. Here’s a conceptual outline of what that C code would do:

#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <bpf/libbpf.h>
#include "syscall_monitor.bpf.h" // Header generated by bpftool for map definitions

static volatile bool exiting = false;

void sig_handler(int sig) {
    exiting = true;
}

int handle_event(void *ctx, void *data, size_t data_sz)
{
    const struct syscall_event *e = data;
    printf("PID: %u, Syscall ID: %un", e->pid, e->syscall_id);
    return 0;
}

int main(int argc, char **argv)
{
    struct bpf_object *obj;
    struct bpf_map *events_map;
    struct perf_buffer *pb = NULL;

    signal(SIGINT, sig_handler);
    signal(SIGTERM, sig_handler);

    // Assume bpf_object__open_file and bpf_object__load_file logic here
    // For simplicity, directly open the BPF FS map

    int map_fd = bpf_obj_get("/sys/fs/bpf/events"); // Path to your ring buffer map
    if (map_fd < 0) {
        fprintf(stderr, "ERROR: Failed to open BPF map /sys/fs/bpf/events: %sn", strerror(errno));
        return 1;
    }

    pb = perf_buffer__new(map_fd, 64, handle_event, NULL, NULL);
    if (!pb) {
        fprintf(stderr, "ERROR: Failed to create perf buffer: %sn", strerror(errno));
        close(map_fd);
        return 1;
    }

    printf("Monitoring syscall events... Press Ctrl-C to stop.n");
    while (!exiting) {
        perf_buffer__poll(pb, 100); // Poll every 100ms
    }

    perf_buffer__free(pb);
    close(map_fd);
    return 0;
}

This userspace program (which needs to be compiled for Android) would attach to the `events` map (assumed to be pinned to `/sys/fs/bpf/events`) and continuously print incoming syscall events. Building such an application for Android would involve cross-compiling with the NDK and linking against `libbpf`.

Deployment Considerations and Next Steps

Deploying a custom kernel with eBPF on an Android device is a complex task. You’ll typically replace the `boot.img` (containing the kernel and ramdisk) for older devices or `init_boot.img` for newer GKI-enabled devices. Always ensure you have a way to recover your device (e.g., fastboot access) before flashing custom kernels.

Further enhancements could include:

• **Conditional Tracing**: Only trace specific PIDs or syscalls.
• **Data Filtering**: Filter events in the eBPF program to reduce userspace overhead.
• **Advanced Maps**: Use `BPF_MAP_TYPE_STACK_TRACE` to capture call stacks.
• **User-friendly Interface**: Develop a dedicated Android application that loads eBPF programs, configures them, and presents the traced data in an intuitive manner.
• **SELinux Policy**: Craft specific SELinux policies to allow eBPF operations without disabling the entire security framework.

eBPF opens up a powerful new dimension for understanding and optimizing Android’s core system behavior. While the setup requires expertise in kernel compilation and a deep understanding of the Android platform, the rewards in terms of debuggability and performance insights are substantial.