Introduction: The Criticality of Touch Latency

In the world of mobile computing, user experience is paramount, and few factors influence it as profoundly as touch responsiveness. High touch latency – the delay between a finger touching the screen and the visual feedback appearing – can make an otherwise powerful device feel sluggish and unresponsive. While modern Android devices generally offer excellent performance, subtle delays can still exist, often hidden deep within the kernel or userspace. This article delves into a practical case study, demonstrating how to meticulously identify and mitigate touch latency bottlenecks on Android devices using Ftrace, the powerful tracing utility built directly into the Linux kernel.

Ftrace provides unparalleled visibility into the kernel’s inner workings, offering both function callgraph and event-based tracing capabilities. By correlating these detailed traces with user interactions, developers and system engineers can pinpoint exactly where delays occur, turning an abstract “slow feel” into actionable data points.

Understanding the Android Touch Pipeline

Before we trace, it’s crucial to understand the journey of a touch event:

1. Hardware Digitizer: The physical touch panel detects a touch and converts it into electrical signals.
2. Input Driver (Kernel): The kernel’s input driver (e.g., a touchscreen driver) reads these signals, decodes them, and generates an `input_event`.
3. Input Subsystem (Kernel): The `input_event` travels through the generic input subsystem, potentially involving event handlers.
4. Event Hub (Userspace HAL): A daemon (often part of `inputflinger` or a vendor HAL) reads these kernel events.
5. SurfaceFlinger/InputFlinger: Input events are passed to `InputFlinger`, which processes them and sends them to the appropriate application via binder IPC. `SurfaceFlinger` then orchestrates the composition and rendering.
6. Application: The target application processes the touch event, updates its UI, and requests a redraw.
7. GPU/Display Controller: The rendered frame is sent to the display.

Each step introduces potential latency. Our goal with Ftrace is to measure these delays and find the longest poles in the tent.

Setting Up Your Android Device for Ftrace

To use Ftrace, you’ll need an Android device with ADB access, preferably rooted or running a debug/userdebug build. Necessary kernel configurations (`CONFIG_FTRACE`, `CONFIG_FUNCTION_TRACER`, `CONFIG_FUNCTION_GRAPH_TRACER`, `CONFIG_EVENTS_TRACER`) must be enabled in your kernel build. For most debug/userdebug builds, these are already active.

Accessing Ftrace via ADB

Ftrace controls are exposed through the `debugfs` filesystem, typically mounted at `/sys/kernel/tracing`.

adb shellsu# Ensure debugfs is mounted, usually automaticmount -t debugfs none /sys/kernel/tracing

Step-by-Step Ftrace Analysis: Capturing Touch Latency Data

1. Clear and Disable Tracing

Always start with a clean slate to avoid stale data.

echo 0 > /sys/kernel/tracing/tracing_onecho > /sys/kernel/tracing/trace

2. Enable Relevant Events

We’re interested in the journey of the touch event. Crucial events include:

• Input Events: `input/input_event` (raw event from driver), `input/touch_down`, `input/touch_move`, `input/touch_up`.
• Scheduler Events: `sched/sched_switch`, `sched/sched_wakeup` (to see CPU contention/scheduling delays).
• Display/UI Events: `vsync_event`, `binder_transaction`, `sf_frame_posted` (from `SurfaceFlinger`), `hwcomposer_event`. These often require specific kernel/userspace patches to expose fully, but `binder_transaction` is generally available.

Example for enabling input and scheduler events:

echo 1 > /sys/kernel/tracing/events/input/input_event/enableecho 1 > /sys/kernel/tracing/events/sched/sched_switch/enableecho 1 > /sys/kernel/tracing/events/sched/sched_wakeup/enableecho 1 > /sys/kernel/tracing/events/raw_syscalls/sys_enter/enableecho 1 > /sys/kernel/tracing/events/raw_syscalls/sys_exit/enable

3. Configure Function Graph Tracer (Optional but Recommended)

To deeply analyze specific kernel paths, use `function_graph`. Limit it to relevant functions to avoid excessive data. For touch latency, focus on input drivers, IRQ handlers, and related kernel threads.

echo function_graph > /sys/kernel/tracing/current_tracerecho 'input_dev_*' > /sys/kernel/tracing/set_ftrace_filter # Or specific driver functions, e.g., 'elantech_*'echo '!__schedule' > /sys/kernel/tracing/set_ftrace_notrace # Exclude noisy functions

If `function_graph` generates too much data, switch back to `function` tracer or focus solely on events.

4. Capture the Trace

Perform the action that exhibits latency (e.g., a specific touch gesture in an app).

echo 1 > /sys/kernel/tracing/tracing_on # Start tracing# Perform touch action on deviceecho 0 > /sys/kernel/tracing/tracing_on # Stop tracing

5. Extract and Analyze Trace Data

Pull the trace file from the device:

adb pull /sys/kernel/tracing/trace /tmp/touch_latency_trace.txt

Analyze the `touch_latency_trace.txt` file on your host machine. Tools like `trace-cmd` (with `trace-cmd report`) or `kernelshark` provide a graphical interface. For manual inspection, look for timestamp differences. Key indicators:

• Input Latency: Timestamp of `input_event` to the first related userspace activity (e.g., a `binder_transaction` or app’s render cycle).
• Kernel Processing Delay: Long duration between an `input_event` and the next `sched_switch` to the input-processing task, or significant time spent within the input driver’s `function_graph`.
• Userspace Handoff: Delays between input events being delivered to `InputFlinger` and `SurfaceFlinger` initiating rendering.

A snippet of trace output might look like this (simplified):

         -0     [001] d..3 10245.123456: input_event: type=EV_ABS code=ABS_X value=1000...         -0     [001] d..3 10245.123470: function: input_event_handler <- my_touch_driver_irq...         -0     [001] d..3 10245.123510: function: my_touch_driver_process_data...         -0     [001] d..3 10245.123700: sched_switch: prev_comm=idle prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=inputflinger next_pid=123 next_prio=99...   inputflinger-123  [000] d..3 10245.123750: binder_transaction: ... target_node=5 target_handle=...         -0     [001] d..3 10245.124000: sched_switch: prev_comm=inputflinger prev_pid=123 prev_prio=99 prev_state=R ==> next_comm=surfaceflinger next_pid=456 next_prio=98...surfaceflinger-456 [002] d..3 10245.124100: vsync_event: ...

In this example, `my_touch_driver_process_data` takes ~190µs (123.700 – 123.510), and the switch to `inputflinger` occurs quickly after, but there’s a total ~600µs from `input_event` to `vsync_event`. Each timestamp represents a critical point to optimize.

Case Study: Identifying and Mitigating a Bottleneck

Consider a scenario where our Ftrace analysis consistently shows a significant delay (e.g., 5-10ms) immediately after the `input_event` is generated, before `InputFlinger` takes over.

The Problem: Kernel-side Input Processing Lag

Detailed `function_graph` tracing within the input driver and `set_ftrace_filter` revealed that a specific function, `custom_panel_calibration_routine`, was being called on *every* touch event, not just on initialization or calibration triggers. This function, intended for one-time setup, involved complex calculations and I2C communication, leading to the observed delay.

The Solution: Code Optimization and Event-Driven Design

The fix involved:

1. Refactoring `custom_panel_calibration_routine`: Modify the driver to call this routine only once during device boot or resume, storing the results.
2. Minimizing Critical Path Work: Ensure the `input_event` path is as lean as possible, deferring non-essential processing to workqueues or kthreads outside the immediate interrupt context.

Before Optimization (`function_graph` snippet):

        `-- my_touch_driver_irq() {          ...          `-- custom_panel_calibration_routine() {             // Takes 5-10ms             ...          } // custom_panel_calibration_routine          ...       } // my_touch_driver_irq`

After Optimization (conceptual change):

       `-- my_touch_driver_irq() {          // Only essential processing          // Input event quickly passed up          ...       } // my_touch_driver_irq`      // custom_panel_calibration_routine() called elsewhere, asynchronously or once.

Re-tracing after implementing this change showed the 5-10ms delay disappeared, significantly improving overall touch responsiveness.

Advanced Ftrace Techniques

• Per-CPU Tracing: Investigate contention or specific CPU core behavior. `echo N > /sys/kernel/tracing/per_cpu/cpuN/buffer_size_kb`
• Filtering Events: Use `set_event_filter` to narrow down events based on their fields (e.g., `echo ‘pid==123’` > `/sys/kernel/tracing/events/sched/sched_switch/filter`).
• Stack Tracing: Combine `ftrace` with `perf` for stack backtraces, giving context to where a function was called from.

Conclusion

Ftrace is an indispensable tool for deep-dive performance analysis on Android’s Linux kernel. By meticulously enabling event and function graph tracing, capturing relevant data during user interaction, and analyzing the resulting timestamps, you can transform vague performance complaints into precise, actionable engineering tasks. This case study demonstrates how a systematic approach using Ftrace can uncover and resolve subtle, yet impactful, touch latency issues, ultimately leading to a more fluid and satisfying user experience.