Beyond the Numbers: Measuring Android Performance Through User Perception

Abstract

Android, with its vast global adoption and diverse hardware ecosystem, poses unique challenges for performance benchmarking, particularly from a user-centric perspective. Traditional benchmarks often fail to capture the intricacies of user-perceived performance, relying on component-level metrics or synthetic workloads that do not reflect real-world usage. This paper proposes Real-Time User-Experience, RTUX, a novel benchmarking tool designed to measure Android system performance as perceived by users. RTUX employs external camera-based GUI state recognition and scenario-based testing to evaluate app loadtimes and in-app transitions under diverse conditions. Using CNN models and a unique system structure, RTUX reliably replays human-like interactions, enabling repeatable and robust performance assessments. Through experiments with 100 scenario repetitions involving popular Android apps, we uncover some system bottlenecks, such as suboptimal writeback configurations and I/O scheduler inefficiencies. The tool demonstrates how targeted optimizations can yield tangible improvements in user experience.