Impact of Joint Heat and Memory Constraints of Mobile Device in Edge-Assisted On-Device Artificial Intelligence

Abstract

Recently, consumer demand for artificial intelligence (AI) applications using deep neural network (DNN) model such as large language model (LLM), miXed Reality (XR), and AI assistants has been steadily increasing. Hitherto, on-device AI and offloaded analytics with the help of mobile edge computing (MEC) have been extensively studied to realize AI services on top of mobile devices. However, both technologies suffer from the limited resources of mobile devices, such as thermal resilience, battery capacity, and memory size. To tackle this problem, we first extensively examine the impact of heat and memory constraints of a mobile device when networking and processing resources and multi-dimensional DNN model sizes are dynamically managed for AI applications via motivating measurement. From the experimental results, we conjecture that the threshold-based approach for joint consideration of heat and memory constraints would increase the performance of AI applications in terms of energy, frames per second (FPS), and inference accuracy. Hence, we propose a threshold-based H&M algorithm that jointly adjusts offloading, Dynamic Voltage and Frequency Scaling (DVFS), and DNN model size, aiming to maximize inference accuracy while keeping target FPS with memory and heat constraints in various environments. Finally, we implement the proposed scheme on a mobile device and an MEC server and evaluate its performance and adaptability via extensive experiments. © 2024 Owner/Author.