EcoCore: Dynamic Core Management for Improving Energy Efficiency in Latency-Critical Applications

Abstract

Modern data centers face increasing pressure to improve energy efficiency while guaranteeing Service Level Objectives (SLOs) for Latency-Critical (LC) applications. Resource management in public cloud environments, typically operating at the node or instance level, often results in underutilized idle cores and unnecessary energy consumption, highlighting the need for fine-grained core-level management. Existing studies on core management primarily focus on allocating cores for application threads, aiming to maximize idle cores without violating SLOs, but often neglect the impact of network packet processing on core idleness and energy consumption. In this work, we demonstrate that separate core allocation for network packet processing is essential to optimize both performance and energy efficiency in LC applications. Additionally, we show that co-managing packet processing intervals alongside core allocation further enhances core idleness and reduces energy consumption without compromising SLO compliance. Based on these insights, we propose EcoCore, a dynamic core management technique that jointly manages core allocation for application threads and network packet processing while adaptively adjusting packet processing intervals. EcoCore employs a lightweight predictive model to estimate energy consumption and tail latency, enabling it to select energy-efficient configurations without violating SLOs. Through comprehensive evaluations, including deployment on the AWS cloud platform, EcoCore demonstrates its practicality, reducing energy consumption by 11.7 on average and by up to 20.3 without SLO violations. In the AWS cloud environment, EcoCore extends core sleep states by up to 45.9, resulting in additional energy savings of up to 35.8. These results highlight EcoCore 's potential for bridging the gap between coarse-grained resource management and fine-grained core management in data centers.