MTAT: Adaptive Fast Memory Management for Co-located Latency-Critical Workloads in Tiered Memory System

Abstract

Modern data centers increasingly employ multi-tenant deployment models in which multiple applications or virtual machines share a single physical server. However, existing tiered memory management schemes classify pages solely by access frequency to govern promotions and demotions across memory tiers without accounting for the distinct access patterns of latency-critical (LC) and best-effort (BE) workloads. LC workloads demand low-latency service yet lack sustained high-frequency access; consequently, frequency-based tiering demotes LC data to slower memory (SMem), degrading responsiveness and violating service-level objectives (SLOs).To address these challenges, we propose MTAT, an adaptive tiered memory management framework that guarantees the SLO of LC workloads while maintaining overall system performance for BE workloads. Rather than relying solely on hotness-based page placement, MTAT employs distinct policies for LC and BE workloads by isolating them into dedicated fast memory (FMem) partitions. Specifically, MTAT employs reinforcement learning to identify the minimal FMem capacity necessary to satisfy stringent SLOs, supporting rapid response to sudden demand surges, and uses a simulated annealing algorithm to allocate the remaining FMem fairly among co-located BE workloads. Compared to state-of-the-art tiered memory page-placement solutions, MTAT improves the maximum throughput of LC workloads by up to 1.7× and enhances BE workloads' fairness by up to 3.3×, all while incurring only a 19% throughput penalty at worst.