Packet-Level CXL.cache Analysis With OpenCIS

Abstract

CXL.enables devices to coherently access host memory, but its behavior is difficult to understand without packet-level visibility into ownership transfers, cache line invalidations, and host-device request flows. Packet-level visibility into CXL.cache is thus crucial for CXL research, but it is largely ignored by existing CXL tools. Therefore, we present OpenCIS, an open-source CXL simulation platform on the transaction layer that supports CXL.cache and flexible CXL topologies. We show that OpenCIS's transaction-layer visibility is useful for both CXL.cache workload-side optimization and protocol-side exploration by implementing two workloads using OpenCIS, namely PageRank and FlushData. For PageRank, we explore the importance of CXL-aware workload-side optimization by showing the differences between two workload-sharding approaches under a multi-accelerator scenario. For FlushData, we reveal the benefits of a new CXL.cache opcode that actively pushes needed data to accelerators from the host.