Building Efficient Key-value Storage Systems with Emerging Storage Technologies
- Title
- Building Efficient Key-value Storage Systems with Emerging Storage Technologies
- Alternative Title
- 최신 저장 기술을 활용한 고효율 키-값 저장 시스템 연구
- Author(s)
- Jinhyung Koo
- DGIST Authors
- Jinhyung Koo ; Sungjin Lee ; Yeseong Kim
- Advisor
- 이성진
- Co-Advisor(s)
- Yeseong Kim
- Issued Date
- 2025
- Awarded Date
- 2025-02-01
- Type
- Thesis
- Description
- Key-value storage system, Key-value store, Key-value cache, Multi-tiered memory system, File system
- Abstract
- This dissertation investigates efficient key-value (KV) storage systems using emerging storage technologies to meet modern data center demands. The study introduces four systems—LightStore, BigKV, MigFlow, and KEVIN—to reduce costs and improve performance. LightStore replaces x86 servers with lightweight, ARM-based SSD nodes directly connected to networks, achieving 2.0× better power efficiency, 2.3× greater space efficiency, and up to 7.4× higher energy efficiency. BigKV optimizes flash-based KV caching for large objects, improving throughput by 3.5× and reducing latency by 57%, enabling cost-effective, petabyte-scale caching. MigFlow enhances multi-tiered memory systems using persistent memory and CXL. With MigOpt, an offline optimizer, MigFlow reduces migration overhead and achieves up to 2.76× better performance while cutting migration traffic by 60.6%. KEVIN integrates file systems directly with KV storage, eliminating block-based overhead. It improves metadata performance by 6.2× and boosts throughput by 68% for real workloads. This work demonstrates how emerging technologies can enhance scalability, efficiency, and cost-effectiveness in hyperscale data centers. Keywords: Key-value storage system, Key-value store, Key-value cache, Multi-tiered memory system, File system.|본 논문은 차세대 스토리지 기술을 활용하여 효율적인 키-값 저장 시스템을 설계하고 현대 데이터 센터의 성능과 비용 문제를 해결하는 방법을 제시한다. 이를 위해 LightStore, BigKV, MigFlow, KEVIN 네 가지 시스템을 제안한다. LightStore는 ARM 기반의 SSD 노드를 도입해 기존 x86 기반 키-값 스토리지 서버를 대신하는 역할을 수행하며, 이로써 전력 효율을 2.0배, 공간 효율을 2.3배 향상시키며 에너지 효율은 최대 7.4배 높인다. BigKV는 대형 객체를 효율적으로 캐싱하는 플래시 기반 키-값 캐시로서, 처리량을 3.5배 향상시키고 지연 시간을 57% 줄여 페타바이트 캐싱 비용을 절감한다. MigFlow는 PMEM과 CXL을 활용 등을 사용하는 이기종 멀티 티어 메모리 최적화 시스템으로, MigOpt의 오프라인 분석을 통해 마이그레이션 오버헤드를 줄이고 성능을 최대 2.76배 향상시킨다. KEVIN은 파일 시스템을 키-값 저장소와 직접 통합해 블록 추상화를 제거하고, 메타데이터 성능을 6.2배, 처리량을 68% 개선한다. 본 연구는 차세대 스토리지 기술을 기반으로 확장성과 비용 효율성을 개선하는 새로운 키-값 저장 시스템을 제시한다.
- Table Of Contents
-
1 Introduction 1
2 LightStore 5
2.1 Introduction 5
2.2 Related Work 9
2.3 LightStore Overview 10
2.4 Design of LightStore Node 12
2.4.1 LightStore Controller 12
2.4.2 LightStore Software 14
2.5 Expected Operating Cost 23
2.6 Experimental Results 24
2.6.1 Prototype and Experimental Setup 25
2.6.2 LightStore as a Key-value Store 27
2.6.3 Evaluation under Datacenter Applications 36
3 BigKV 40
3.1 Introduction 40
3.2 Background and Related Work 43
3.2.1 NAND Flash and All-Flash Array 44
3.2.2 Persistent Key-value Stores 45
3.2.3 Key-value Caches for Flash 46
3.3 Motivation 47
3.3.1 Huge Metadata and High Overhead 48
3.3.2 Space Waste by Expired Objects 50
3.3.3 Fault Tolerance vs Parity Cost 51
3.4 Design and Implementation of BigKV 52
3.4.1 Overall Architecture of BigKV 54
3.4.2 Object Indexing with Two-level Metadata 55
3.4.3 TTL-aware Space Management 60
3.4.4 Reactive Fault Tolerance Mechanism 64
3.5 Experiments 66
3.5.1 Experimental Setup 66
3.5.2 Experimental Results 68
4 MigFlow 77
4.1 Introduction 77
4.2 Background and Related Work 80
4.2.1 Tiered Memory System 81
4.2.2 Traditional 2-tier Memory System 82
4.2.3 Multi-tiered Memory System 83
4.3 Motivation 85
4.4 MigOpt 89
4.4.1 Existing Research for Exploring the Optimality 90
4.4.2 Providing the Optimality for Multi-tiered Systems 91
4.4.3 Analysis of MigOpt Results 93
4.5 MigFlow 97
4.5.1 MigFlow Overview 97
4.5.2 Top-down Allocation and Page Monitoring 99
4.5.3 Migration Policy 100
4.6 Experiments 102
4.6.1 Experimental Setup 103
4.6.2 Experimental Results 105
5 KEVIN 109
5.1 Introduction 109
5.2 Background and Related Work 113
5.2.1 Traditional Block I/O Interface 113
5.2.2 Review of In-Storage Indexing 114
5.2.3 File System over Key-value Store 116
5.2.4 LSM-Tree Basics 117
5.3 Overall Architecture of KEVIN 118
5.3.1 Mapping of File and Directory 119
5.3.2 Indexing of KV Objects 121
5.3.3 Mitigating Indexing Overhead 124
5.4 Implementing VFS Operations 127
5.5 Crash Consistency 130
5.5.1 Maintaining Consistency in KEVINFS 131
5.5.2 Transaction Processing in KEVINSSD 133
5.6 Experiments 135
5.6.1 Experimental Setup 135
5.6.2 Experimental Results 136
6 Conclusions 147
References 150
- URI
-
http://hdl.handle.net/20.500.11750/57995
http://dgist.dcollection.net/common/orgView/200000837359
- Degree
- Doctor
- Publisher
- DGIST
- Related Researcher
-
-
Lee, Sungjin
- Research Interests Computer System; System Software; Storage System; Non-volatile Memory; Flash-based SSD; Distributed Storage Systems
-
- Files in This Item:
-
There are no files associated with this item.
- Appears in Collections:
- Department of Electrical Engineering and Computer Science Theses Ph.D.
Items in Repository are protected by copyright, with all rights reserved, unless otherwise indicated.