dCompaction: Delayed Compaction for the LSM-Tree

doi:10.1007/s10766-016-0472-z

	dCompaction: Delayed Compaction for the LSM-Tree
	Pan, Fengfeng 1,2; Yue, Yinliang 3; Xiong, Jin 2
	2017-12-01
发表期刊	INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING
ISSN	0885-7458
卷号	45 期号:6 页码:1310-1325
摘要	Key-value (KV) stores have become a backbone of large-scale applications in today's data centers. Write-optimized data structures like the Log-Structured Merge-tree (LSM-tree) and their variants are widely used in KV storage systems like BigTable and RocksDB. Conventional LSM-tree organizes KV items into multiple, successively larger components, and uses compaction to push KV items from one smaller component to another adjacent larger component until the KV items reach the largest component. Unfortunately, current compaction scheme incurs significant write amplification due to repeated KV item reads and writes, and then results in poor throughput. We propose a new compaction scheme, delayed compaction (dCompaction), that decreases write amplification. dCompaction postpones some compactions and gather them into the following compaction. In this way, it avoids KV item reads and writes during compaction, and consequently improves the throughput of LSM-tree based KV stores. We implement dCompaction on RocksDB, and conduct extensive experiments. Validation using YCSB framework shows that compared with RocksDB dCompaction has about 30% write performance improvements and also comparable read performance.
关键词	Key-value store LSM-tree Write amplification Delayed compaction Virtual compaction Real compaction
DOI	10.1007/s10766-016-0472-z
收录类别	SCI
语种	英语
资助项目	MOST's 13th FYP project of China[2016YFB1000202] ; National Science Foundation of China[61303056] ; National Science Foundation of China[61379042] ; Youth Innovation Promotion Association, CAS[2016146]
WOS研究方向	Computer Science
WOS类目	Computer Science, Theory & Methods
WOS记录号	WOS:000411558500005
出版者	SPRINGER/PLENUM PUBLISHERS
引用统计	被引频次：27[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/6811
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Pan, Fengfeng
作者单位	1.Univ Chinese Acad Sci, Beijing, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing, Peoples R China 3.Chinese Acad Sci, Inst Informat Engn, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Pan, Fengfeng,Yue, Yinliang,Xiong, Jin. dCompaction: Delayed Compaction for the LSM-Tree[J]. INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING,2017,45(6):1310-1325.
APA	Pan, Fengfeng,Yue, Yinliang,&Xiong, Jin.(2017).dCompaction: Delayed Compaction for the LSM-Tree.INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING,45(6),1310-1325.
MLA	Pan, Fengfeng,et al."dCompaction: Delayed Compaction for the LSM-Tree".INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING 45.6(2017):1310-1325.