HV-SNSP: A Low-Overhead Data Recovery Method Based on Cross-Checking

doi:10.1109/ACCESS.2023.3235787

	HV-SNSP: A Low-Overhead Data Recovery Method Based on Cross-Checking
	Song, Ying 1,2,3; Mu, Tiantong 1; Wang, Bo 4
	2023
发表期刊	IEEE ACCESS
ISSN	2169-3536
卷号	11 页码:5737-5745
摘要	The failure of a single unreliable commodity component is very common in large-scale distributed storage systems. In order to ensure the reliability of data in large-scale distributed storage systems, many studies have emerged one after another. Among them, Erasure Codes are widely used in actual storage systems, such as Hadoop Distributed File System (HDFS), which provides high fault-tolerance with lower storage overhead. However, usually the recovery of erasure-coded storage system when encountering node failure results in severe cross-node and cross-rack bandwidth consumption, which affects the efficiency of failure recovery and wastes additional resources. In this paper, we improve the erasure coding storage strategy in distributed storage systems, and propose a low-overhead data recovery method based on cross-checking, namely HV-SNSP. In HV-SNSP, horizontal and vertical cross parity checking is realized by adding RS parity inside the data node, that is, H-RS(n,H-k)-V(RS(n ', k ') )storage architecture. Based on H-RS(n,H- k)-V-RS(n ',V-k '), a low-cost supply node selection strategy, namely SNSP, is designed, and nodes with shorter network distance and lower load are selected to participate in recovery. This strategy can effectively reduce the amount of data transmission, shorten the recovery time, and improve the recovery efficiency. The experimental results show that compared with traditional RS, HV-SNSP can reduce the amount of cross-rack data transmission by 62.5% during data recovery, and can shorten the recovery time by up to 42.41%; Compared with D-3, HV-SNSP can reduce the occupation of cross-rack bandwidth by 25% and shorten the recovery time by 36.58%.
关键词	Data recovery distributed storage system erasure coding
DOI	10.1109/ACCESS.2023.3235787
收录类别	SCI
语种	英语
WOS研究方向	Computer Science ; Engineering ; Telecommunications
WOS类目	Computer Science, Information Systems ; Engineering, Electrical & Electronic ; Telecommunications
WOS记录号	WOS:000922835900001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/19962
专题	中国科学院计算技术研究所期刊论文
通讯作者	Song, Ying
作者单位	1.Beijing Informat Sci & Technol Univ, Beijing Key Lab Internet Culture & Digital Dissemi, Beijing 100101, Peoples R China 2.Beijing Informat Sci & Technol Univ, Beijing Adv Innovat Ctr Mat Genome Engn, Beijing 100101, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100086, Peoples R China 4.Zhengzhou Univ Light Ind, Software Engn Coll, Zhengzhou 450002, Peoples R China
推荐引用方式 GB/T 7714	Song, Ying,Mu, Tiantong,Wang, Bo. HV-SNSP: A Low-Overhead Data Recovery Method Based on Cross-Checking[J]. IEEE ACCESS,2023,11:5737-5745.
APA	Song, Ying,Mu, Tiantong,&Wang, Bo.(2023).HV-SNSP: A Low-Overhead Data Recovery Method Based on Cross-Checking.IEEE ACCESS,11,5737-5745.
MLA	Song, Ying,et al."HV-SNSP: A Low-Overhead Data Recovery Method Based on Cross-Checking".IEEE ACCESS 11(2023):5737-5745.