Fast Data-Obtaining Algorithm for Data Assimilation with Large Data Set

doi:10.1007/s10766-019-00653-y

	Fast Data-Obtaining Algorithm for Data Assimilation with Large Data Set
	Xiao, Junmin 1,2; Zhang, Guizhao 3; Gao, Yanan 4; Ho, Xuehai 3; Tan, Guangming 1,2
	2019-12-06
发表期刊	INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING
ISSN	0885-7458
页码	21
摘要	Data assimilation is an analysis technique which combines observations and the numerical results from theoretical models to deduce more realistic and accurate data. It is widely used in investigations of the atmosphere, ocean and land surface. Due to the complicated data structure of the inputs from dynamical models and the increase of the amount of model data, the parallelization of data assimilation suffers from high overhead on file reading and data communication. In this paper, we propose a flexible parallel data access approach for reading a large number of data from disks firstly. Using this approach, the data access conflict is avoided successfully, and the frequency of disk addressing operations is also decreased significantly. Next, we design a communication-avoiding strategy to reduce the communication volume at the cost of some additional computations. Furthermore, we present a "pipe-flow" scheme for data exchange to conduct conflict-free message passing. Consequently, a fast data-obtaining algorithm is developed for the data assimilation. Our experiments show that the fast data-obtaining algorithm gains a performance of 5x speedup compared with the baseline, which is excellent at data-obtaining for the parallel data assimilation. Due to the reduction of disk addressing operations, the new approach achieves 6x speedup on average for the file reading process. Since a large amount of data movement can be avoided, the new approach achieves 2.7x speedup on average for the communication between processors.
关键词	Data assimilation I O optimization Communication optimization Parallel implementation Domain localization
DOI	10.1007/s10766-019-00653-y
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2016YFC1401706] ; National Natural Science Foundation of China[61802369]
WOS研究方向	Computer Science
WOS类目	Computer Science, Theory & Methods
WOS记录号	WOS:000529493100001
出版者	SPRINGER/PLENUM PUBLISHERS
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/15466
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Xiao, Junmin
作者单位	1.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 4.Beijing Inst Control Engn, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Xiao, Junmin,Zhang, Guizhao,Gao, Yanan,et al. Fast Data-Obtaining Algorithm for Data Assimilation with Large Data Set[J]. INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING,2019:21.
APA	Xiao, Junmin,Zhang, Guizhao,Gao, Yanan,Ho, Xuehai,&Tan, Guangming.(2019).Fast Data-Obtaining Algorithm for Data Assimilation with Large Data Set.INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING,21.
MLA	Xiao, Junmin,et al."Fast Data-Obtaining Algorithm for Data Assimilation with Large Data Set".INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING (2019):21.