Optimizing the LINPACK Algorithm for Large-Scale PCIe-Based CPU-GPU Heterogeneous Systems

doi:10.1109/TPDS.2021.3067731

	Optimizing the LINPACK Algorithm for Large-Scale PCIe-Based CPU-GPU Heterogeneous Systems
	Tan, Guangming; Shui, Chaoyang; Wang, Yinshan; Yu, Xianzhi; Yan, Yujin
	2021-09-01
发表期刊	IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS
ISSN	1045-9219
卷号	32 期号:9 页码:2367-2380
摘要	There is a widening gap between GPU and other components (CPU, PCIe bus and communication network) in heterogeneous parallel system. The gap forces us to orchestrate cooperative execution among these components much more carefully than ever before. By taking the LINPACK benchmark as a case study, this article proposes a fine-grained pipelining algorithm on large-scale CPU-GPU heterogeneous cluster systems. First, we build an algorithmic model that reveals a new approach to GPU-centric and fine-grained pipelining algorithm design. Then, we present four model-driven pipelining algorithms that incrementally squeeze bubbles in the pipeline so that it is occupied by more useful floating-point calculations. The algorithms are implemented on both the AMD and NVIDIA GPU platforms. The finally optimized LINPACK program achieves 107 PFlops on 25, 600 GPUs (70 percent floating-point efficiency). Several insights have been drawn to suggest tradeoff of algorithm design, programming support, and architecture design.
关键词	Pipeline processing Graphics processing units Computer architecture Supercomputers Clustering algorithms Programming Optimization LINPACK algorithm software pipeline performance model heterogeneous computing cluster
DOI	10.1109/TPDS.2021.3067731
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2018YFB0204400] ; National Key Research and Development Program of China[XDC01030000] ; National Key Research and Development Program of China[XDC05010100] ; National Natural Science Foundation of China[62032023] ; National Natural Science Foundation of China[61972377] ; National Natural Science Foundation of China[61702483] ; CAS[QYZDJ-SSW-JSC035]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:000638398900004
出版者	IEEE COMPUTER SOC
引用统计	被引频次：9[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/16648
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Shui, Chaoyang
作者单位	Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100864, Peoples R China
推荐引用方式 GB/T 7714	Tan, Guangming,Shui, Chaoyang,Wang, Yinshan,et al. Optimizing the LINPACK Algorithm for Large-Scale PCIe-Based CPU-GPU Heterogeneous Systems[J]. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,2021,32(9):2367-2380.
APA	Tan, Guangming,Shui, Chaoyang,Wang, Yinshan,Yu, Xianzhi,&Yan, Yujin.(2021).Optimizing the LINPACK Algorithm for Large-Scale PCIe-Based CPU-GPU Heterogeneous Systems.IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,32(9),2367-2380.
MLA	Tan, Guangming,et al."Optimizing the LINPACK Algorithm for Large-Scale PCIe-Based CPU-GPU Heterogeneous Systems".IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 32.9(2021):2367-2380.