Graphine: Programming Graph-Parallel Computation of Large Natural Graphs for Multicore Clusters

doi:10.1109/TPDS.2015.2453978

	Graphine: Programming Graph-Parallel Computation of Large Natural Graphs for Multicore Clusters
	Yan, Jie 1; Tan, Guangming 1; Mo, Zeyao 2; Sun, Ninghui 1
	2016-06-01
发表期刊	IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS
ISSN	1045-9219
卷号	27 期号:6 页码:1647-1659
摘要	Graph-parallel computation has become a crucial component in emerging applications of web search, data analytics and machine learning. In practice, most graphs derived from real-world phenomena are very large and scale-free. Unfortunately, distributed graph-parallel computation of these natural graphs still suffers strong scalability issues on contemporary multicore clusters. To embrace the multicore architecture in distributed graph-parallel computation, we propose the framework Graphine, which features (i) A Scatter-Combine computation abstraction that is evolved from the traditional vertex-centric approach by fusing the paired scatter and gather operations, executed separately on two edge sides, into a one-sided scatter. Further coupled with active message mechanism, it potentially reduces intermediate message cost and enables fine-grained parallelism on multicore architecture. (ii) An Agent-Graph data model, which leverages an idea similar to vertex-cut but conceptually splits the remote replica into two agent types of scatter and combiner, resulting in less communication. We implement the Graphine framework and evaluate it using several representative algorithms on six large real-world graphs and a series of synthetic graphs with power-law degree distributions. We show that Graphine achieves sublinear scalability with the number of cores per node, number of nodes, and graph sizes (up to one billion vertices), and is 2 similar to 15 times faster than the state-of-the-art PowerGraph on a cluster of 16 multicore nodes.
关键词	Graph-parallel parallel framework computational model
DOI	10.1109/TPDS.2015.2453978
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[61272134] ; National Natural Science Foundation of China[31327901] ; National Natural Science Foundation of China[91430218] ; National Natural Science Foundation of China[60921002] ; National Natural Science Foundation of China[60925009] ; National Natural Science Foundation of China[61472395] ; National 863 Program[2009AA01A129] ; 973 Program[2012CB316502] ; 973 Program[2011CB302502]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:000376106400008
出版者	IEEE COMPUTER SOC
引用统计	被引频次：7[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/8600
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Yan, Jie; Tan, Guangming; Mo, Zeyao; Sun, Ninghui
作者单位	1.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100864, Peoples R China 2.Inst Appl Phys & Computat Math, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Yan, Jie,Tan, Guangming,Mo, Zeyao,et al. Graphine: Programming Graph-Parallel Computation of Large Natural Graphs for Multicore Clusters[J]. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,2016,27(6):1647-1659.
APA	Yan, Jie,Tan, Guangming,Mo, Zeyao,&Sun, Ninghui.(2016).Graphine: Programming Graph-Parallel Computation of Large Natural Graphs for Multicore Clusters.IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,27(6),1647-1659.
MLA	Yan, Jie,et al."Graphine: Programming Graph-Parallel Computation of Large Natural Graphs for Multicore Clusters".IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 27.6(2016):1647-1659.