i(2)MapReduce: Incremental MapReduce for Mining Evolving Big Data

doi:10.1109/TKDE.2015.2397438

	i(2)MapReduce: Incremental MapReduce for Mining Evolving Big Data
	Zhang, Yanfeng 1; Chen, Shimin 2; Wang, Qiang 3; Yu, Ge 3
	2015-07-01
发表期刊	IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING
ISSN	1041-4347
卷号	27 期号:7 页码:1906-1919
摘要	As new data and updates are constantly arriving, the results of data mining applications become stale and obsolete over time. Incremental processing is a promising approach to refreshing mining results. It utilizes previously saved states to avoid the expense of re-computation from scratch. In this paper, we propose i(2)MapReduce, a novel incremental processing extension to MapReduce, the most widely used framework for mining big data. Compared with the state-of-the-art work on Incoop, i(2)MapReduce (i) performs key-value pair level incremental processing rather than task level re-computation, (ii) supports not only one-step computation but also more sophisticated iterative computation, which is widely used in data mining applications, and (iii) incorporates a set of novel techniques to reduce I/O overhead for accessing preserved fine-grain computation states. We evaluate i(2)MapReduce using a one-step algorithm and four iterative algorithms with diverse computation characteristics. Experimental results on Amazon EC2 show significant performance improvements of i(2)MapReduce compared to both plain and iterative MapReduce performing re-computation.
关键词	Incremental processing MapReduce iterative computation big data
DOI	10.1109/TKDE.2015.2397438
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[61300023] ; National Natural Science Foundation of China[61433008] ; National Natural Science Foundation of China[61272179] ; Fundamental Research Funds for the Central Universities[N141605001] ; Fundamental Research Funds for the Central Universities[N120816001] ; China Mobile Fund[MCM20125021] ; MOE-Intel Special Fund of Information Technology[MOE-INTEL-2012-06] ; CAS Hundred Talents program ; NSFC Innovation Research Group[61221062]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Information Systems ; Engineering, Electrical & Electronic
WOS记录号	WOS:000355937800013
出版者	IEEE COMPUTER SOC
引用统计	被引频次：29[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/9734
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhang, Yanfeng
作者单位	1.Northeastern Univ, Ctr Comp, Shenyang 110819, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100864, Peoples R China 3.Northeastern Univ, Shenyang 110819, Peoples R China
推荐引用方式 GB/T 7714	Zhang, Yanfeng,Chen, Shimin,Wang, Qiang,et al. i(2)MapReduce: Incremental MapReduce for Mining Evolving Big Data[J]. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,2015,27(7):1906-1919.
APA	Zhang, Yanfeng,Chen, Shimin,Wang, Qiang,&Yu, Ge.(2015).i(2)MapReduce: Incremental MapReduce for Mining Evolving Big Data.IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,27(7),1906-1919.
MLA	Zhang, Yanfeng,et al."i(2)MapReduce: Incremental MapReduce for Mining Evolving Big Data".IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 27.7(2015):1906-1919.