A heterogeneous computing system for data mining workflows in multi-agent environments

	A heterogeneous computing system for data mining workflows in multi-agent environments
	Luo, Ping ; Lu, Kevin ; Huang, Rui ; He, Qing ; Shi, Zhongzhi
	2006-11-01
发表期刊	EXPERT SYSTEMS
ISSN	0266-4720
卷号	23 期号:5 页码:258-272
摘要	The computing-intensive data mining (DM) process calls for the support of a heterogeneous computing system, which consists of multiple computers with different configurations connected by a high-speed large-area network for increased computational power and resources. The DM process can be described as a multi-phase pipeline process, and in each phase there could be many optional methods. This makes the workflow for DM very complex and it can be modeled only by a directed acyclic graph (DAG). A heterogeneous computing system needs an effective and efficient scheduling framework, which orchestrates all the computing hardware to perform multiple competitive DM workflows. Motivated by the need for a practical solution of the scheduling problem for the DM workflow, this paper proposes a dynamic DAG scheduling algorithm according to the characteristics of an execution time estimation model for DM jobs. Based on an approximate estimation of job execution time, this algorithm first maps DM jobs to machines in a decentralized and diligent (defined in this paper) manner. Then the performance of this initial mapping can be improved through job migrations when necessary. The scheduling heuristic used considers the factors of both the minimal completion time criterion and the critical path in a DAG. We implement this system in an established multi-agent system environment, in which the reuse of existing DM algorithms is achieved by encapsulating them into agents. The system evaluation and its usage in oil well logging analysis are also discussed.
关键词	data mining heterogeneous computing directed acyclic graph multi-agent system environment
收录类别	SCI
语种	英语
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Theory & Methods
WOS记录号	WOS:000241201000003
出版者	BLACKWELL PUBLISHING
引用统计	被引频次：4[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/10526
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Luo, Ping
作者单位	1.Chinese Acad Sci, Key Lab Intelligent Informat Proc, Inst Comp Technol, Beijing, Peoples R China 2.Brunel Univ, Uxbridge UB8 3PH, Middx, England
推荐引用方式 GB/T 7714	Luo, Ping,Lu, Kevin,Huang, Rui,et al. A heterogeneous computing system for data mining workflows in multi-agent environments[J]. EXPERT SYSTEMS,2006,23(5):258-272.
APA	Luo, Ping,Lu, Kevin,Huang, Rui,He, Qing,&Shi, Zhongzhi.(2006).A heterogeneous computing system for data mining workflows in multi-agent environments.EXPERT SYSTEMS,23(5),258-272.
MLA	Luo, Ping,et al."A heterogeneous computing system for data mining workflows in multi-agent environments".EXPERT SYSTEMS 23.5(2006):258-272.