A heterogeneous computing system for data mining workflows

	A heterogeneous computing system for data mining workflows
	Luo, Ping ; Lu, Kevin ; He, Qing ; Shi, Zhongzhi
	2006
发表期刊	FLEXIBLE AND EFFICIENT INFORMATION HANDLING
ISSN	0302-9743
卷号	4042 页码:177-189
摘要	The computing-intensive Data Mining (DM) process calls for the support of a Heterogeneous Computing (HC) system, which consists of multiple computers with different configurations, connected by a high-speed LAN, for increased computational power and resources. DM process can be described as a multi-phase pipeline process, and in each phase there could be many optional methods. This makes the workflow of DM very complex and can be modelled only by a Directed Acyclic Graph (DAG). An HC system needs an effective and efficient scheduling framework, which orchestrates all the computing hardware to perform multiple competitive DM workflows. Motivated by the need of a practical solution of the scheduling problem for the DM workflow, this paper proposes a dynamic DAG scheduling algorithm according to the characteristics of execution time estimation model for DM jobs. Based on an approximate estimation of job execution time, this algorithm first maps DM jobs to machines in a decentralized and diligent (defined in this paper) manner. Then the performance of this initial mapping can be improved through job migrations when necessary. The scheduling heuristic used in it considers the factors of both the minimal completion time criterion and the critical path in a DAG. We implement this system in an established Multi-Agent System (MAS) environment, in which the reuse of existing DM algorithms is achieved by encapsulating them into agents. Practical classification problems are used to test and measure the system performance. The detailed experiment procedure and result analysis are also discussed in this paper.
收录类别	SCI
语种	英语
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Theory & Methods
WOS记录号	WOS:000239454500015
出版者	SPRINGER-VERLAG BERLIN
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/10436
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Luo, Ping
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100080, Peoples R China 2.Brunel Univ, Uxbridge UB8 3PH, Middx, England 3.Chinese Acad Sci, Grad Sch, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Luo, Ping,Lu, Kevin,He, Qing,et al. A heterogeneous computing system for data mining workflows[J]. FLEXIBLE AND EFFICIENT INFORMATION HANDLING,2006,4042:177-189.
APA	Luo, Ping,Lu, Kevin,He, Qing,&Shi, Zhongzhi.(2006).A heterogeneous computing system for data mining workflows.FLEXIBLE AND EFFICIENT INFORMATION HANDLING,4042,177-189.
MLA	Luo, Ping,et al."A heterogeneous computing system for data mining workflows".FLEXIBLE AND EFFICIENT INFORMATION HANDLING 4042(2006):177-189.