A Heuristic Big Data Processing Multi Task Efficient Deployment Method Based on QoS Aware Clustering and Bayesian Classification in Cloud Environment

doi:10.1109/TSC.2025.3592228

	A Heuristic Big Data Processing Multi Task Efficient Deployment Method Based on QoS Aware Clustering and Bayesian Classification in Cloud Environment
	Yu, ZiYuan 1; Yang, Peng 1; Wu, WanJing 1; Tan, Liang 1,2; Ye, DanLian 1; She, Kun 3
	2025-09-01
发表期刊	IEEE TRANSACTIONS ON SERVICES COMPUTING
ISSN	1939-1374
卷号	18 期号:5 页码:2517-2530
摘要	The efficient deployment of Big Data processing tasks in cloud environments is the basic core function of Big Data processing, which refers to the effective deployment of tasks to the computing resources of cloud platforms, achieving high-performance and high-throughput data processing. In this process, task deployment needs to consider load balancing on the cloud platform to ensure that tasks can be evenly deployed to each computing node. However, currently in the process of providing services on cloud platforms, the available resources of all hosts will be automatically and dynamically readjusted, and it cannot be guaranteed that each task will be deployed to the host with the most remaining resources. This load imbalance in the platform will result in computational results that cannot be returned to users in a timely and effective manner. So, a heuristic multi-task efficient deployment approach for Big Data processing based on QoS awareness and Bayesian classification in cloud environments called QBC is proposed. The QBC first performs long-run QoS awareness on hosts in the cloud; Then, based on user task requirements, selects host nodes that meet QoS constraints to form a candidate set, and performs Bayesian classification to find the host node which has highest a posteriori probability to serve as the clustering center; third, designs an objective function based on euclidean spatial distance to acquire the optimum host clustering set in the candidate set; Finally, deploys the user's tasks to this optimal host cluster set. The experimental results show that this approach implements optimization of long-run load balancing in Big Data cloud platforms with minimal resource consumption, enhances the ability of the cloud platform to provide external support, and thus promotes efficient deployment of multitasking in Big Data processing under cloud computing. The proposed QBC based framework reduces the overall Energy Consumption by an average of 47.98%, MakeSpan by an average of 24.42%, Total Cost by an average of 30.17%, Average Waiting Time by an average of 36.92%, and the Throughput is increased by an average of 41.93% as compared to the existing algorithms.
关键词	Load management Cloud computing Big Data Quality of service Multitasking Heuristic algorithms Bayes methods Virtual machines Resource management Real-time systems big data Bayesian classification QoS-aware multi-task deployment
DOI	10.1109/TSC.2025.3592228
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[61373126] ; National Natural Science Foundation of China[62301348] ; Sichuan Provincial Science and Technology Department Project[2022YFG0161] ; Sichuan Provincial Science and Technology Department Project[2023YFG0295]
WOS研究方向	Computer Science
WOS类目	Computer Science, Information Systems ; Computer Science, Software Engineering
WOS记录号	WOS:001591693600035
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41609
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Tan, Liang
作者单位	1.Sichuan Normal Univ, Sch Comp Sci, Chengdu 610101, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 3.Univ Elect Sci & Technol China, Chengdu 610054, Peoples R China
推荐引用方式 GB/T 7714	Yu, ZiYuan,Yang, Peng,Wu, WanJing,et al. A Heuristic Big Data Processing Multi Task Efficient Deployment Method Based on QoS Aware Clustering and Bayesian Classification in Cloud Environment[J]. IEEE TRANSACTIONS ON SERVICES COMPUTING,2025,18(5):2517-2530.
APA	Yu, ZiYuan,Yang, Peng,Wu, WanJing,Tan, Liang,Ye, DanLian,&She, Kun.(2025).A Heuristic Big Data Processing Multi Task Efficient Deployment Method Based on QoS Aware Clustering and Bayesian Classification in Cloud Environment.IEEE TRANSACTIONS ON SERVICES COMPUTING,18(5),2517-2530.
MLA	Yu, ZiYuan,et al."A Heuristic Big Data Processing Multi Task Efficient Deployment Method Based on QoS Aware Clustering and Bayesian Classification in Cloud Environment".IEEE TRANSACTIONS ON SERVICES COMPUTING 18.5(2025):2517-2530.