QoE-Driven Antenna Tuning in Cellular Networks With Cooperative Multi-Agent Reinforcement Learning

doi:10.1109/TMC.2022.3230711

	QoE-Driven Antenna Tuning in Cellular Networks With Cooperative Multi-Agent Reinforcement Learning
	Liu, Xuewen 1; Chuai, Gang 2; Wang, Xin 3; Xu, Zhiwei 4; Gao, Weidong 2; Zhang, Kaisa 2; Liu, Qian 5; Maimaiti, Saidiwaerdi 2; Zuo, Peiliang 1
	2024-02-01
发表期刊	IEEE TRANSACTIONS ON MOBILE COMPUTING
ISSN	1536-1233
卷号	23 期号:2 页码:1186-1199
摘要	Antenna tuning plays an essential role in ensuring high quality wireless communications. Targeting for higher Quality of Service (QoS), many existing network antenna tuning schemes are based on expert knowledge, rule-based policies or conventional optimization theory. However, maximizing the traffic-related QoS does not guarantee that all customers experience good services. In addition, existing schemes are often limited to some handcrafted rules or heuristics and lack of adaptability especially in a time-varying environment. Quality of Experience (QoE), a user-centric metric, can better measure users' satisfaction for services in wireless networks. This article proposes the cooperative tuning of antennas based on QoE, a paradigm shift from network-centric QoS to user-centric QoE domain. In a normal cellular network, besides the need of improving the overall QoE, it requires handling faults from different cells. As Multi-agent Reinforcement Learning (MARL) has the capability of self-learning the dynamics of environment, we propose an antenna configuration algorithm based on multi-goal MARL. In our framework, agents from different cells not only need to cooperate with each other to achieve the global goal of increasing the overall QoE of the wireless network but also complete some personal goals by combating the faults encountered in their own cells. To accelerate the training efficiency, we introduce a novel two-stage curriculum learning. To reduce the collection time of each QoE sample, we develop an accurate and timely QoE/QoS mapping model with the cascading of a Random Forest Classifier (RFC) and a Deep Neural Network (DNN) (abbreviated as RFC-DNN), which can help us obtain QoE by collecting QoS measurements and perform QoE-based antenna configurations with smaller time granularity. Our proposed RFC-DNN model can reduce the time by 70% when predicting the QoE of a single sample. A huge amount of time will be saved in MARL when tens of thousands of transitions/samples need to be collected. The performance results show that our proposed antenna tuning schemes can not only address specific faults in each cell, but also significantly improve the global average QoE with a faster and more stable convergence speed.
关键词	Antenna tuning multi-goal MARL QoE/QoS mapping self-optimization
DOI	10.1109/TMC.2022.3230711
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Project of China
WOS研究方向	Computer Science ; Telecommunications
WOS类目	Computer Science, Information Systems ; Telecommunications
WOS记录号	WOS:001140706700043
出版者	IEEE COMPUTER SOC
引用统计	被引频次：4[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38386
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhang, Kaisa; Zuo, Peiliang
作者单位	1.Beijing Elect Sci & Technol Inst, Dept Elect & Commun Engn, Beijing 100070, Peoples R China 2.Beijing Univ Posts & Telecommun, Dept Informat & Commun Engn, Beijing 100876, Peoples R China 3.SUNY Stony Brook, Dept Elect & Comp Engn, Stony Brook, NY 11794 USA 4.Chinese Acad Sci, Inst Comp, Beijing 100080, Peoples R China 5.Chongqing Univ Posts & Telecommun, Sch Commun & Informat Engn, Chongqing 400065, Peoples R China
推荐引用方式 GB/T 7714	Liu, Xuewen,Chuai, Gang,Wang, Xin,et al. QoE-Driven Antenna Tuning in Cellular Networks With Cooperative Multi-Agent Reinforcement Learning[J]. IEEE TRANSACTIONS ON MOBILE COMPUTING,2024,23(2):1186-1199.
APA	Liu, Xuewen.,Chuai, Gang.,Wang, Xin.,Xu, Zhiwei.,Gao, Weidong.,...&Zuo, Peiliang.(2024).QoE-Driven Antenna Tuning in Cellular Networks With Cooperative Multi-Agent Reinforcement Learning.IEEE TRANSACTIONS ON MOBILE COMPUTING,23(2),1186-1199.
MLA	Liu, Xuewen,et al."QoE-Driven Antenna Tuning in Cellular Networks With Cooperative Multi-Agent Reinforcement Learning".IEEE TRANSACTIONS ON MOBILE COMPUTING 23.2(2024):1186-1199.