Deep Distributional Reinforcement Learning-Based Adaptive Routing With Guaranteed Delay Bounds

doi:10.1109/TNET.2024.3425652

	Deep Distributional Reinforcement Learning-Based Adaptive Routing With Guaranteed Delay Bounds
	Liu, Jianmin 1; Li, Dan 1,2; Xu, Yongjun 3
	2024-07-15
发表期刊	IEEE-ACM TRANSACTIONS ON NETWORKING
ISSN	1063-6692
页码	15
摘要	Real-time applications that require timely data delivery over wireless multi-hop networks within specified deadlines are growing increasingly. Effective routing protocols that can guarantee real-time QoS are crucial, yet challenging, due to the unpredictable variations in end-to-end delay caused by unreliable wireless channels. In such conditions, the upper bound on the end-to-end delay, i.e., worst-case end-to-end delay, should be guaranteed within the deadline. However, existing routing protocols with guaranteed delay bounds cannot strictly guarantee real-time QoS because they assume that the worst-case end-to-end delay is known and ignore the impact of routing policies on the worst-case end-to-end delay determination. In this paper, we relax this assumption and propose DDRL-ARGB, an Adaptive Routing with Guaranteed delay Bounds using Deep Distributional Reinforcement Learning (DDRL). DDRL-ARGB adopts DDRL to jointly determine the worst-case end-to-end delay and learn routing policies. To accurately determine worst-case end-to-end delay, DDRL-ARGB employs a quantile regression deep Q-network to learn the end-to-end delay cumulative distribution. To guarantee real-time QoS, DDRL-ARGB optimizes routing decisions under the constraint of worst-case end-to-end delay within the deadline. To improve traffic congestion, DDRL-ARGB considers the network congestion status when making routing decisions. Extensive results show that DDRL-ARGB can accurately calculate worst-case end-to-end delay, and can strictly guarantee real-time QoS under a small tolerant violation probability against two state-of-the-art routing protocols.
关键词	Deep distributional reinforcement learning real-time QoS guarantees adaptive routing wireless multi-hop networks Deep distributional reinforcement learning real-time QoS guarantees adaptive routing wireless multi-hop networks
DOI	10.1109/TNET.2024.3425652
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2022YFB2901200]
WOS研究方向	Computer Science ; Engineering ; Telecommunications
WOS类目	Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic ; Telecommunications
WOS记录号	WOS:001273051400001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/39680
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Li, Dan
作者单位	1.Tsinghua Univ, Dept Comp Sci & Technol, Beijing 100084, Peoples R China 2.Zhongguancun Lab, Beijing 100095, Peoples R China 3.Inst Comp Technol, Chinese Acad Sci, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Liu, Jianmin,Li, Dan,Xu, Yongjun. Deep Distributional Reinforcement Learning-Based Adaptive Routing With Guaranteed Delay Bounds[J]. IEEE-ACM TRANSACTIONS ON NETWORKING,2024:15.
APA	Liu, Jianmin,Li, Dan,&Xu, Yongjun.(2024).Deep Distributional Reinforcement Learning-Based Adaptive Routing With Guaranteed Delay Bounds.IEEE-ACM TRANSACTIONS ON NETWORKING,15.
MLA	Liu, Jianmin,et al."Deep Distributional Reinforcement Learning-Based Adaptive Routing With Guaranteed Delay Bounds".IEEE-ACM TRANSACTIONS ON NETWORKING (2024):15.