A Survey on Causal Reinforcement Learning

doi:10.1109/TNNLS.2024.3403001

	A Survey on Causal Reinforcement Learning
	Zeng, Yan 1,2; Cai, Ruichu 3,4; Sun, Fuchun 2; Huang, Libo 5; Hao, Zhifeng 6
	2024-11-28
发表期刊	IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
ISSN	2162-237X
页码	21
摘要	While reinforcement learning (RL) achieves tremendous success in sequential decision-making problems of many domains, it still faces key challenges of data inefficiency and the lack of interpretability. Interestingly, many researchers have leveraged insights from the causality literature recently, bringing forth flourishing works to unify the merits of causality and address well the challenges from RL. As such, it is of great necessity and significance to collate these causal RL (CRL) works, offer a review of CRL methods, and investigate the potential functionality from causality toward RL. In particular, we divide the existing CRL approaches into two categories according to whether their causality-based information is given in advance or not. We further analyze each category in terms of the formalization of different models, ranging from the Markov decision process (MDP), partially observed MDP (POMDP), multiarmed bandits (MABs), imitation learning (IL), and dynamic treatment regime (DTR). Each of them represents a distinct type of causal graphical illustration. Moreover, we summarize the evaluation matrices and open sources, while we discuss emerging applications, along with promising prospects for the future development of CRL.
关键词	Cause effect analysis Surveys Mathematical models Drugs Sun Reviews Reinforcement learning Decision making Data models Computer science Causal discovery causal inference causal reinforcement learning (CRL) Markov decision process (MDP) sequential decision-making
DOI	10.1109/TNNLS.2024.3403001
收录类别	SCI
语种	英语
资助项目	New Generation Artificial Intelligence Key Field Research and Development Plan of Guangdong Province[2021B0101410002] ; National Science Fund for Excellent Young Scholars[62122022] ; National Science and Technology Major Project of the Ministry of Science and Technology of China[2018AAA0102900] ; Natural Science Foundation of China[61876043] ; Natural Science Foundation of China[61976052] ; Natural Science Foundation of China[62306019] ; Guangdong Provincial Science and Technology Innovation Strategy Fund[2019B121203012] ; Beijing Natural Science Foundation[4244098] ; China Postdoctoral Science Foundation[2022M711812] ; Tsinghua University Initiative Scientific Research Program ; Research and Development Program of Beijing Municipal Education Commission[KM202410011016]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:001367610400001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41137
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Sun, Fuchun
作者单位	1.Beijing Technol & Business Univ, Sch Math & Stat, Beijing 102401, Peoples R China 2.Tsinghua Univ, Dept Comp Sci & Technol, Beijing 100048, Peoples R China 3.Guangdong Univ Technol, Sch Comp Sci, Guangzhou 510006, Peoples R China 4.Pazhou Lab Huangpu, Guangzhou 510555, Peoples R China 5.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 6.Shantou Univ, Coll Sci, Shantou 515063, Peoples R China
推荐引用方式 GB/T 7714	Zeng, Yan,Cai, Ruichu,Sun, Fuchun,et al. A Survey on Causal Reinforcement Learning[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2024:21.
APA	Zeng, Yan,Cai, Ruichu,Sun, Fuchun,Huang, Libo,&Hao, Zhifeng.(2024).A Survey on Causal Reinforcement Learning.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,21.
MLA	Zeng, Yan,et al."A Survey on Causal Reinforcement Learning".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2024):21.