A Value Iteration Algorithm for Stochastic Linear Quadratic Regulator

doi:10.1007/s10957-025-02777-3

	A Value Iteration Algorithm for Stochastic Linear Quadratic Regulator
	Wang, Hongxia 1; Liu, Yihang 2; Liu, Xiangqian 1
	2025-07-18
发表期刊	JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS
ISSN	0022-3239
卷号	207 期号:2 页码:14
摘要	In this paper, we propose a novel value iteration algorithm for online adaptive optimal control of discrete-time stochastic linear quadratic regulator (LQR) problems. The algorithm iteratively solves the algebraic Riccati equation (ARE) using online information of states and inputs, without requiring the knowledge of the system dynamics. It does not require a discount factor because the issue of excitation noise bias is not present in our algorithm. Firstly, we review the optimal solution for the stochastic LQR problem. Secondly, we offer an offline model-based algorithm for solving ARE and prove its convergence. Thirdly, we present a data-driven online value iteration algorithm for solving ARE. Finally, we evaluate the proposed algorithm by an example.
关键词	Stochastic linear quadratic regulation Reinforcement learning Value iteration Policy iteration
DOI	10.1007/s10957-025-02777-3
收录类别	SCI
语种	英语
资助项目	the Original Exploratory Program Project of National Natural Science Foundation of China[62450004] ; Original Exploratory Program Project of National Natural Science Foundation of China[U23A20325] ; National Natural Science Foundation of China[ZR2024MF045] ; Natural Science Foundation of Shandong Province
WOS研究方向	Operations Research & Management Science ; Mathematics
WOS类目	Operations Research & Management Science ; Mathematics, Applied
WOS记录号	WOS:001531142000002
出版者	SPRINGER/PLENUM PUBLISHERS
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/42083
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Hongxia
作者单位	1.Shandong Univ Sci & Technol, Coll Elect Engn & Automat, Qingdao 266510, Peoples R China 2.Chinese Acad Sci, CAS Key Lab AI Safety, Inst Comp Technol, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Wang, Hongxia,Liu, Yihang,Liu, Xiangqian. A Value Iteration Algorithm for Stochastic Linear Quadratic Regulator[J]. JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS,2025,207(2):14.
APA	Wang, Hongxia,Liu, Yihang,&Liu, Xiangqian.(2025).A Value Iteration Algorithm for Stochastic Linear Quadratic Regulator.JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS,207(2),14.
MLA	Wang, Hongxia,et al."A Value Iteration Algorithm for Stochastic Linear Quadratic Regulator".JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS 207.2(2025):14.