QR-DeepONet: resolve abnormal convergence issue in deep operator network

doi:10.1088/2632-2153/ada0a5

	QR-DeepONet: resolve abnormal convergence issue in deep operator network
	Zhao, Jie 1; Xie, Biwei 1,2; Li, Xingquan 1,3
	2024-12-01
发表期刊	MACHINE LEARNING-SCIENCE AND TECHNOLOGY
卷号	5 期号:4 页码:10
摘要	Deep operator network (DeepONet) has been proven to be highly successful in operator learning tasks. Theoretical analysis indicates that the generation error of DeepONet should decrease as the basis dimension increases, thus providing a systematic way to reduce its generalization errors (GEs) by varying the network hyperparameters. However, in practice, we found that, depending on the problem being solved and the activation function used, the GEs fluctuate unpredictably, contrary to theoretical expectations. Upon analyzing the output matrix of the trunk net, we determined that this behavior stems from the learned basis functions being highly linearly dependent, which limits the expressivity of the vanilla DeepONet. To address these limitations, we propose QR decomposition enhanced DeepONet (QR-DeepONet), an enhanced version of DeepONet using QR decomposition. These modifications ensured that the learned basis functions were linearly independent and orthogonal to each other. The numerical results demonstrate that the GEs of QR-DeepONet follow theoretical predictions that decrease monotonically as the basis dimension increases and outperform vanilla DeepONet. Consequently, the proposed method successfully fills the gap between the theory and practice.
关键词	operator learning QR decomposition machine learning neural network
DOI	10.1088/2632-2153/ada0a5
收录类别	SCI
语种	英语
资助项目	Major Key Project of PCL ; [PCL2023A03]
WOS研究方向	Computer Science ; Science & Technology - Other Topics
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Interdisciplinary Applications ; Multidisciplinary Sciences
WOS记录号	WOS:001388376700001
出版者	IOP Publishing Ltd
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/40803
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhao, Jie
作者单位	1.Pengcheng Lab, Shenzhen 518055, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 3.Minnan Normal Univ, Sch Math & Stat, Zhangzhou, Peoples R China
推荐引用方式 GB/T 7714	Zhao, Jie,Xie, Biwei,Li, Xingquan. QR-DeepONet: resolve abnormal convergence issue in deep operator network[J]. MACHINE LEARNING-SCIENCE AND TECHNOLOGY,2024,5(4):10.
APA	Zhao, Jie,Xie, Biwei,&Li, Xingquan.(2024).QR-DeepONet: resolve abnormal convergence issue in deep operator network.MACHINE LEARNING-SCIENCE AND TECHNOLOGY,5(4),10.
MLA	Zhao, Jie,et al."QR-DeepONet: resolve abnormal convergence issue in deep operator network".MACHINE LEARNING-SCIENCE AND TECHNOLOGY 5.4(2024):10.