Frequency-importance gaussian splatting for real-time lightweight radiance field rendering

doi:10.1007/s11042-024-18679-x

	Frequency-importance gaussian splatting for real-time lightweight radiance field rendering
	Chen, Lizhe 1; Hu, Yan 1; Zhang, Yu 1; Ge, Yuyao 1,2,3; Zhang, Haoyu 1; Cai, Xingquan 1
	2024-03-12
发表期刊	MULTIMEDIA TOOLS AND APPLICATIONS
ISSN	1380-7501
页码	25
摘要	Recently, there have been significant developments in the realm of novel view synthesis relying on radiance fields. By incorporating the Splatting technique, a new approach named Gaussian Splatting has achieved superior rendering quality and real-time performance. However, the training process of the approach incurs significant performance overhead, and the model obtained from training is very large. To address these challenges, we improve Gaussian Splatting and propose Frequency-Importance Gaussian Splatting. Our method reduces the performance overhead by extracting the frequency features of the scene. First, we analyze the advantages and limitations of the spatial sampling strategy of the Gaussian Splatting method from the perspective of sampling theory. Second, we design the Enhanced Gaussian to more effectively express the high-frequency information, while reducing the performance overhead. Third, we construct a frequency-sensitive loss function to enhance the network's ability to perceive the frequency domain and optimize the spatial structure of the scene. Finally, we propose a Dynamically Adaptive Density Control Strategy based on the degree of reconstruction of the background of the scene, which adaptive the spatial sample point generation strategy dynamically according to the training results and prevents the generation of redundant data in the model. We conducted experiments on several commonly used datasets, and the results show that our method has significant advantages over the original method in terms of memory overhead and storage usage and can maintain the image quality of the original method.
关键词	Real-time rendering Radiance field Novel view synthesis Lightweight
DOI	10.1007/s11042-024-18679-x
收录类别	SCI
语种	英语
资助项目	Funding Project of Humanities and Social Sciences of the Ministry of Education in China
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Information Systems ; Computer Science, Software Engineering ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:001181121800006
出版者	SPRINGER
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38779
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Cai, Xingquan
作者单位	1.North China Univ Technol, Coll Informat, Beijing 100144, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci, Beijing 101408, Peoples R China
推荐引用方式 GB/T 7714	Chen, Lizhe,Hu, Yan,Zhang, Yu,et al. Frequency-importance gaussian splatting for real-time lightweight radiance field rendering[J]. MULTIMEDIA TOOLS AND APPLICATIONS,2024:25.
APA	Chen, Lizhe,Hu, Yan,Zhang, Yu,Ge, Yuyao,Zhang, Haoyu,&Cai, Xingquan.(2024).Frequency-importance gaussian splatting for real-time lightweight radiance field rendering.MULTIMEDIA TOOLS AND APPLICATIONS,25.
MLA	Chen, Lizhe,et al."Frequency-importance gaussian splatting for real-time lightweight radiance field rendering".MULTIMEDIA TOOLS AND APPLICATIONS (2024):25.