SeG-Gaussian:Segmentation-Guided 3D Gaussian Optimization for Novel View Synthesis

doi:10.1109/TVCG.2025.3615421

CSpace

	SeG-Gaussian:Segmentation-Guided 3D Gaussian Optimization for Novel View Synthesis
	Zhang, Ling-Xiao 1,2; Jiang, Chenbo 3; Lai, Yu-Kun 4; Gao, Lin 1,2
	2025-12-01
发表期刊	IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
ISSN	1077-2626
卷号	31 期号:12 页码:10815-10827
摘要	Radiance field based methods have recently revolutionized novel view synthesis of scenes captured with multi-view photos. A significant recent advance is 3D Gaussian Splatting (3DGS), which utilizes a set of 3D Gaussians to represent a radiance field, yielding high-fidelity results in real-time rendering. However, we have observed that 3DGS struggles to capture the necessary details in sparsely observed regions, where there is not enough gradient for effective split and clone operations. In this paper, we present a novel solution to address this limitation. Our key idea is to leverage segmentation information to identify poorly optimized regions within the 3D Gaussian representation. By applying split or clone operations on the corresponding 3D Gaussians in these regions, we aim to refine the spatial distribution of Gaussians and enhance the overall quality of high-fidelity 3D scene reconstruction. To further optimize the reconstruction process, we introduce two spatial regularization terms: repulsion loss and smoothness loss. These terms effectively minimize overlap and redundancy among Gaussians, reducing outliers in the synthesized geometry. By incorporating these regularization techniques, our approach achieves state-of-the-art performance in real-time novel view synthesis and significantly improves visibility in less observed regions, leading to a more compact and accurate 3D scene representation.
关键词	Three-dimensional displays Rendering (computer graphics) Neural radiance field Cloning Real-time systems Training Geometry Optimization Graphical models Distribution functions Gaussian splatting radiance fields semantic guidance regularization
DOI	10.1109/TVCG.2025.3615421
收录类别	SCI
语种	英语
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:001611559200016
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/43102
专题	中国科学院计算技术研究所
通讯作者	Gao, Lin
作者单位	1.Chinese Acad Sci, Beijing Key Lab Mobile Comp & Pervas Device, Inst Comp Technol, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 101408, Peoples R China 3.McGill Univ, Montreal, PQ H3A 0G4, Canada 4.Cardiff Univ, Sch Comp Sci & Informat, Cardiff CF24 4AG, Wales
推荐引用方式 GB/T 7714	Zhang, Ling-Xiao,Jiang, Chenbo,Lai, Yu-Kun,et al. SeG-Gaussian:Segmentation-Guided 3D Gaussian Optimization for Novel View Synthesis[J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,2025,31(12):10815-10827.
APA	Zhang, Ling-Xiao,Jiang, Chenbo,Lai, Yu-Kun,&Gao, Lin.(2025).SeG-Gaussian:Segmentation-Guided 3D Gaussian Optimization for Novel View Synthesis.IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,31(12),10815-10827.
MLA	Zhang, Ling-Xiao,et al."SeG-Gaussian:Segmentation-Guided 3D Gaussian Optimization for Novel View Synthesis".IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 31.12(2025):10815-10827.