Data-efficient 3D instance segmentation by transferring knowledge from synthetic scans

doi:10.1016/j.patrec.2024.02.001

	Data-efficient 3D instance segmentation by transferring knowledge from synthetic scans
	Wu, Xiaodong 1; Wang, Ruiping 1; Chen, Xilin 1
	2024-03-01
发表期刊	PATTERN RECOGNITION LETTERS
ISSN	0167-8655
卷号	179 页码:151-157
摘要	The 3D comprehension ability of indoor environments is critical for robots. While deep learning-based methods have improved performance, they require significant amounts of annotated training data. Nevertheless, the cost of scanning and annotating point cloud data in real scenes is high, leading to data scarcity. Consequently, there is an urgent need to investigate data-efficient methods for point cloud instance segmentation. To tackle this issue, we propose to leverage the geometric and scene context knowledge inherent in synthetic data to reduce the need for annotation on real data. Specifically, we simulate the process of human scanning and collecting point cloud data in real -world scenes and construct three large-scale synthetic point cloud datasets using synthetic scenes. The scale of these three datasets is more than ten times that of currently available real -world data. Experimental results demonstrate that the incorporation of synthetic point cloud data can increase instance segmentation performance by over 18.8 percentage points. Further, to address the problem of domain shift between synthetic and real data, we propose a target-aware pre -training method. It integrates both real and synthetic data during the pre -training process, allowing the model to learn a feature representation that can effectively generalize to downstream real data. Experiments show that our method achieved stable improvements on all three synthetic datasets. The data and code will be publicly available in the future.
关键词	Point cloud segmentation Synthetic data Domain adaptation
DOI	10.1016/j.patrec.2024.02.001
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2021ZD0111901] ; Natural Science Foundation of China[U21B2025] ; Natural Science Foundation of China[U19B2036]
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:001197118000001
出版者	ELSEVIER
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38767
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Ruiping
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc Chinese Acad Sci, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Wu, Xiaodong,Wang, Ruiping,Chen, Xilin. Data-efficient 3D instance segmentation by transferring knowledge from synthetic scans[J]. PATTERN RECOGNITION LETTERS,2024,179:151-157.
APA	Wu, Xiaodong,Wang, Ruiping,&Chen, Xilin.(2024).Data-efficient 3D instance segmentation by transferring knowledge from synthetic scans.PATTERN RECOGNITION LETTERS,179,151-157.
MLA	Wu, Xiaodong,et al."Data-efficient 3D instance segmentation by transferring knowledge from synthetic scans".PATTERN RECOGNITION LETTERS 179(2024):151-157.