NeUDF: Learning Neural Unsigned Distance Fields With Volume Rendering

doi:10.1109/TPAMI.2023.3335353

	NeUDF: Learning Neural Unsigned Distance Fields With Volume Rendering
	Liu, Yu-Tao 1,2; Wang, Li 1,2; Yang, Jie 1; Chen, Weikai 3; Meng, Xiaoxu 3; Yang, Bo 3; Gao, Lin 1,2
	2024-04-01
发表期刊	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
ISSN	0162-8828
卷号	46 期号:4 页码:2364-2377
摘要	Multi-view shape reconstruction has achieved impressive progresses thanks to the latest advances in the neural implicit rendering. However, existing methods based on signed distance function (SDF) are limited to closed surfaces, failing to reconstruct a wide range of real-world objects that contain open-surface structures. In this work, we introduce a new neural rendering framework, coded NeUDF, that can reconstruct surfaces with arbitrary topologies solely from multi-view supervision. To gain the flexibility of representing arbitrary surfaces, NeUDF leverages the unsigned distance function (UDF) as surface representation. While a naive extension of SDF-based neural renderer cannot scale to UDF, we formalize the rules of neural volume rendering for open surface reconstruction (e.g., self-consistent, unbiased, occlusion-aware), and derive a dedicated rendering weight function specially tailored for UDF. Furthermore, to cope with open surface rendering, where the in/out test is no longer valid, we present a dedicated normal regularization strategy to resolve the surface orientation ambiguity. We extensively evaluate our method over a number of challenging datasets, including two typical open surface datasets MGN (Bhatnagar et al., 2019) and Deep Fashion 3D (Zhu et al., 2020). Experimental results demonstrate that NeUDF can significantly outperform the state-of-the-art methods in the task of multi-view surface reconstruction, especially for the complex shapes with open boundaries.
关键词	Surface reconstruction Rendering (computer graphics) Image reconstruction Shape Image color analysis Topology Surface texture Unsigned distance fields volume rendering shape reconstruction from multi-view images open surfaces
DOI	10.1109/TPAMI.2023.3335353
收录类别	SCI
语种	英语
资助项目	CCF-Tencent Open Fund
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001180891600032
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38723
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Gao, Lin
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing Key Lab Mobile Comp & Pervas Device, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Tencent Games Digital Content Technol Ctr, Santa Monica, CA 90404 USA
推荐引用方式 GB/T 7714	Liu, Yu-Tao,Wang, Li,Yang, Jie,et al. NeUDF: Learning Neural Unsigned Distance Fields With Volume Rendering[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2024,46(4):2364-2377.
APA	Liu, Yu-Tao.,Wang, Li.,Yang, Jie.,Chen, Weikai.,Meng, Xiaoxu.,...&Gao, Lin.(2024).NeUDF: Learning Neural Unsigned Distance Fields With Volume Rendering.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,46(4),2364-2377.
MLA	Liu, Yu-Tao,et al."NeUDF: Learning Neural Unsigned Distance Fields With Volume Rendering".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 46.4(2024):2364-2377.