DreamUDF: Generating Unsigned Distance Fields from A Single Image

doi:10.1145/3687769

	DreamUDF: Generating Unsigned Distance Fields from A Single Image
	Liu, Yu-tao 1,2,3; Gao, Xuan 1,2,3; Chen, Weikai 4; Yang, Jie 1; Meng, Xiaoxu 4; Yang, Bo 4; Gao, Lin 1,2,3
	2024-12-01
发表期刊	ACM TRANSACTIONS ON GRAPHICS
ISSN	0730-0301
卷号	43 期号:6 页码:21
摘要	Recent advances in diffusion models and neural implicit surfaces have shown promising progress in generating 3D models. However, existing generative frameworks are limited to closed surfaces, failing to cope with a wide range of commonly seen shapes that have open boundaries. In this work, we present DreamUDF, a novel framework for generating high-quality 3D objects with arbitrary topologies from a single image. To address the challenge of generating proper topology given sparse and ambiguous observations, we propose to incorporate both the data priors from a multi-view diffusion model and the geometry priors brought by an unsigned distance field (UDF) reconstructor. In particular, we leverage a joint framework that consists of 1) a generation module that produces a neural radiance field for photo realistic renderings from arbitrary views; and 2) a reconstruction module that distills the learnable radiance field into surfaces with arbitrary topologies. We further introduce a field coupler that bridges the radiance field and UDF under a novel optimization scheme. This allows the two modules to mutually boost each other during training. Extensive experiments and evaluations demonstrate that DreamUDF achieves high-quality reconstruction and robust 3D generation on both closed and open surfaces with arbitrary topologies, compared to the previous works.
关键词	Unsigned distance field single-view 3D generation diffusion model neural radiance field
DOI	10.1145/3687769
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation China[62322210] ; National Natural Science Foundation China[62302484] ; Innovation Funding ICT, CAS[E461020] ; Beijing Municipal Natural Science Foundation for Distinguished Young Scholars[JQ21013] ; Beijing Munic-ipal Science and Technology Commission[Z231100005923031] ; China Postdoctoral Science Foundation[BX20230377] ; China Postdoctoral Science Foundation[2023M743568]
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:001367506000001
出版者	ASSOC COMPUTING MACHINERY
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41141
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Gao, Lin
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing Key Lab Mobile Comp & Pervas Device, Beijing, Peoples R China 2.Sch Comp Sci & Technol, Beijing, Peoples R China 3.Univ Chinese Acad Sci, Beijing, Peoples R China 4.Tencent Games, DCC Algorithm Res Ctr, Shenzhen, Peoples R China
推荐引用方式 GB/T 7714	Liu, Yu-tao,Gao, Xuan,Chen, Weikai,et al. DreamUDF: Generating Unsigned Distance Fields from A Single Image[J]. ACM TRANSACTIONS ON GRAPHICS,2024,43(6):21.
APA	Liu, Yu-tao.,Gao, Xuan.,Chen, Weikai.,Yang, Jie.,Meng, Xiaoxu.,...&Gao, Lin.(2024).DreamUDF: Generating Unsigned Distance Fields from A Single Image.ACM TRANSACTIONS ON GRAPHICS,43(6),21.
MLA	Liu, Yu-tao,et al."DreamUDF: Generating Unsigned Distance Fields from A Single Image".ACM TRANSACTIONS ON GRAPHICS 43.6(2024):21.