Interactive NeRF Geometry Editing With Shape Priors

doi:10.1109/TPAMI.2023.3315068

	Interactive NeRF Geometry Editing With Shape Priors
	Yuan, Yu-Jie 1,2; Sun, Yang-Tian 1,2; Lai, Yu-Kun 3; Ma, Yuewen 4; Jia, Rongfei 5; Kobbelt, Leif 6; Gao, Lin 1,2
	2023-12-01
发表期刊	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
ISSN	0162-8828
卷号	45 期号:12 页码:14821-14837
摘要	Neural Radiance Fields (NeRFs) have shown great potential for tasks like novel view synthesis of static 3D scenes. Since NeRFs are trained on a large number of input images, it is not trivial to change their content afterwards. Previous methods to modify NeRFs provide some control but they do not support direct shape deformation which is common for geometry representations like triangle meshes. In this paper, we present a NeRF geometry editing method that first extracts a triangle mesh representation of the geometry inside a NeRF. This mesh can be modified by any 3D modeling tool (we use ARAP mesh deformation). The mesh deformation is then extended into a volume deformation around the shape which establishes a mapping between ray queries to the deformed NeRF and the corresponding queries to the original NeRF. The basic shape editing mechanism is extended towards more powerful and more meaningful editing handles by generating box abstractions of the NeRF shapes which provide an intuitive interface to the user. By additionally assigning semantic labels, we can even identify and combine parts from different objects. We demonstrate the performance and quality of our method in a number of experiments on synthetic data as well as real captured scenes.
关键词	Neural radiance fields geometry editing shape deformation interactive editing
DOI	10.1109/TPAMI.2023.3315068
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[62061136007] ; Beijing Municipal Natural Science Foundation for Distinguished Young Scholars[JQ21013] ; Royal Society Newton Advanced Fellowship[NAF \ R2\192151] ; Youth Innovation Promotion AssociationCAS
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001130146400048
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38362
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Gao, Lin
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing Key Lab Mobile Comp & Pervas Device, Beijing 100045, Peoples R China 2.Univ Chinese Acad Sci, Beijing 101408, Peoples R China 3.Cardiff Univ, Sch Comp Sci Informat, Cardiff CF10 3AT, Wales 4.PICO, ByteDance, Beijing 100086, Peoples R China 5.Beijing ZaoWuHuiJing Technol Co Ltd, Beijing 102206, Peoples R China 6.Rhein Westfal TH Aachen, Comp Graph Grp, D-52062 Aachen, Germany
推荐引用方式 GB/T 7714	Yuan, Yu-Jie,Sun, Yang-Tian,Lai, Yu-Kun,et al. Interactive NeRF Geometry Editing With Shape Priors[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2023,45(12):14821-14837.
APA	Yuan, Yu-Jie.,Sun, Yang-Tian.,Lai, Yu-Kun.,Ma, Yuewen.,Jia, Rongfei.,...&Gao, Lin.(2023).Interactive NeRF Geometry Editing With Shape Priors.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,45(12),14821-14837.
MLA	Yuan, Yu-Jie,et al."Interactive NeRF Geometry Editing With Shape Priors".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 45.12(2023):14821-14837.