SceneHGN: Hierarchical Graph Networks for 3D Indoor Scene Generation With Fine-Grained Geometry

doi:10.1109/TPAMI.2023.3237577

	SceneHGN: Hierarchical Graph Networks for 3D Indoor Scene Generation With Fine-Grained Geometry
	Gao, Lin 1,2; Sun, Jia-Mu 1,2; Mo, Kaichun 3; Lai, Yu-Kun 4; Guibas, Leonidas J.; Yang, Jie 1,2
	2023-07-01
发表期刊	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
ISSN	0162-8828
卷号	45 期号:7 页码:8902-8919
摘要	3D indoor scenes are widely used in computer graphics, with applications ranging from interior design to gaming to virtual and augmented reality. They also contain rich information, including room layout, as well as furniture type, geometry, and placement. High-quality 3D indoor scenes are highly demanded while it requires expertise and is time-consuming to design high-quality 3D indoor scenes manually. Existing research only addresses partial problems: some works learn to generate room layout, and other works focus on generating detailed structure and geometry of individual furniture objects. However, these partial steps are related and should be addressed together for optimal synthesis. We propose SCENEHGN, a hierarchical graph network for 3D indoor scenes that takes into account the full hierarchy from the room level to the object level, then finally to the object part level. Therefore for the first time, our method is able to directly generate plausible 3D room content, including furniture objects with fine-grained geometry, and their layout. To address the challenge, we introduce functional regions as intermediate proxies between the room and object levels to make learning more manageable. To ensure plausibility, our graph-based representation incorporates both vertical edges connecting child nodes with parent nodes from different levels, and horizontal edges encoding relationships between nodes at the same level. Our generation network is a conditional recursive neural network (RvNN) based variational autoencoder (VAE) that learns to generate detailed content with fine-grained geometry for a room, given the room boundary as the condition. Extensive experiments demonstrate that ourmethod produces superior generation results, even when comparing results of partial steps with alternative methods that can only achieve these. We also demonstrate that our method is effective for various applications such as part-level room editing, room interpolation, and room generation by arbitrary room boundaries.
关键词	3Dindoor scene synthesis deep generative model fine-grained mesh generation graph neural network recursive neural network relationship graphs variational autoencoder
DOI	10.1109/TPAMI.2023.3237577
收录类别	SCI
语种	英语
资助项目	ARL[W911NF-21-2-0104] ; Vannevar Bush Faculty Fellowship ; Beijing Municipal Natural Science Foundation[JQ21013] ; National Natural Science Foundation of China[6261136007] ; Open Research Projects of Zhejiang Lab[2021KE0AB06] ; Youth Innovation Promotion Association CAS
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001004665900064
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/21262
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Gao, Lin
作者单位	1.Chinese Acad Sci, Beijing Key Lab Mobile Comp & Pervas Device, Inst Comp Technol, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Stanford Univ, Stanford, CA 94305 USA 4.Cardiff Univ, Sch Comp Sci & Informat, Cardiff CF10 3AT, Wales
推荐引用方式 GB/T 7714	Gao, Lin,Sun, Jia-Mu,Mo, Kaichun,et al. SceneHGN: Hierarchical Graph Networks for 3D Indoor Scene Generation With Fine-Grained Geometry[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2023,45(7):8902-8919.
APA	Gao, Lin,Sun, Jia-Mu,Mo, Kaichun,Lai, Yu-Kun,Guibas, Leonidas J.,&Yang, Jie.(2023).SceneHGN: Hierarchical Graph Networks for 3D Indoor Scene Generation With Fine-Grained Geometry.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,45(7),8902-8919.
MLA	Gao, Lin,et al."SceneHGN: Hierarchical Graph Networks for 3D Indoor Scene Generation With Fine-Grained Geometry".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 45.7(2023):8902-8919.