Haisor: Human-aware Indoor Scene Optimization via Deep Reinforcement Learning

doi:10.1145/3632947

	Haisor: Human-aware Indoor Scene Optimization via Deep Reinforcement Learning
	Sun, Jia-Mu 1; Yang, Jie 1; Mo, Kaichun 2; Lai, Yu-Kun 3; Guibas, Leonidas 2; Gao, Lin 1
	2024-04-01
发表期刊	ACM TRANSACTIONS ON GRAPHICS
ISSN	0730-0301
卷号	43 期号:2 页码:17
摘要	3D scene synthesis facilitates and benefits many real-world applications. Most scene generators focus on making indoor scenes plausible via learning from training data and leveraging extra constraints such as adjacency and symmetry. Although the generated 3D scenes are mostly plausible with visually realistic layouts, they can be functionally unsuitable for human users to navigate and interact with furniture. Our key observation is that human activity plays a critical role and sufficient free space is essential for human-scene interactions. This is exactly where many existing synthesized scenes fail-the seemingly correct layouts are often not fit for living. To tackle this, we present a human-aware optimization framework Haisor for 3D indoor scene arrangement via reinforcement learning, which aims to find an action sequence to optimize the indoor scene layout automatically. Based on the hierarchical scene graph representation, an optimal action sequence is predicted and performed via Deep Q-Learning with Monte Carlo Tree Search (MCTS), where MCTS is our key feature to search for the optimal solution in long-term sequences and large action space. Multiple human-aware rewards are designed as our core criteria of human-scene interaction, aiming to identify the next smart action by leveraging powerful reinforcement learning. Our framework is optimized end-to-end by giving the indoor scenes with part-level furniture layout including part mobility information. Furthermore, our methodology is extensible and allows utilizing different reward designs to achieve personalized indoor scene synthesis. Extensive experiments demonstrate that our approach optimizes the layout of 3D indoor scenes in a human-aware manner, which is more realistic and plausible than original state-of-the-art generator results, and our approach produces superior smart actions, outperforming alternative baselines.
关键词	Scene optimization scene synthesis human aware reinforcement learning Monte Carlo search robot simulation imitation learning
DOI	10.1145/3632947
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[62322210] ; Beijing Municipal Natural Science Foundation for Distinguished Young Scholars[JQ21013] ; Beijing Municipal Science and Technology Commission[Z231100005923031] ; ARL[W911NF-21-2-0104] ; Vannevar Bush Faculty Fellowship
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:001208809900002
出版者	ASSOC COMPUTING MACHINERY
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/39003
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Gao, Lin
作者单位	1.Chinese Acad Sci, Beijing Key Lab Mobile Comp & Pervas Device, Inst Comp Technol, Beijing 100190, Peoples R China 2.Stanford Univ, Dept Comp Sci, 450 Serra Mall, Stanford, CA 94305 USA 3.Cardiff Univ, Sch Comp Sci & Informat, Cardiff CF10 3AT, Wales
推荐引用方式 GB/T 7714	Sun, Jia-Mu,Yang, Jie,Mo, Kaichun,et al. Haisor: Human-aware Indoor Scene Optimization via Deep Reinforcement Learning[J]. ACM TRANSACTIONS ON GRAPHICS,2024,43(2):17.
APA	Sun, Jia-Mu,Yang, Jie,Mo, Kaichun,Lai, Yu-Kun,Guibas, Leonidas,&Gao, Lin.(2024).Haisor: Human-aware Indoor Scene Optimization via Deep Reinforcement Learning.ACM TRANSACTIONS ON GRAPHICS,43(2),17.
MLA	Sun, Jia-Mu,et al."Haisor: Human-aware Indoor Scene Optimization via Deep Reinforcement Learning".ACM TRANSACTIONS ON GRAPHICS 43.2(2024):17.