Learning Effective RGB-D Representations for Scene Recognition

doi:10.1109/TIP.2018.2872629

	Learning Effective RGB-D Representations for Scene Recognition
	Song, Xinhang 1,2; Jiang, Shuqiang 1,2; Herranz, Luis 3; Chen, Chengpeng 1,2
	2019-02-01
发表期刊	IEEE TRANSACTIONS ON IMAGE PROCESSING
ISSN	1057-7149
卷号	28 期号:2 页码:980-993
摘要	Deep convolutional networks can achieve impressive results on RGB scene recognition thanks to large data sets such as places. In contrast, RGB-D scene recognition is still underdeveloped in comparison, due to two limitations of RGB-D data we address in this paper. The first limitation is the lack of depth data for training deep learning models. Rather than fine tuning or transferring RGB-specific features, we address this limitation by proposing an architecture and a two-step training approach that directly learns effective depth-specific features using weak supervision via patches. The resulting RGB-D model also benefits from more complementary multimodal features. Another limitation is the short range of depth sensors (typically 0.5 m to 5.5 m), resulting in depth images not capturing distant objects in the scenes that RGB images can. We show that this limitation can he addressed by using RGB-D videos, where more comprehensive depth information is accumulated as the camera travels across the scenes. Focusing on this scenario, we introduce the ISIA RGB-D video data set to evaluate RGB-D scene recognition with videos. Our video recognition architecture combines convolutional and recurrent neural networks that are trained in three steps with increasingly complex data to learn effective features (i.e., patches, frames, and sequences). Our approach obtains the state-of-the-art performances on RGB-D image (NYUD2 and SUN RGB-D) and video (ISIA RGB-D) scene recognition.
关键词	Scene recognition deep learning multimodal RGB-D video CNN RNN
DOI	10.1109/TIP.2018.2872629
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[61532018] ; Lenovo Outstanding Young Scientists Program ; National Program for Special Support of Eminent Professionals ; National Program for Support of Top-notch Young Professionals ; National Postdoctoral Program for Innovative Talents[BX201700255] ; China Postdoctoral Science Foundation[2018M631583] ; European Union Research and Innovation Program under the Marie Sklodowska-Curie Grant[6655919]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:000448501800011
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：34[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/3639
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Jiang, Shuqiang
作者单位	1.Chinese Acad Sci, Key Lab Intelligent Informat Proc, Inst Comp Technol, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Univ Autonoma Barcelona, Comp Vis Ctr, E-08193 Barcelona, Spain
推荐引用方式 GB/T 7714	Song, Xinhang,Jiang, Shuqiang,Herranz, Luis,et al. Learning Effective RGB-D Representations for Scene Recognition[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2019,28(2):980-993.
APA	Song, Xinhang,Jiang, Shuqiang,Herranz, Luis,&Chen, Chengpeng.(2019).Learning Effective RGB-D Representations for Scene Recognition.IEEE TRANSACTIONS ON IMAGE PROCESSING,28(2),980-993.
MLA	Song, Xinhang,et al."Learning Effective RGB-D Representations for Scene Recognition".IEEE TRANSACTIONS ON IMAGE PROCESSING 28.2(2019):980-993.