Multi-Task Deep Relative Attribute Learning for Visual Urban Perception

doi:10.1109/TIP.2019.2932502

	Multi-Task Deep Relative Attribute Learning for Visual Urban Perception
	Min, Weiqing 1,2; Mei, Shuhuan 1,3; Liu, Linhu 1,2; Wang, Yi 1; Jiang, Shuqiang 1,2
	2020
发表期刊	IEEE TRANSACTIONS ON IMAGE PROCESSING
ISSN	1057-7149
卷号	29 页码:657-669
摘要	Visual urban perception aims to quantify perceptual attributes (e.g., safe and depressing attributes) of physical urban environment from crowd-sourced street-view images and their pairwise comparisons. It has been receiving more and more attention in computer vision for various applications, such as perceptive attribute learning and urban scene understanding. Most existing methods adopt either 1) a regression model trained using image features and ranked scores converted from pairwise comparisons for perceptual attribute prediction or 2) a pairwise ranking algorithm to independently learn each perceptual attribute. However, the former fails to directly exploit pairwise comparisons while the latter ignores the relationship among different attributes. To address them, we propose a multi-task deep relative attribute learning network (MTDRALN) to learn all the relative attributes simultaneously via multi-task Siamese networks, where each Siamese network will predict one relative attribute. Combined with deep relative attribute learning, we utilize the structured sparsity to exploit the prior from natural attribute grouping, where all the attributes are divided into different groups based on semantic relatedness in advance. As a result, MTDRALN is capable of learning all the perceptual attributes simultaneously via multi-task learning. Besides the ranking sub-network, MTDRALN further introduces the classification sub-network, and these two types of losses from two sub-networks jointly constrain parameters of the deep network to make the network learn more discriminative visual features for relative attribute learning. In addition, our network can be trained in an end-to-end way to make deep feature learning and multi-task relative attribute learning reinforces each other. Extensive experiments on the large-scale Place Pulse 2.0 dataset validate the advantage of our proposed network. Our qualitative results along with visualization of saliency maps also show that the proposed network is able to learn effective features for perceptual attributes.
关键词	Visualization Task analysis Deep learning Urban areas Correlation Computer vision Predictive models Visual urban perception relative attribute multi-task learning
DOI	10.1109/TIP.2019.2932502
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[61972378] ; National Natural Science Foundation of China[61532018] ; National Natural Science Foundation of China[61602437] ; Beijing Natural Science Foundation[L182054] ; National Program for Special Support of Eminent Professionals ; National Program for Support of Top-notch Young Professionals ; State Key Laboratory of Robotics
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:000497434700030
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：31[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/14937
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Jiang, Shuqiang
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Shandong Univ Sci & Technol, Coll Math & Syst Sci, Qingdao 266590, Shandong, Peoples R China
推荐引用方式 GB/T 7714	Min, Weiqing,Mei, Shuhuan,Liu, Linhu,et al. Multi-Task Deep Relative Attribute Learning for Visual Urban Perception[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2020,29:657-669.
APA	Min, Weiqing,Mei, Shuhuan,Liu, Linhu,Wang, Yi,&Jiang, Shuqiang.(2020).Multi-Task Deep Relative Attribute Learning for Visual Urban Perception.IEEE TRANSACTIONS ON IMAGE PROCESSING,29,657-669.
MLA	Min, Weiqing,et al."Multi-Task Deep Relative Attribute Learning for Visual Urban Perception".IEEE TRANSACTIONS ON IMAGE PROCESSING 29(2020):657-669.