Multi-Scale Multi-View Deep Feature Aggregation for Food Recognition

doi:10.1109/TIP.2019.2929447

	Multi-Scale Multi-View Deep Feature Aggregation for Food Recognition
	Jiang, Shuqiang 1,2; Min, Weiqing 1,2; Liu, Linhu 1,2; Luo, Zhengdong 1,2
	2020
发表期刊	IEEE TRANSACTIONS ON IMAGE PROCESSING
ISSN	1057-7149
卷号	29 页码:265-276
摘要	Recently, food recognition has received more and more attention in image processing and computer vision for its great potential applications in human health. Most of the existing methods directly extracted deep visual features via convolutional neural networks (CNNs) for food recognition. Such methods ignore the characteristics of food images and are, thus, hard to achieve optimal recognition performance. In contrast to general object recognition, food images typically do not exhibit distinctive spatial arrangement and common semantic patterns. In this paper, we propose a multi-scale multi-view feature aggregation (MSMVFA) scheme for food recognition. MSMVFA can aggregate high-level semantic features, mid-level attribute features, and deep visual features into a unified representation. These three types of features describe the food image from different granularity. Therefore, the aggregated features can capture the semantics of food images with the greatest probability. For that solution, we utilize additional ingredient knowledge to obtain mid-level attribute representation via ingredient-supervised CNNs. High-level semantic features and deep visual features are extracted from class-supervised CNNs. Considering food images do not exhibit distinctive spatial layout in many cases, MSMVFA fuses multi-scale CNN activations for each type of features to make aggregated features more discriminative and invariable to geometrical deformation. Finally, the aggregated features are more robust, comprehensive, and discriminative via two-level fusion, namely multi-scale fusion for each type of features and multi-view aggregation for different types of features. In addition, MSMVFA is general and different deep networks can be easily applied into this scheme. Extensive experiments and evaluations demonstrate that our method achieves state-of-the-art recognition performance on three popular large-scale food benchmark datasets in Top-1 recognition accuracy. Furthermore, we expect this paper will further the agenda of food recognition in the community of image processing and computer vision.
关键词	Food recognition ingredient knowledge feature aggregation convolutional neural networks
DOI	10.1109/TIP.2019.2929447
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[61532018] ; National Natural Science Foundation of China[61602437] ; Beijing Natural Science Foundation[L182054] ; National Program for Special Support of Eminent Professionals ; National Program for Support of Top-notch Young Professionals
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:000487069300020
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：101[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/4656
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Jiang, Shuqiang
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Jiang, Shuqiang,Min, Weiqing,Liu, Linhu,et al. Multi-Scale Multi-View Deep Feature Aggregation for Food Recognition[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2020,29:265-276.
APA	Jiang, Shuqiang,Min, Weiqing,Liu, Linhu,&Luo, Zhengdong.(2020).Multi-Scale Multi-View Deep Feature Aggregation for Food Recognition.IEEE TRANSACTIONS ON IMAGE PROCESSING,29,265-276.
MLA	Jiang, Shuqiang,et al."Multi-Scale Multi-View Deep Feature Aggregation for Food Recognition".IEEE TRANSACTIONS ON IMAGE PROCESSING 29(2020):265-276.