Few-shot Food Recognition via Multi-view Representation Learning

doi:10.1145/3391624

	Few-shot Food Recognition via Multi-view Representation Learning
	Jiang, Shuqiang 1; Min, Weiqing 1; Lyu, Yongqiang 2,3; Liu, Linhu 1
	2020-09-01
发表期刊	ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS
ISSN	1551-6857
卷号	16 期号:3 页码:20
摘要	This article considers the problem of few-shot learning for food recognition. Automatic food recognition can support various applications, e.g., dietary assessment and food journaling. Most existing works focus on food recognition with large numbers of labelled samples, and fail to recognize food categories with few samples. To address this problem, we propose a Multi-View Few-Shot Learning (MVFSL) framework to explore additional ingredient information for few-shot food recognition. Besides category-oriented deep visual features, we introduce ingredient-supervised deep network to extract ingredient-oriented features. As general and intermediate attributes of food, ingredient-oriented features are informative and complementary to category-oriented features, and thus they play an important role in improving food recognition. Particularly in few-shot food recognition, ingredient information can bridge the gap between disjoint training categories and test categories. To take advantage of ingredient information, we fuse these two kinds of features by first combining their feature maps from their respective deep networks and then convolving combined feature maps. Such convolution is further incorporated into a multi-view relation network, which is capable of comparing pairwise images to enable fine-grained feature learning. MVFSL is trained in an end-to-end fashion for joint optimization on two types of feature learning subnetworks and relation subnetworks. Extensive experiments on different food datasets have consistently demonstrated the advantage of MVFSL in multi-view feature fusion. Furthermore, we extend another two types of networks, namely, Siamese Network and Matching Network, by introducing ingredient information for few-shot food recognition. Experimental results have also shown that introducing ingredient information into these two networks can improve the performance of few-shot food recognition.
关键词	Food recognition few-shot learning visual recognition deep learning
DOI	10.1145/3391624
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[61532018] ; National Natural Science Foundation of China[61972378] ; National Natural Science Foundation of China[U19B2040] ; Beijing Natural Science Foundation[L182054] ; National Program for Special Support of Eminent Professionals ; National Program for Support of Top-notch Young Professionals
WOS研究方向	Computer Science
WOS类目	Computer Science, Information Systems ; Computer Science, Software Engineering ; Computer Science, Theory & Methods
WOS记录号	WOS:000569375200013
出版者	ASSOC COMPUTING MACHINERY
引用统计	被引频次：26[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/15561
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Jiang, Shuqiang
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, 6 Kexueyuan South Rd, Beijing 100190, Peoples R China 2.Qingdao KingAgroot Precis Agr Technol Co Ltd, Qingdao, Peoples R China 3.Shandong Reebow Automat Equipment Co LTD, Qingdao Branch, Room 1901,Bldg 5, Qingdao, Shandong, Peoples R China
推荐引用方式 GB/T 7714	Jiang, Shuqiang,Min, Weiqing,Lyu, Yongqiang,et al. Few-shot Food Recognition via Multi-view Representation Learning[J]. ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,2020,16(3):20.
APA	Jiang, Shuqiang,Min, Weiqing,Lyu, Yongqiang,&Liu, Linhu.(2020).Few-shot Food Recognition via Multi-view Representation Learning.ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,16(3),20.
MLA	Jiang, Shuqiang,et al."Few-shot Food Recognition via Multi-view Representation Learning".ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS 16.3(2020):20.