Ingredient Prediction via Context Learning Network With Class-Adaptive Asymmetric Loss

doi:10.1109/TIP.2023.3318958

	Ingredient Prediction via Context Learning Network With Class-Adaptive Asymmetric Loss
	Luo, Mengjiang 1,2; Min, Weiqing 2,3,4; Wang, Zhiling 1; Song, Jiajun 1,2,4; Jiang, Shuqiang 3
	2023
发表期刊	IEEE TRANSACTIONS ON IMAGE PROCESSING
ISSN	1057-7149
卷号	32 页码:5509-5523
摘要	Ingredient prediction has received more and more attention with the help of image processing for its diverse real-world applications, such as nutrition intake management and cafeteria self-checkout system. Existing approaches mainly focus on multi-task food category-ingredient joint learning to improve final recognition by introducing task relevance, while seldom pay attention to making good use of inherent characteristics of ingredients independently. Actually, there are two issues for ingredient prediction. First, compared with fine-grained food recognition, ingredient prediction needs to extract more comprehensive features of the same ingredient and more detailed features of various ingredients from different regions of the food image. Because it can help understand various food compositions and distinguish the differences within ingredient features. Second, the ingredient distributions are extremely unbalanced. Existing loss functions can not simultaneously solve the imbalance between positive-negative samples belonging to each ingredient and significant differences among all classes. To solve these problems, we propose a novel framework named Class-Adaptive Context Learning Network (CACLNet) for ingredient prediction. In order to extract more comprehensive and detailed features, we introduce Ingredient Context Learning (ICL) to reduce the negative impact of complex background in food images and construct internal spatial connections among ingredient regions of food objects in a self-supervised manner, which can strengthen the contacts of the same ingredients through region interactions. In order to solve the imbalance of different classes among ingredients, we propose one novel Class-Adaptive Asymmetric Loss (CAAL) to focus on various ingredient classes adaptively. Besides, considering that the over-suppression of negative samples will over-fit positive samples of those rare ingredients, CAAL alleviates this continuous suppression according to the imbalanced ratios based on gradients while maintaining the contribution of positive samples by lesser suppression. Extensive evaluation on two popular benchmark datasets (Vireo Food-172, UEC Food-100) demonstrates our proposed method achieves the state-of-the-art performance. Further qualitative analysis and visualization show the effectiveness of our method.
关键词	Ingredient prediction multi-label classification deep learning
DOI	10.1109/TIP.2023.3318958
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[61972378] ; National Natural Science Foundation of China[62125207] ; National Natural Science Foundation of China[U1936203] ; National Natural Science Foundation of China[U19B2040] ; CAAI-Huawei MindSpore Open Fund
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001082264400007
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/21110
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Min, Weiqing
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 4.Chinese Acad Sci, Inst Intelligent Comp Technol, Suzhou 215002, Peoples R China
推荐引用方式 GB/T 7714	Luo, Mengjiang,Min, Weiqing,Wang, Zhiling,et al. Ingredient Prediction via Context Learning Network With Class-Adaptive Asymmetric Loss[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2023,32:5509-5523.
APA	Luo, Mengjiang,Min, Weiqing,Wang, Zhiling,Song, Jiajun,&Jiang, Shuqiang.(2023).Ingredient Prediction via Context Learning Network With Class-Adaptive Asymmetric Loss.IEEE TRANSACTIONS ON IMAGE PROCESSING,32,5509-5523.
MLA	Luo, Mengjiang,et al."Ingredient Prediction via Context Learning Network With Class-Adaptive Asymmetric Loss".IEEE TRANSACTIONS ON IMAGE PROCESSING 32(2023):5509-5523.