Mixed Dish Recognition With Contextual Relation and Domain Alignment

doi:10.1109/TMM.2021.3075037

	Mixed Dish Recognition With Contextual Relation and Domain Alignment
	Deng, Lixi 1,2,3; Chen, Jingjing 4; Ngo, Chong-Wah 5; Sun, Qianru 5; Tang, Sheng 1; Zhang, Yongdong 6; Chua, Tat-Seng 7
	2022
发表期刊	IEEE TRANSACTIONS ON MULTIMEDIA
ISSN	1520-9210
卷号	24 页码:2034-2045
摘要	Mixed dish is a food category that contains different dishes mixed in one plate, and is popular in Eastern and Southeast Asia. Recognizing the individual dishes in a mixed dish image is important for health related applications, e.g. to calculate the nutrition values of the dish. However, most existing methods that focus on single dish classification are not applicable to the recognition of mixed dish images. The main challenge of mixed dish recognition comes from three aspects: a wide range of dish types, the complex dish combination with severe overlap between different dishes and the large visual variances of same dish type caused by different cooking/cutting methods applied in different canteens. In order to tackle these problems, we propose the contextual relation network that encodes the implicit and explicit contextual relations among multiple dishes from region-level features and label-level co-occurrence respectively. Besides, to address the visual variances of dish instances from different canteens, we introduce the domain adaption networks to align both local and global features, and eliminating domain gaps of dish features across different canteens. In addition, we collect a mixed dish image dataset containing 9254 mixed dish images from 6 canteens in Singapore. Extensive experiments on both our dataset and public one validate that our methods can achieve top performance for localizing and recognizing multiple dishes and solve the domain shift problem to a certain extent in mixed dish images.
关键词	Visualization Semantics Feature extraction Image recognition Training Testing Context modeling Mixed dish recognition Contextual relation Domain alignment
DOI	10.1109/TMM.2021.3075037
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2017YFB1002202] ; A*STAR under its AME YIRG[A20E6c0101] ; National Natural Science Foundation of China[61871004] ; National Natural Science Foundation of China[2020A077] ; Sea-NExT Joint Lab
WOS研究方向	Computer Science ; Telecommunications
WOS类目	Computer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications
WOS记录号	WOS:000778959200020
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：6[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/18887
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Chen, Jingjing
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.JD Com, Beijing 100049, Peoples R China 3.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 4.Fudan Univ, Shanghai Key Labortaory Intelligent Informat Proc, Sch Comp Sci, Shanghai 200433, Peoples R China 5.Singapore Management Univ, Sch Comp & Informat Syst, Singapore 188065, Singapore 6.Univ Sci & Technol China, Sch Informat Sci & Technol, Hefei 230022, Peoples R China 7.Natl Univ Singapore, Singapore 117543, Singapore
推荐引用方式 GB/T 7714	Deng, Lixi,Chen, Jingjing,Ngo, Chong-Wah,et al. Mixed Dish Recognition With Contextual Relation and Domain Alignment[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2022,24:2034-2045.
APA	Deng, Lixi.,Chen, Jingjing.,Ngo, Chong-Wah.,Sun, Qianru.,Tang, Sheng.,...&Chua, Tat-Seng.(2022).Mixed Dish Recognition With Contextual Relation and Domain Alignment.IEEE TRANSACTIONS ON MULTIMEDIA,24,2034-2045.
MLA	Deng, Lixi,et al."Mixed Dish Recognition With Contextual Relation and Domain Alignment".IEEE TRANSACTIONS ON MULTIMEDIA 24(2022):2034-2045.