Multimodal Food Learning

doi:10.1145/3715143

	Multimodal Food Learning
	Min, Weiqing 1; Hong, Xingjian 1; Liu, Yuxin 1; Huang, Mingyu 1; Jin, Ying 1; Zhou, Pengfei 1; Xu, Leyi 1; Wang, Yilin 1; Jiang, Shuqiang 1; Rui, Yong 2
	2025-07-01
发表期刊	ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS
ISSN	1551-6857
卷号	21 期号:7 页码:28
摘要	Food-centered study has received more attention in the multimedia community for its profound impact on our survival, nutrition and health, pleasure, and enjoyment. Our experience of food is typically multisensory: We see food objects, smell its odors, taste its flavors, feel its texture, and hear sounds when chewing. Therefore, multimodal food learning is vital in food-centered study, which aims to relate information from multiple food modalities to support various multimedia tasks, ranging from recognition, retrieval, generation, recommendation, and interaction, enabling applications in different fields like healthcare and agriculture. However, there is no surveys on this topic to our knowledge. To fill this gap, this article formalizes multimodal food learning and comprehensively surveys its typical tasks, technical achievements, existing datasets, and applications to provide the blueprint with researchers and practitioners. Based on the current state of the art, we identify both open research issues and promising research directions, such as multimodal food learning benchmark construction, multimodal food foundation model construction, and multimodality diet estimation. We also point out that closer cooperation from researchers between multimedia and food science can handle some existing challenges and meanwhile open up more new opportunities to advance the fast development of multimodal food learning. This is the first comprehensive survey in this topic and we anticipate about 170 reviewed research articles can benefit academia and industry in this community and beyond.
关键词	Multimodal Food Learning Cross-modal Retrieval Cross-modal Generation Food Recognition
DOI	10.1145/3715143
收录类别	SCI
语种	英语
资助项目	Beijing Natural Science Foundation[JQ24021] ; National Natural Science Foundation of China[62125207] ; National Natural Science Foundation of China[62472411]
WOS研究方向	Computer Science
WOS类目	Computer Science, Information Systems ; Computer Science, Software Engineering ; Computer Science, Theory & Methods
WOS记录号	WOS:001540879800003
出版者	ASSOC COMPUTING MACHINERY
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/42005
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Jiang, Shuqiang
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 2.Lenovo Grp Ltd, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Min, Weiqing,Hong, Xingjian,Liu, Yuxin,et al. Multimodal Food Learning[J]. ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,2025,21(7):28.
APA	Min, Weiqing.,Hong, Xingjian.,Liu, Yuxin.,Huang, Mingyu.,Jin, Ying.,...&Rui, Yong.(2025).Multimodal Food Learning.ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,21(7),28.
MLA	Min, Weiqing,et al."Multimodal Food Learning".ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS 21.7(2025):28.