Refined Knowledge Transfer for Language-Based Person Search

doi:10.1109/TMM.2023.3251104

	Refined Knowledge Transfer for Language-Based Person Search
	Wu, Ziqiang 1; Ma, Bingpeng 1; Chang, Hong 2,3,4; Shan, Shiguang 3,4,5
	2023
发表期刊	IEEE TRANSACTIONS ON MULTIMEDIA
ISSN	1520-9210
卷号	25 页码:9315-9329
摘要	This paper proposes a novel method, named Refined Knowledge Transfer (RKT), for language-based person search. Existing state-of-the-art methods do not deal with knowledge imbalance between image and text. In detail, textual identity knowledge is limited, but the image contains more identity knowledge. We propose Cross-Modal Knowledge Transfer (CMKT) to enhance textual identity knowledge by image to address this problem. Besides, multiple texts of one image include more identity knowledge than a single text. Thus, we propose Intra-Modal Knowledge Transfer (IMKT) to enhance textual identity knowledge by other texts. These two types of knowledge transfer will enhance the identity knowledge in text. Additionally, by considering that identity-irrelevant knowledge is transferred to text, we propose Knowledge Refiner (KR) to refine the knowledge in text. KR is capable of preserving identity knowledge and discarding identity-irrelevant knowledge. By combining CMKT, IMKT, and KR, RKT makes textual identity knowledge more salient. Extensive experiments show the state-of-the-art performance of RKT on the CUHK-PEDES and our proposed PRW-PEDES-CN datasets. In addition, the decent generalization ability of RKT is also validated on the Flickr30K, CUB, and Flowers datasets.
关键词	Language-based person search knowledge enhancement knowledge enhancement cross-modal knowledge transfer intra-modal knowledge transfer intra-modal knowledge transfer knowledge refiner
DOI	10.1109/TMM.2023.3251104
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China
WOS研究方向	Computer Science ; Telecommunications
WOS类目	Computer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications
WOS记录号	WOS:001133324200011
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：19[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38418
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Ma, Bingpeng
作者单位	1.Univ Chinese Acad Sci UCAS, Sch Sci & Technol, Beijing 100049, Peoples R China 2.Chinese Acad Sci, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 3.UCAS, Sch Comp Sci & Technol, Beijing 100049, Peoples R China 4.Peng Cheng Lab, Shenzhen 518055, Peoples R China 5.Chinese Acad Sci, Key Lab Intelligent Informat Proc, ICT, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Wu, Ziqiang,Ma, Bingpeng,Chang, Hong,et al. Refined Knowledge Transfer for Language-Based Person Search[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2023,25:9315-9329.
APA	Wu, Ziqiang,Ma, Bingpeng,Chang, Hong,&Shan, Shiguang.(2023).Refined Knowledge Transfer for Language-Based Person Search.IEEE TRANSACTIONS ON MULTIMEDIA,25,9315-9329.
MLA	Wu, Ziqiang,et al."Refined Knowledge Transfer for Language-Based Person Search".IEEE TRANSACTIONS ON MULTIMEDIA 25(2023):9315-9329.