UniFa: A unified feature hallucination framework for any-shot object detection

doi:10.1016/j.patrec.2025.01.015

	UniFa: A unified feature hallucination framework for any-shot object detection
	Nie, Hui ; Wang, Ruiping 1; Chen, Xilin
	2025-03-01
发表期刊	PATTERN RECOGNITION LETTERS
ISSN	0167-8655
卷号	189 页码:207-213
摘要	Any-shot object detection seeks to simultaneously detect base (many-shot), few-shot and zero-shot categories. The primary challenge lies in insufficient visual data for rare (few-shot and zero-shot) categories, hindering effective training. Existing methods use visual feature generation to alleviate it, but the quality of the generated features is low and limited to zero-shot object detection task (i.e., only including zero-shot categories). This mainly arises from semantic information for feature generation trained on unimodal data lacking visual- awareness, and the significant distinctness of generated features across categories. To tackle these issues, we introduce the Unified Feature Hallucination (UniFa) framework, which generates high-quality features for two rare categories. Utilizing CLIP's text encoder, we transform category names into visual-aware semantic information for generating visual features, facilitating better visual-semantic alignment. A semantically blended feature enhancer is utilized to merge features from any two categories, producing denser and more realistic features. The effectiveness of our approach is confirmed through extensive experiments on MSCOCO datasets.
关键词	Any-shot object detection Feature hallucination Visual-aware semantic information
DOI	10.1016/j.patrec.2025.01.015
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2021ZD0111901] ; National Key R&D Program of China[2023YFF1105104] ; Natural Science Foundation of China[U21B2025]
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:001424575900001
出版者	ELSEVIER
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/40737
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Ruiping
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Nie, Hui,Wang, Ruiping,Chen, Xilin. UniFa: A unified feature hallucination framework for any-shot object detection[J]. PATTERN RECOGNITION LETTERS,2025,189:207-213.
APA	Nie, Hui,Wang, Ruiping,&Chen, Xilin.(2025).UniFa: A unified feature hallucination framework for any-shot object detection.PATTERN RECOGNITION LETTERS,189,207-213.
MLA	Nie, Hui,et al."UniFa: A unified feature hallucination framework for any-shot object detection".PATTERN RECOGNITION LETTERS 189(2025):207-213.