Toward Generalized Multistage Clustering: Multiview Self-Distillation

doi:10.1109/TNNLS.2024.3479280

	Toward Generalized Multistage Clustering: Multiview Self-Distillation
	Wang, Jiatai 1,2; Xu, Zhiwei 2,3; Wang, Xin 4; Li, Tao 1,2
	2024-11-11
发表期刊	IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
ISSN	2162-237X
页码	14
摘要	Existing multistage clustering methods independently learn the salient features from multiple views and then perform the clustering task. Particularly, multiview clustering (MVC) has attracted a lot of attention in multiview or multimodal scenarios. MVC aims at exploring common semantics and pseudo-labels from multiple views and clustering in a self-supervised manner. However, limited by noisy data and inadequate feature learning, such a clustering paradigm generates overconfident pseudo-labels that misguide the model to produce inaccurate predictions. Therefore, it is desirable to have a method that can correct this pseudo-label mistraction in multistage clustering to avoid bias accumulation. To alleviate the effect of overconfident pseudo-labels and improve the generalization ability of the model, this article proposes a novel multistage deep MVC framework where multiview self-distillation (DistilMVC) is introduced to distill dark knowledge of label distribution. Specifically, in the feature subspace at different hierarchies, we explore the common semantics of multiple views through contrastive learning and obtain pseudo-labels by maximizing the mutual information between views. Additionally, a teacher network is responsible for distilling pseudo-labels into dark knowledge, supervising the student network and improving its predictive capabilities to enhance its robustness. Extensive experiments on real-world multiview datasets show that our method has better clustering performance than the state-of-the-art (SOTA) methods.
关键词	Semantics Feature extraction Contrastive learning Mutual information Clustering methods Representation learning Computational modeling Training Knowledge engineering Predictive models Hierarchical contrastive learning multistage clustering multiview self-distillation mutual information between views
DOI	10.1109/TNNLS.2024.3479280
收录类别	SCI
语种	英语
资助项目	National Science Foundation of China[61962045] ; National Science Foundation of China[62272248] ; Program for Young Talents of Science and Technology in Universities of Inner Mongolia Autonomous Region[NJYT23104] ; Basic Scientific Research Expenses Program of Universities[JY20220273] ; Basic Scientific Research Expenses Program of Universities[JY20240002]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:001358229700001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41179
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Xu, Zhiwei; Li, Tao
作者单位	1.Nankai Univ, Coll Comp Sci, Tianjin 300350, Peoples R China 2.Haihe Lab Informat Technol Applicat Innovat ITAI, Tianjin 300459, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 4.SUNY Stony Brook, Dept Elect & Comp Engn, Stony Brook, NY 11794 USA
推荐引用方式 GB/T 7714	Wang, Jiatai,Xu, Zhiwei,Wang, Xin,et al. Toward Generalized Multistage Clustering: Multiview Self-Distillation[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2024:14.
APA	Wang, Jiatai,Xu, Zhiwei,Wang, Xin,&Li, Tao.(2024).Toward Generalized Multistage Clustering: Multiview Self-Distillation.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,14.
MLA	Wang, Jiatai,et al."Toward Generalized Multistage Clustering: Multiview Self-Distillation".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2024):14.