DMutDE: Dual-View Mutual Distillation Framework for Knowledge Graph Embeddings

doi:10.1109/TNNLS.2025.3608503

	DMutDE: Dual-View Mutual Distillation Framework for Knowledge Graph Embeddings
	Liu, Ruizhou 1,2; Wu, Zhe 2; Wu, Yiling 2; Cao, Zongsheng 3; Xu, Qianqian 4; Huang, Qingming 5
	2025-10-07
发表期刊	IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
ISSN	2162-237X
页码	14
摘要	Knowledge graphs (KGs) have caught more and more attention in recent years. Currently, in some practical scenarios, KG embedding (KGE) models are expected to reduce their spatial complexity without losing much performance to address the challenges of storage limitations and knowledge reasoning efficiency. To achieve this, existing works use one or more large and high-performance teacher models to improve the performance of a lightweight student model via knowledge distillation (KD), thus meeting the requirements of some practical complicated applications. However, in resource-constrained scenarios, obtaining high-performance teacher models is challenging due to high training costs and significant storage requirements. Thus, enhancing the student model's performance without large teacher models is crucial. To address this issue, we propose Dual-View Mutual Distillation Framework for Knowledge Graph Embeddings (DMutDE), a distillation framework leveraging mutual learning for peer-to-peer distillation between two KGE models with different architectures. In KGE models, we notice that the way of modeling relational directed edges determines the model view of KGE model for learning KG data. Thus, integrating the model views from two different KGE models by KD into a student KGE model can improve its generalization, so as to increase its performance. To identify an effective dual-view fusion method, we design two modules in the DMutDE framework. Specifically, we design a novel soft-label fusion (SLF) module for noise filtering and response knowledge transfer. Then, we propose an entity embedding distillation (EED) module to distill structural features from each other. Finally, we conduct several comprehensive experiments on the standard open-source benchmarks to demonstrate that our framework achieves the state-of-the-art results. The code is available at https://github.com/RuizhouLiu/DMutDE
关键词	Training Data models Predictive models Cognition Knowledge graphs Electronic mail Translation Semantics Noise Costs Knowledge distillation (KD) knowledge graph (KG) knowledge graph embedding (KGE)
DOI	10.1109/TNNLS.2025.3608503
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[62441232] ; National Natural Science Foundation of China[62236008] ; National Natural Science Foundation of China[62472238] ; National Natural Science Foundation of China[U21B203]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:001591763300001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41681
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wu, Zhe; Huang, Qingming
作者单位	1.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 100190, Peoples R China 2.Peng Cheng Lab, Shenzhen 518055, Peoples R China 3.Chinese Acad Sci, Inst Informat Engn, Beijing 100190, Peoples R China 4.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 5.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China
推荐引用方式 GB/T 7714	Liu, Ruizhou,Wu, Zhe,Wu, Yiling,et al. DMutDE: Dual-View Mutual Distillation Framework for Knowledge Graph Embeddings[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2025:14.
APA	Liu, Ruizhou,Wu, Zhe,Wu, Yiling,Cao, Zongsheng,Xu, Qianqian,&Huang, Qingming.(2025).DMutDE: Dual-View Mutual Distillation Framework for Knowledge Graph Embeddings.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,14.
MLA	Liu, Ruizhou,et al."DMutDE: Dual-View Mutual Distillation Framework for Knowledge Graph Embeddings".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2025):14.