HIRE: Distilling high-order relational knowledge from heterogeneous graph neural networks

doi:10.1016/j.neucom.2022.08.022

	HIRE: Distilling high-order relational knowledge from heterogeneous graph neural networks
	Liu, Jing 1,2; Zheng, Tongya 3; Hao, Qinfen 1
	2022-10-01
发表期刊	NEUROCOMPUTING
ISSN	0925-2312
卷号	507 页码:67-83
摘要	Researchers have recently proposed plenty of heterogeneous graph neural networks (HGNNs) due to the ubiquity of heterogeneous graphs in both academic and industrial areas. Instead of pursuing a more powerful HGNN model, in this paper, we are interested in devising a versatile plug-and-play module, which accounts for distilling relational knowledge from pre-trained HGNNs. To the best of our knowledge, we are the first to propose a HIgh-order RElational (HIRE) knowledge distillation framework on heterogeneous graphs, which can significantly boost the prediction performance regardless of model architectures of HGNNs. Concretely, our HIRE framework initially performs first-order node-level knowledge distillation, which encodes the semantics of the teacher HGNN with its prediction logits. Meanwhile, the second-order relation-level knowledge distillation imitates the relational correlation between node embeddings of different types generated by the teacher HGNN. Extensive experiments on various popular HGNNs models and three real-world heterogeneous graphs demonstrate that our method obtains consistent and considerable performance enhancement, proving its effectiveness and generalization ability. (c) 2022 Elsevier B.V. All rights reserved.
关键词	Graph embedding Heterogeneous graph Graph neural networks Knowledge distillation
DOI	10.1016/j.neucom.2022.08.022
收录类别	SCI
语种	英语
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:000911757100006
出版者	ELSEVIER
引用统计	被引频次：3[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/20014
专题	中国科学院计算技术研究所期刊论文
通讯作者	Hao, Qinfen
作者单位	1.Chinese Acad Sci, Inst Comp Technol, SKLP, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.Zhejiang Univ, Coll Comp Sci & Technol, Hangzhou, Peoples R China
推荐引用方式 GB/T 7714	Liu, Jing,Zheng, Tongya,Hao, Qinfen. HIRE: Distilling high-order relational knowledge from heterogeneous graph neural networks[J]. NEUROCOMPUTING,2022,507:67-83.
APA	Liu, Jing,Zheng, Tongya,&Hao, Qinfen.(2022).HIRE: Distilling high-order relational knowledge from heterogeneous graph neural networks.NEUROCOMPUTING,507,67-83.
MLA	Liu, Jing,et al."HIRE: Distilling high-order relational knowledge from heterogeneous graph neural networks".NEUROCOMPUTING 507(2022):67-83.