Towards generalizable Graph Contrastive Learning: An information theory perspective

doi:10.1016/j.neunet.2024.106125

CSpace

	Towards generalizable Graph Contrastive Learning: An information theory perspective
	Yuan, Yige; Xu, Bingbing; Shen, Huawei; Cao, Qi; Cen, Keting; Zheng, Wen; Cheng, Xueqi
	2024-04-01
发表期刊	NEURAL NETWORKS
ISSN	0893-6080
卷号	172 页码:17
摘要	Graph Contrastive Learning (GCL) is increasingly employed in graph representation learning with the primary aim of learning node/graph representations from a predefined pretext task that can generalize to various downstream tasks. Meanwhile, the transition from a specific pretext task to diverse and unpredictable downstream tasks poses a significant challenge for GCL's generalization ability. Most existing GCL approaches maximize mutual information between two views derived from the original graph, either randomly or heuristically. However, the generalization ability of GCL and its theoretical principles are still less studied. In this paper, we introduce a novel metric GCL-GE, to quantify the generalization gap between predefined pretext and agnostic downstream tasks. Given the inherent intractability of GCL-GE, we leverage concepts from information theory to derive a mutual information upper bound that is independent of the downstream tasks, thus enabling the metric's optimization despite the variability in downstream tasks. Based on the theoretical insight, we propose InfoAdv, a GCL framework to directly enhance generalization by jointly optimizing GCL-GE and InfoMax. Extensive experiments validate the capability of InfoAdv to enhance performance across a wide variety of downstream tasks, demonstrating its effectiveness in improving the generalizability of GCL.
关键词	Graph Contrastive Learning Generalization Information theory
DOI	10.1016/j.neunet.2024.106125
收录类别	SCI
语种	英语
资助项目	National Natural Science Foun-dation of China[U21B2046] ; National Natural Science Foun-dation of China[62202448] ; Na-tional Key R&D Program of China[2022YFB3103700] ; Na-tional Key R&D Program of China[2022YFB3103704]
WOS研究方向	Computer Science ; Neurosciences & Neurology
WOS类目	Computer Science, Artificial Intelligence ; Neurosciences
WOS记录号	WOS:001179930100001
出版者	PERGAMON-ELSEVIER SCIENCE LTD
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38769
专题	中国科学院计算技术研究所
通讯作者	Yuan, Yige; Xu, Bingbing
作者单位	Chinese Acad Sci, Data Intelligence Syst Res Ctr, Inst Comp Technol, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Yuan, Yige,Xu, Bingbing,Shen, Huawei,et al. Towards generalizable Graph Contrastive Learning: An information theory perspective[J]. NEURAL NETWORKS,2024,172:17.
APA	Yuan, Yige.,Xu, Bingbing.,Shen, Huawei.,Cao, Qi.,Cen, Keting.,...&Cheng, Xueqi.(2024).Towards generalizable Graph Contrastive Learning: An information theory perspective.NEURAL NETWORKS,172,17.
MLA	Yuan, Yige,et al."Towards generalizable Graph Contrastive Learning: An information theory perspective".NEURAL NETWORKS 172(2024):17.