Graph-Agnostic Linear Transformers

doi:10.1016/j.neunet.2026.108595

CSpace

	Graph-Agnostic Linear Transformers
	Guo, Zhiyu 1,2; Liu, Yang 1; Ao, Xiang 1,2,3; Tang, Yateng ; Chen, Xinhuan ; Zheng, Xuehao ; He, Qing 1,2
	2026-06-01
发表期刊	NEURAL NETWORKS
ISSN	0893-6080
卷号	198 页码:12
摘要	Graph Transformers (GTs), as emerging foundational encoders for graph-structured data, have shown promising performance due to the integration of local graph structures with global attention mechanisms. However, the complex attention functions and their coupling with graph structures incur significant computational overhead, particularly in large-scale graphs. In this paper, we decouple graph structures from Transformers and propose the Graph-Agnostic Linear Transformer (GALiT). In GALiT, graph structures are solely utilized to denoise raw node features before training, as our findings reveal that these denoised features have integrated the main information of the graph structure and can replace it to guide Transformers. By excluding graph structures from the training and inference stages, GALiT serves as a graph-agnostic model which significantly reduces computational complexity. Additionally, we simplify the linear attention functions inherited from traditional Transformers, which further reduces computational overhead while still capturing the relationships between nodes. Through weighted combination, we integrate the denoised features into the attention mechanism, as our theoretical analysis reveals the key role of the synergy between linear attention and denoised features in enhancing representation diversity. Despite decoupling graph structures and simplifying attention mechanisms, our model surprisingly outperforms most GNNs and GTs on benchmark graphs. Experimental results indicate that GALiT achieves high efficiency while maintaining or even enhancing performance.
关键词	Graph neural network Graph transformer Linear attention Graph-agnostic model
DOI	10.1016/j.neunet.2026.108595
收录类别	SCI
语种	英语
WOS研究方向	Computer Science ; Neurosciences & Neurology
WOS类目	Computer Science, Artificial Intelligence ; Neurosciences
WOS记录号	WOS:001676295000001
出版者	PERGAMON-ELSEVIER SCIENCE LTD
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/42852
专题	中国科学院计算技术研究所
通讯作者	Liu, Yang
作者单位	1.Chinese Acad Sci, Inst Comp Technol, State Key Lab AI Safety, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Inst Intelligent Comp Technol, Suzhou 215100, Peoples R China
推荐引用方式 GB/T 7714	Guo, Zhiyu,Liu, Yang,Ao, Xiang,et al. Graph-Agnostic Linear Transformers[J]. NEURAL NETWORKS,2026,198:12.
APA	Guo, Zhiyu.,Liu, Yang.,Ao, Xiang.,Tang, Yateng.,Chen, Xinhuan.,...&He, Qing.(2026).Graph-Agnostic Linear Transformers.NEURAL NETWORKS,198,12.
MLA	Guo, Zhiyu,et al."Graph-Agnostic Linear Transformers".NEURAL NETWORKS 198(2026):12.