PMHR: Path-Based Multi-Hop Reasoning Incorporating Rule-Enhanced Reinforcement Learning and KG Embeddings

doi:10.3390/electronics13234847

	PMHR: Path-Based Multi-Hop Reasoning Incorporating Rule-Enhanced Reinforcement Learning and KG Embeddings
	Ma, Ang 1; Yu, Yanhua 1; Shi, Chuan 1; Zhen, Shuai 1; Pang, Liang 2; Chua, Tat-Seng 3
	2024-12-01
发表期刊	ELECTRONICS
ISSN	2079-9292
卷号	13 期号:23 页码:18
摘要	Multi-hop reasoning provides a means for inferring indirect relationships and missing information from knowledge graphs (KGs). Reinforcement learning (RL) was recently employed for multi-hop reasoning. Although RL-based methods provide explainability, they face challenges such as sparse rewards, spurious paths, large action spaces, and long training and running times. In this study, we present a novel approach that combines KG embeddings and RL strategies for multi-hop reasoning called path-based multi-hop reasoning (PMHR). We address the issues of sparse rewards and spurious paths by incorporating a well-designed reward function that combines soft rewards with rule-based rewards. The rewards are adjusted based on the target entity and the path to it. Furthermore, we perform action filtering and utilize the vectors of entities and relations acquired through KG embeddings to initialize the environment, thereby significantly reducing the runtime. Experiments involving a comprehensive performance evaluation, efficiency analysis, ablation studies, and a case study were performed. The experimental results on benchmark datasets demonstrate the effectiveness of PMHR in improving KG reasoning accuracy while preserving interpretability. Compared to existing state-of-the-art models, PMHR achieved Hit@1 improvements of 0.63%, 2.02%, and 3.17% on the UMLS, Kinship, and NELL-995 datasets, respectively. PMHR provides not only improved reasoning accuracy and explainability but also optimized computational efficiency, thereby offering a robust solution for multi-hop reasoning.
关键词	knowledge graphs knowledge graph reasoning reinforcement learning multi-hop reasoning
DOI	10.3390/electronics13234847
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China ; National Key Research and Development Program[2020YFB2104503] ; [U22B2019]
WOS研究方向	Computer Science ; Engineering ; Physics
WOS类目	Computer Science, Information Systems ; Engineering, Electrical & Electronic ; Physics, Applied
WOS记录号	WOS:001377725500001
出版者	MDPI
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41073
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Yu, Yanhua
作者单位	1.Beijing Univ Posts & Telecommun, Coll Comp Sci, Beijing 100876, Peoples R China 2.Univ Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 3.Natl Univ Singapore, Sea NExT Joint Lab, Singapore 119077, Singapore
推荐引用方式 GB/T 7714	Ma, Ang,Yu, Yanhua,Shi, Chuan,et al. PMHR: Path-Based Multi-Hop Reasoning Incorporating Rule-Enhanced Reinforcement Learning and KG Embeddings[J]. ELECTRONICS,2024,13(23):18.
APA	Ma, Ang,Yu, Yanhua,Shi, Chuan,Zhen, Shuai,Pang, Liang,&Chua, Tat-Seng.(2024).PMHR: Path-Based Multi-Hop Reasoning Incorporating Rule-Enhanced Reinforcement Learning and KG Embeddings.ELECTRONICS,13(23),18.
MLA	Ma, Ang,et al."PMHR: Path-Based Multi-Hop Reasoning Incorporating Rule-Enhanced Reinforcement Learning and KG Embeddings".ELECTRONICS 13.23(2024):18.