Attention-Enabled Multi-layer Subword Joint Learning for Chinese Word Embedding

doi:10.1007/s12559-025-10431-3

	Attention-Enabled Multi-layer Subword Joint Learning for Chinese Word Embedding
	Xue, Pengpeng 1; Xiong, Jing 2; Tan, Liang 1,3; Liu, Zhongzhu 4; Liu, Kanglong 5
	2025-04-01
发表期刊	COGNITIVE COMPUTATION
ISSN	1866-9956
卷号	17 期号:2 页码:16
摘要	In recent years, Chinese word embeddings have attracted significant attention in the field of natural language processing (NLP). The complex structures and diverse influences of Chinese characters present distinct challenges for semantic representation. As a result, Chinese word embeddings are primarily investigated in conjunction with characters and their subcomponents. Previous research has demonstrated that word vectors frequently fail to capture the subtle semantics embedded within the complex structure of Chinese characters. Furthermore, they often neglect the varying contributions of subword information to semantics at different levels. To tackle these challenges, we present a weight-based word vector model that takes into account the internal structure of Chinese words at various levels. The model further categorizes the internal structure of Chinese words into six layers of subword information: words, characters, components, pinyin, strokes, and structures. The semantics of Chinese words can be derived by integrating the subword information from various layers. Moreover, the model considers the varying contributions of each subword layer to the semantics of Chinese words. It utilizes an attention mechanism to determine the weights between and within the subword layers, facilitating the comprehensive extraction of word semantics. The word-level subwords act as the attention mechanism query for subwords in other layers to learn semantic bias. Experimental results show that the proposed word vector model achieves enhancements in various evaluation metrics, such as word similarity, word analogy, text categorization, and case studies.
关键词	Chinese word embedding Semantic analysis Attention mechanism Feature substring Morphological information Pronunciation information
DOI	10.1007/s12559-025-10431-3
收录类别	SCI
语种	英语
资助项目	Sichuan Provincial Science and Technology Department Project[2022YFG0161] ; Sichuan Provincial Science and Technology Department Project[2023YFG0295] ; National Natural Science Foundation of China[61373126]
WOS研究方向	Computer Science ; Neurosciences & Neurology
WOS类目	Computer Science, Artificial Intelligence ; Neurosciences
WOS记录号	WOS:001435384700001
出版者	SPRINGER
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/40709
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Tan, Liang
作者单位	1.Sichuan Normal Univ, Sch Comp Sci, Chengdu 610101, Sichuan, Peoples R China 2.Chongqing Coll Mobile Commun, Chongqing Key Lab Publ Big Data Secur Technol, Chongqing 401420, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 4.Huizhou Univ, Sch Math & Stat, Huizhou 516007, Guangdong, Peoples R China 5.Hong Kong Polytech Univ, Dept Chinese & Bilingual Studies, Hong Kong 999077, Peoples R China
推荐引用方式 GB/T 7714	Xue, Pengpeng,Xiong, Jing,Tan, Liang,et al. Attention-Enabled Multi-layer Subword Joint Learning for Chinese Word Embedding[J]. COGNITIVE COMPUTATION,2025,17(2):16.
APA	Xue, Pengpeng,Xiong, Jing,Tan, Liang,Liu, Zhongzhu,&Liu, Kanglong.(2025).Attention-Enabled Multi-layer Subword Joint Learning for Chinese Word Embedding.COGNITIVE COMPUTATION,17(2),16.
MLA	Xue, Pengpeng,et al."Attention-Enabled Multi-layer Subword Joint Learning for Chinese Word Embedding".COGNITIVE COMPUTATION 17.2(2025):16.