Introspection unit in memory network: Learning to generalize inference in OOV scenarios

doi:10.1016/j.neucom.2019.07.111

	Introspection unit in memory network: Learning to generalize inference in OOV scenarios
	Yang, Qichuan 1; He, Zhiqiang 1,2,3; Zhan, Zhiqiang 2; Zhang, Yang 3; Li, Rang 3; Hu, Changjian 3
	2020-02-28
发表期刊	NEUROCOMPUTING
ISSN	0925-2312
卷号	379 页码:30-40
摘要	Inference in natural language processing (NLP) is a tough task. Although plenty of models have been proposed in recent years, they are usually restricted to infer within a limited vocabulary or handcrafted training templates. In this paper, we propose the introspection unit (IU), a new neural module which can be incorporated with memory networks to deal with inference tasks in out of vocabulary (OOV) and rare named entities (RNEs) scenarios. Specifically, when encountering a new word, IU compares its part-of-speech context with the training dataset to extract a similar sample, and then embeds the new word into a target position to construct a simulated sample. The target position is located by the result of part-of-speech tagging. Finally, using the simulated sample, IU helps memory networks to learn the context and characteristic of the new word. In experiments, we evaluate the effectiveness of IU with the memory network on four inference datasets: a name OOV dataset, a place OOV dataset, a more challenging synthetical mixture OOV dataset and a realistic dialogue dataset. The experimental results demonstrate that IU effectively generalizes the inference ability of memory networks to OOV scenarios and improves the inference accuracies significantly. Furthermore, we visualize both the introspection process and the effect of IU in word embeddings and memories. (C) 2019 Elsevier B.V. All rights reserved.
关键词	Memory network Out of vocabulary Introspection unit Language inference
DOI	10.1016/j.neucom.2019.07.111
收录类别	SCI
语种	英语
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:000507464700003
出版者	ELSEVIER
引用统计	被引频次：3[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/15003
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	He, Zhiqiang; Zhang, Yang
作者单位	1.Beihang Univ, Sch Comp Sci & Engn, Beijing, Peoples R China 2.Chinese Acad Sci, Univ Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 3.Lenovo Ltd, Lenovo Corp Res & Dev, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Yang, Qichuan,He, Zhiqiang,Zhan, Zhiqiang,et al. Introspection unit in memory network: Learning to generalize inference in OOV scenarios[J]. NEUROCOMPUTING,2020,379:30-40.
APA	Yang, Qichuan,He, Zhiqiang,Zhan, Zhiqiang,Zhang, Yang,Li, Rang,&Hu, Changjian.(2020).Introspection unit in memory network: Learning to generalize inference in OOV scenarios.NEUROCOMPUTING,379,30-40.
MLA	Yang, Qichuan,et al."Introspection unit in memory network: Learning to generalize inference in OOV scenarios".NEUROCOMPUTING 379(2020):30-40.