Channel-Aware Decoupling Network for Multiturn Dialog Comprehension

doi:10.1109/TNNLS.2022.3220047

	Channel-Aware Decoupling Network for Multiturn Dialog Comprehension
	Zhang, Zhuosheng 1,2; Zhao, Hai 1,2; Liu, Longxiang 3,4
	2022-11-14
发表期刊	IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
ISSN	2162-237X
页码	12
摘要	Training machines to understand natural language and interact with humans is one of the major goals of artificial intelligence. Recent years have witnessed an evolution from matching networks to pretrained language models (PrLMs). In contrast to the plain-text modeling as the focus of the PrLMs, dialog texts involve multiple speakers and reflect special characteristics, such as topic transitions and structure dependencies, between distant utterances. However, the related PrLM models commonly represent dialogs sequentially by processing the pairwise dialog history as a whole. Thus, the hierarchical information on either utterance interrelation or speaker roles coupled in such representations is not well addressed. In this work, we propose compositional learning for holistic interaction across the utterances beyond the sequential contextualization from PrLMs, in order to capture the utterance-aware and speaker-aware representations entailed in a dialog history. We decouple the contextualized word representations by masking mechanisms in transformer-based PrLM, making each word only focus on the words in the current utterance, other utterances, and two speaker roles (i.e., utterances of the sender and utterances of the receiver), respectively. In addition, we employ domain-adaptive training strategies to help the model adapt to the dialog domains. Experimental results show that our method substantially boosts the strong PrLM baselines in four public benchmark datasets, achieving new state-of-the-art performance over previous methods.
关键词	Deep neural networks dialog modeling natural language generation open domain conversation system
DOI	10.1109/TNNLS.2022.3220047
收录类别	SCI
语种	英语
资助项目	Key Projects of National Natural Science Foundation of China[U1836222] ; Key Projects of National Natural Science Foundation of China[61733011]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:000886836300001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/19873
专题	中国科学院计算技术研究所期刊论文
通讯作者	Zhao, Hai
作者单位	1.Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Key Lab Shanghai Educ Commiss Intelligent Interact, Shanghai 200240, Peoples R China 2.Shanghai Jiao Tong Univ, MoE Key Lab Artificial Intelligence, Shanghai 200240, Peoples R China 3.Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China 4.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100045, Peoples R China
推荐引用方式 GB/T 7714	Zhang, Zhuosheng,Zhao, Hai,Liu, Longxiang. Channel-Aware Decoupling Network for Multiturn Dialog Comprehension[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2022:12.
APA	Zhang, Zhuosheng,Zhao, Hai,&Liu, Longxiang.(2022).Channel-Aware Decoupling Network for Multiturn Dialog Comprehension.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,12.
MLA	Zhang, Zhuosheng,et al."Channel-Aware Decoupling Network for Multiturn Dialog Comprehension".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2022):12.