Adaptively sharing multi-levels of distributed representations in multi-task learning

doi:10.1016/j.ins.2022.01.035

	Adaptively sharing multi-levels of distributed representations in multi-task learning
	Wang, Tianxin 1,2; Zhuang, Fuzhen 3,4; Sun, Ying 1,2; Zhang, Xiangliang 5; Lin, Leyu 6; Xia, Feng 6; He, Lei 6; He, Qing 1,2
	2022-04-01
发表期刊	INFORMATION SCIENCES
ISSN	0020-0255
卷号	591 页码:226-234
摘要	In multi-task learning, the performance is often sensitive to the relationships between tasks. Thus it is important to study how to exploit the complex relationships across different tasks. One line of research captures the complex task relationships, by increasing the model capacity and thus requiring a large training dataset. However in many real-world applications, the amount of labeled data is limited. In this paper, we propose a light weight and specially designed architecture, which aims to model task relationships for small or middle-sized datasets. The proposed framework learns a task-specific ensemble of subnetworks in different depths, and is able to adapt the model architecture for the given data. The task-specific ensemble parameters are learned simultaneously with the weights of the network by optimizing a single loss function defined with respect to the end task. The hierarchical model structure is able to share both general and specific distributed representations to capture the inherent relationships between tasks. We validate our approach on various types of tasks, including synthetic task, article recommendation task and vision task. The results demonstrate the advantages of our model over several competitive baselines especially when the tasks are less-related.(c) 2022 Published by Elsevier Inc.
关键词	Multi-task learning Deep learning Machine learning
DOI	10.1016/j.ins.2022.01.035
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2021ZD0113602] ; National Natural Science Foundation of China[62176014] ; National Natural Science Foundation of China[U1836206] ; National Natural Science Foundation of China[U1811461]
WOS研究方向	Computer Science
WOS类目	Computer Science, Information Systems
WOS记录号	WOS:000770687400012
出版者	ELSEVIER SCIENCE INC
引用统计	被引频次：5[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/18943
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhuang, Fuzhen
作者单位	1.Chinese Acad Sci, CAS, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Beihang Univ, Inst Artificial Intelligence, Beijing 100191, Peoples R China 4.Beihang Univ, Sch Comp Sci, SKLSDE, Beijing 100191, Peoples R China 5.Univ Notre Dame, Comp Sci & Engn, Notre Dame, IN 46556 USA 6.Tencent, WeChat Grp, Shenzhen, Peoples R China
推荐引用方式 GB/T 7714	Wang, Tianxin,Zhuang, Fuzhen,Sun, Ying,et al. Adaptively sharing multi-levels of distributed representations in multi-task learning[J]. INFORMATION SCIENCES,2022,591:226-234.
APA	Wang, Tianxin.,Zhuang, Fuzhen.,Sun, Ying.,Zhang, Xiangliang.,Lin, Leyu.,...&He, Qing.(2022).Adaptively sharing multi-levels of distributed representations in multi-task learning.INFORMATION SCIENCES,591,226-234.
MLA	Wang, Tianxin,et al."Adaptively sharing multi-levels of distributed representations in multi-task learning".INFORMATION SCIENCES 591(2022):226-234.