Video modeling and learning on Riemannian manifold for emotion recognition in the wild

doi:10.1007/s12193-015-0204-5

	Video modeling and learning on Riemannian manifold for emotion recognition in the wild
	Liu, Mengyi; Wang, Ruiping; Li, Shaoxin; Huang, Zhiwu; Shan, Shiguang; Chen, Xilin
	2016-06-01
发表期刊	JOURNAL ON MULTIMODAL USER INTERFACES
ISSN	1783-7677
卷号	10 期号:2 页码:113-124
摘要	In this paper, we present the method for our submission to the emotion recognition in the wild challenge (EmotiW). The challenge is to automatically classify the emotions acted by human subjects in video clips under real-world environment. In our method, each video clip can be represented by three types of image set models (i.e. linear subspace, covariance matrix, and Gaussian distribution) respectively, which can all be viewed as points residing on some Riemannian manifolds. Then different Riemannian kernels are employed on these set models correspondingly for similarity/ distance measurement. For classification, three types of classifiers, i.e. kernel SVM, logistic regression, and partial least squares, are investigated for comparisons. Finally, an optimal fusion of classifiers learned from different kernels and different modalities (video and audio) is conducted at the decision level for further boosting the performance. We perform extensive evaluations on the EmotiW 2014 challenge data (including validation set and blind test set), and evaluate the effects of different components in our pipeline. It is observed that our method has achieved the best performance reported so far. To further evaluate the generalization ability, we also perform experiments on the EmotiW 2013 data and two well-known lab-controlled databases: CK+ and MMI. The results show that the proposed framework significantly outperforms the state-of-the-art methods.
关键词	Emotion recognition Video modeling Riemannian manifold EmotiW challenge
DOI	10.1007/s12193-015-0204-5
收录类别	SCI
语种	英语
资助项目	973 Program[2015CB351802] ; Natural Science Foundation of China[61390511] ; Natural Science Foundation of China[61222211] ; Natural Science Foundation of China[61379083] ; Youth Innovation Promotion Association CAS[2015085]
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Cybernetics
WOS记录号	WOS:000378580400003
出版者	SPRINGER
引用统计	被引频次：5[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/8307
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Shan, Shiguang
作者单位	Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Proc, 6 Kexueyuan South Rd, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Liu, Mengyi,Wang, Ruiping,Li, Shaoxin,et al. Video modeling and learning on Riemannian manifold for emotion recognition in the wild[J]. JOURNAL ON MULTIMODAL USER INTERFACES,2016,10(2):113-124.
APA	Liu, Mengyi,Wang, Ruiping,Li, Shaoxin,Huang, Zhiwu,Shan, Shiguang,&Chen, Xilin.(2016).Video modeling and learning on Riemannian manifold for emotion recognition in the wild.JOURNAL ON MULTIMODAL USER INTERFACES,10(2),113-124.
MLA	Liu, Mengyi,et al."Video modeling and learning on Riemannian manifold for emotion recognition in the wild".JOURNAL ON MULTIMODAL USER INTERFACES 10.2(2016):113-124.