Happy Emotion Recognition From Unconstrained Videos Using 3D Hybrid Deep Features

doi:10.1109/ACCESS.2021.3061744

	Happy Emotion Recognition From Unconstrained Videos Using 3D Hybrid Deep Features
	Samadiani, Najmeh 1; Huang, Guangyan 1; Hu, Yu 2; Li, Xiaowei 2
	2021
发表期刊	IEEE ACCESS
ISSN	2169-3536
卷号	9 页码:35524-35538
摘要	Facial expressions have been proven to be the most effective way for the brain to recognize human emotions in a variety of contexts. With the exponentially increasing research for emotion detection in recent years, facial expression recognition has become an attractive, hot research topic to identify various basic emotions. Happy emotion is one of such basic emotions with many applications, which is more likely recognized by facial expressions than other emotion measurement instruments (e.g., audio/speech, textual and physiological sensing). Nowadays, most methods have been developed for identifying multiple types of emotions, which aim to achieve the best overall precision for all emotions; it is hard for them to optimize the recognition accuracy for single emotion (e.g., happiness). Only a few methods are designed to recognize single happy emotion captured in the unconstrained videos; however, their limitations lie in that the processing of severe head pose variations has not been considered, and the accuracy is still not satisfied. In this paper, we propose a Happy Emotion Recognition model using the 3D hybrid deep and distance features (HappyER-DDF) method to improve the accuracy by utilizing and extracting two different types of deep visual features. First, we employ a hybrid 3D Inception-ResNet neural network and long-short term memory (LSTM) to extract dynamic spatial-temporal features among sequential frames. Second, we detect facial landmarks' features and calculate the distance between each facial landmark and a reference point on the face (e.g., nose peak) to capture their changes when a person starts to smile (or laugh). We implement the experiments using both feature-level and decision-level fusion techniques on three unconstrained video datasets. The results demonstrate that our HappyER-DDF method is arguably more accurate than several currently available facial expression models.
关键词	Feature extraction Emotion recognition Face recognition Videos Three-dimensional displays Long short term memory Visualization Facial landmarks facial expression recognition long short term memory multi-layer neural networks happy emotion recognition
DOI	10.1109/ACCESS.2021.3061744
收录类别	SCI
语种	英语
资助项目	Australia Research Council (ARC)[DP190100587]
WOS研究方向	Computer Science ; Engineering ; Telecommunications
WOS类目	Computer Science, Information Systems ; Engineering, Electrical & Electronic ; Telecommunications
WOS记录号	WOS:000637163800001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：9[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/16778
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Huang, Guangyan
作者单位	1.Deakin Univ, Sch Informat Technol, Burwood, Vic 3125, Australia 2.Univ Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Samadiani, Najmeh,Huang, Guangyan,Hu, Yu,et al. Happy Emotion Recognition From Unconstrained Videos Using 3D Hybrid Deep Features[J]. IEEE ACCESS,2021,9:35524-35538.
APA	Samadiani, Najmeh,Huang, Guangyan,Hu, Yu,&Li, Xiaowei.(2021).Happy Emotion Recognition From Unconstrained Videos Using 3D Hybrid Deep Features.IEEE ACCESS,9,35524-35538.
MLA	Samadiani, Najmeh,et al."Happy Emotion Recognition From Unconstrained Videos Using 3D Hybrid Deep Features".IEEE ACCESS 9(2021):35524-35538.