Fusion of Facial Expresion and EEG for Multi Modal Emotion Recognition

ABSTRACT

Recently, the research of emotional recognition has attained increasing attention as a result of the fast development of non-invasive sensor technologies, machine learning algorithms and computing capabilities, with single modal emotional recognition. The multimodal paradigm brings complementary information for emotional recognition. Therefore, in this study, we introduced a decision-level fusion system to constantly diagnose emotions by fusing electroencephalography (EEG) with the facial expressions. This paper examines the backdrops and hot spots of data-driven multimodal emotion fusion. With the help of real-time mental health monitoring system, several parameters including the ongoing creation of multimodal emotion data sets and the analysis of multimodal characteristics are addressed. 

FULL TEXT