Facial expression recognition has attracted significant attention because of its usefulness in many applications such as human-computer interaction, face animation, analysis of social interaction, etc. Most existing methods deal with images (or image sequences) in which the depicted persons are relatively still and exhibit posed expressions in nearly frontal view. However, most of real-world applications relate to spontaneous human-to-human interactions (e.g., meeting summarization, political debates analysis, etc.), in which the assumption of having immovable subjects is unrealistic. This calls for a joint analysis of head pose and facial expression. Nevertheless, this is a significant research challenge mainly due to the large variation in the appearance of facial expressions under different head poses and the difficulty in decoupling these two sources of variation.
To address the problem of pose-invariant facial expression recognition, we exploit a three-step approach that is based on geometric features (i.e., the positions of the facial landmarks obtained by our point detector). In the first step, we perform head pose estimation by means of multi-class LDA trained on data belonging to a discrete set of head-poses. In the second step, we perform pose normalization by mapping the 2D positions of facial landmarks in non-frontal poses to the corresponding landmarks’ positions in the frontal pose. As these mappings are highly non-linear, we employ state-of-the-art regression models such as Gaussian Process Regression models to learn the mapping functions. Once we have normalized the pose, the last step of our approach is facial expression classification in the frontal pose attained using standard SVM classifier trained for the six basic emotions proposed by Ekman .