The lab is developing tools for static and dynamic 3D facial expression and action unit recognition.
Features based on Local Binary Patterns have been developed for analysis of 3D facial geometries for the purpose of facial action unit detection. They use the normals, depth map, or our proposed Azimuthal Projection Distance Image (APDI) as input. We then extract discriminative features via the LBP algorithm along with Gabor and monogenic filters, or by applying Local Phase Quantisation to these representations.
A facial expression sequence is modelled by a four state model (neutral-onset-apex-offset). The method we developed uses free-form deformations to capture the 3D motion between frames. Vector projections of the motion in the sequence with space and time are produced and used to perform quad-tree decompositions for each of the onset and offset temporal segments in the expression (some examples of quadtree decompositions and the corresponding motion projections are depicted above). Feature extraction and classification can then be performed on each frame in the image sequence, and the results of this used to produce temporal models for the full expression .
G. Sandbach, S. Zafeiriou, M. Pantic, D. Rueckert. Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG'11), Special Session: 3D Facial Behavior Analysis and Understanding. Santa Barbara, CA, USA, pp. 406 - 413, March 2011.