The detection of facial muscle actions, as described in the Facial Action Coding System (FACS), is an ongoing effort with numerous potential applications. Examples of applications are online avatars mimicking a user's expression, automatic detection of user's feelings, and early detection of numerous diseases.
Our lab has a long history in detecting and analysing FACS Action Units (AUs). Early research aimed at detection of AU in static images based on facial component contour detection and an expert system encoding the FACS rules [1], [2], [3]. This was the first approach to address the problem of facial muscle action (AU) detection by combining the analysis of frontal and profile face view. Our subsequent research addressed AU detection in video sequences using Motion History Images [4], [5], and automatically tracked facial points in either profile [6], [7], or frontal facial view [8], [9]. This work was also the first to report on automatic recognition of temporal phases (onset, apex, offset) of facial actions in either frontal [8] or profile facial view [7]. Recent work addresses (near) real-time detection of AUs using appearance-based approaches relying on novel dynamic appearance descriptors such as LPQ-TOP [10].
Currently we are working on detection of AUs in natural conversational data, using novel dynamic appearance descriptors.
Maja Pantic, Bihan Jiang, Sebastian Kaltwang, Dimitrios Kollias, Brais Martinez, Ognjen Rudovic, Michel Francois Valstar, Robert Walecki
B. Martinez, M. F. Valstar, B. Jiang, M. Pantic. IEEE Transactions on Affective Computing. 2017.
G. Sandbach, S. Zafeiriou, M. Pantic. Proceedings of the IEEE International Conference on Image Processing (ICIP 2012). Orlando, FL, USA, pp. 1813 - 1816, October 2012.
O. Rudovic, V. Pavlovic, M. Pantic. Workshop on “What is in the Face”, Proceedings of the 12th European Conference on Computer Vision (ECCV-W'12), Part II. Florence, Italy, pp. 260 - 269, October 2012.
B. Jiang, M. F. Valstar, M. Pantic. Proceedings of the ASE/IEEE International Conference on Social Computing (SocialCom 2012). Amsterdam, The Netherlands, September 2012.
G. Sandbach, S. Zafeiriou, M. Pantic. Proceedings of the British Machine Vision Conference (BMVC 2012). Guildford, UK, September 2012.
B. Jiang, M. F. Valstar, M. Pantic. Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG'11). Santa Barbara, CA, USA, pp. 314 - 321, March 2011.
S. Koelstra, M. Pantic, I. Patras. IEEE Transactions on Pattern Analysis and Machine Intelligence. 32(11): pp. 1940 - 1954, november 2010.
S. Koelstra, M. Pantic. Proceedings of IEEE Int'l Conf. Automatic Face and Gesture Recognition (FG'08), Amsterdam, Netherlands. pp. 1 - 8, September 2008.
M. F. Valstar, M. Pantic. IEEE Int'l Conf. Computer Vision (ICCV'07), Workshop on Human Computer Interaction (HCI'07). Rio de Janeiro, Brazil, pp. 118 - 127, October 2007.
M. Pantic, I. Patras. IEEE Transactions on Systems, Man and Cybernetics - Part B. 36(2): pp. 433 - 449, 2006.
M. F. Valstar, M. Pantic. Proceedings of IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR'06). New York, USA, 3: pp. 149, June 2006.
M. Pantic, I. Patras. Proceedings of IEEE Int'l Conf. Systems, Man and Cybernetics (SMC'05). Waikoloa, Hawaii, pp. 3358 - 3363, October 2005.
M. F. Valstar, I. Patras, M. Pantic. Proceedings of IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR-W'05). San Diego, USA, 3: pp. 76, June 2005.
M. F. Valstar, M. Pantic, I. Patras. Proceedings of IEEE Int'l Conf. Systems, Man and Cybernetics (SMC'04). The Hague, Netherlands, pp. 635 - 640, October 2004.
M. Pantic, L. Rothkrantz. IEEE Transactions on Systems, Man and Cybernetics - Part B. 34(3): pp. 1449 - 1461, 2004.
M. F. Valstar, I. Patras, M. Pantic. Proceedings of IEEE Int'l Workshop on Robot-Human Interaction (RO-MAN'04). Kurashiki, Japan, pp. 253 - 258, September 2004.
M. Pantic, L. Rothkrantz. Proceedings of IEEE Int'l Conf. Fuzzy Systems (IEEE-Fuzz'04). Budapest, Hungary, 1: pp. 39 - 44, July 2004.
M. Pantic, I. Patras. Proceedings of IEEE Int'l Conf. Multimedia and Expo (ICME'04). Taipei, Taiwan, 1: pp. 49 - 52, June 2004.
M. Pantic, I. Patras, L. Rothkrantz. Proceedings of IEEE Int'l Conf. Multimedia and Expo (ICME'02). Lausanne, Switzerland, 1: pp. 37 - 40, August 2002.
M. Pantic, L. Rothkrantz. Image and Vision Computing Journal. 18(11): pp. 881 - 905, 2000.
M. Pantic, L. Rothkrantz. Proceedings of AAAI Int'l Conf. Applications of AI (IAAI'00). Austin, USA, pp. 1026 - 1033, August 2000.
M. Pantic, L. Rothkrantz, H. Koppelaar. Proceedings of European Conf. Multimedia (EUROMEDIA '98). Leicester, UK, pp. 86 - 93, January 1998.