The detection of facial muscle actions, as described in the Facial Action Coding System (FACS), is an ongoing effort with numerous potential applications. Examples of applications are online avatars mimicking a user's expression, automatic detection of user's feelings, and early detection of numerous diseases.
Our lab has a long history in detecting and analysing FACS Action Units (AUs). Early research aimed at detection of AU in static images based on facial component contour detection and an expert system encoding the FACS rules , , . This was the first approach to address the problem of facial muscle action (AU) detection by combining the analysis of frontal and profile face view. Our subsequent research addressed AU detection in video sequences using Motion History Images , , and automatically tracked facial points in either profile , , or frontal facial view , . This work was also the first to report on automatic recognition of temporal phases (onset, apex, offset) of facial actions in either frontal  or profile facial view . Recent work addresses (near) real-time detection of AUs using appearance-based approaches relying on novel dynamic appearance descriptors such as LPQ-TOP .
Currently we are working on detection of AUs in natural conversational data, using novel dynamic appearance descriptors.
O. Rudovic, V. Pavlovic, M. Pantic. Workshop on “What is in the Face”, Proceedings of the 12th European Conference on Computer Vision (ECCV-W'12), Part II. Florence, Italy, pp. 260 - 269, October 2012.
M. Pantic, I. Patras. IEEE Transactions on Systems, Man and Cybernetics - Part B. 36(2): pp. 433 - 449, 2006.
M. Pantic, I. Patras. Proceedings of IEEE Int'l Conf. Systems, Man and Cybernetics (SMC'05). Waikoloa, Hawaii, pp. 3358 - 3363, October 2005.
M. Pantic, L. Rothkrantz. IEEE Transactions on Systems, Man and Cybernetics - Part B. 34(3): pp. 1449 - 1461, 2004.
M. Pantic, L. Rothkrantz. Proceedings of IEEE Int'l Conf. Fuzzy Systems (IEEE-Fuzz'04). Budapest, Hungary, 1: pp. 39 - 44, July 2004.
M. Pantic, I. Patras. Proceedings of IEEE Int'l Conf. Multimedia and Expo (ICME'04). Taipei, Taiwan, 1: pp. 49 - 52, June 2004.
M. Pantic, I. Patras, L. Rothkrantz. Proceedings of IEEE Int'l Conf. Multimedia and Expo (ICME'02). Lausanne, Switzerland, 1: pp. 37 - 40, August 2002.
M. Pantic, L. Rothkrantz. Image and Vision Computing Journal. 18(11): pp. 881 - 905, 2000.
M. Pantic, L. Rothkrantz. Proceedings of AAAI Int'l Conf. Applications of AI (IAAI'00). Austin, USA, pp. 1026 - 1033, August 2000.
M. Pantic, L. Rothkrantz, H. Koppelaar. Proceedings of European Conf. Multimedia (EUROMEDIA '98). Leicester, UK, pp. 86 - 93, January 1998.