In multimedia management and sharing, tags assigned to the content play an important role in search and retrieval. With the help of tags, multimedia contents, such as, images, videos and web pages, can be efficiently classified, indexed and searched. The tagging technique currently adopted by most social media websites is explicit tagging, in which images are annotated based on the image content by human, either experts or amateurs. Although explicit image tagging has been successfully applied in a wide range of WEB 2.0 applications these days, it still has several drawbacks. First, the tags obtained by explicit tagging are likely to be inaccurate in practice. When tagging, people usually behave according to their individual interpretation of the content, personal and social needs without considering whether the tags can be used to improve the performance of retrieval. Second, it requires extra effort from users. Explicit tagging is sometimes really inconvenient for users since adding tags to images may take a large amount of time and energy.
In contrast to explicit tagging, implicit tagging is the technique to annotate multimedia data based on user's nonverbal reactions, such as facial expression and head gesture. The data is tagged in a implicit way because it is based on user's reaction without explicitly requesting a user to associate tag with the data. One possible application of implicit tagging is to assess the correctness of explicit tags. For example, if a user smiles or nodes the head while watching a tagged datum, this may imply that the tag is correct; however, if the user displays an expression of surprise or disappointment, the tag is likely to be incorrect. Another possible application of implicit tagging is to assign new explicit tags. For instance, if the user laughs while watching an untagged datum, it is likely that it contains some funny elements, thus can be labelled as "funny" or "interesting". Our current research focuses on using user's facial expressino and eye movement to predict the correctness of tags.
J. Jiao, M. Pantic. Proceedings ACM Int'l Workshop on Social Signal Processing (SSPW'10), Firenze, Italy. October 2010.
M. Pantic, A. Vinciarelli. IEEE Signal Processing Magazine. 26(6): pp. 173 - 180, 2009.
A. Vinciarelli, N. Suditu, M. Pantic. Proceedings of IEEE Int'l Conf. Multimedia, Expo (ICME'09). Cancun, Mexico, pp. 1428 - 1431, July 2009.