HCI

Active area of research is the design and development of human-centred interfaces which adapt automatically based on naturally occurring, multimodal, human behaviour. Such interfaces include affect-sensitive HCI and systems for implicit (behaviour-based) tagging of multimedia content.

Subtopics:

Related Publications

Human-Centered Implicit Tagging: Overview and Perspectives

M. Soleymani, M. Pantic. IEEE International Conference on Systems, Man and Cybernetics, 2012. IEEE, IEEE Press, pp. 3304 - 3309, October 2012.

Bibtex reference [hide]
@inproceedings{smc2012_implicit,
    author = {M. Soleymani and M. Pantic},
    pages = {3304--3309},
    booktitle = { IEEE International Conference on Systems, Man and Cybernetics, 2012},
    month = {October},
    organization = {IEEE},
    publisher = {IEEE Press},
    title = {Human-Centered Implicit Tagging: Overview and Perspectives},
    year = {2012},
}
Endnote reference [hide]
%0 Conference Proceedings
%T Human-Centered Implicit Tagging: Overview and Perspectives
%A Soleymani, M.
%A Pantic, M.
%B IEEE International Conference on Systems, Man and Cybernetics, 2012
%D 2012
%8 October
%I IEEE Press
%F smc2012_implicit
%P 3304-3309

The SEMAINE database: Annotated multimodal records of emotionally coloured conversations between a person and a limited agent

G. Mckeown, M. F. Valstar, R. Cowie, M. Pantic, M. Schroeder. IEEE Transactions on Affective Computing. 3: pp. 5 - 17, Issue 1. April 2012.

Bibtex reference [hide]
@article{mckeown2011semaine,
    author = {G. Mckeown and M. F. Valstar and R. Cowie and M. Pantic and M. Schroeder},
    pages = {5--17},
    journal = {IEEE Transactions on Affective Computing},
    month = {April},
    note = {Issue 1},
    title = {The SEMAINE database: Annotated multimodal records of emotionally coloured conversations between a person and a limited agent},
    volume = {3},
    year = {2012},
}
Endnote reference [hide]
%0 Journal Article
%T The SEMAINE database: Annotated multimodal records of emotionally coloured conversations between a person and a limited agent
%A Mckeown, G.
%A Valstar, M. F.
%A Cowie, R.
%A Pantic, M.
%A Schroeder, M.
%J IEEE Transactions on Affective Computing
%D 2012
%8 April
%V 3
%F mckeown2011semaine
%O Issue 1
%P 5-17

A multimodal database for affect recognition and implicit tagging

M. Soleymani, J. Lichtenauer, T. Pun, M. Pantic. IEEE Transactions on Affective Computing. 3: pp. 42 - 55, Issue 1. April 2012.

Bibtex reference [hide]
@article{soleymani2011multimodal,
    author = {M. Soleymani and J. Lichtenauer and T. Pun and M. Pantic},
    pages = {42--55},
    journal = {IEEE Transactions on Affective Computing},
    month = {April},
    note = {Issue 1},
    title = {A multimodal database for affect recognition and implicit tagging},
    volume = {3},
    year = {2012},
}
Endnote reference [hide]
%0 Journal Article
%T A multimodal database for affect recognition and implicit tagging
%A Soleymani, M.
%A Lichtenauer, J.
%A Pun, T.
%A Pantic, M.
%J IEEE Transactions on Affective Computing
%D 2012
%8 April
%V 3
%F soleymani2011multimodal
%O Issue 1
%P 42-55

HCI^2 Workbench: A Development Tool for Multimodal Human-Computer Interaction Systems

J. Shen, W. Shi, M. Pantic. Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG'11), ECAG'11- 2nd FG Workshop on Facial and Bodily Expressions for Control and Adaptation of Games . Santa Barbara, CA, USA, pp. 766 - 773, March 2011.

Bibtex reference [hide]
@inproceedings{shen2011hci,
    author = {J. Shen and W. Shi and M. Pantic},
    pages = {766--773},
    address = {Santa Barbara, CA, USA},
    booktitle = {Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG'11), ECAG'11- 2nd FG Workshop on Facial and Bodily Expressions for Control and Adaptation of Games },
    month = {March},
    title = {HCI^2 Workbench: A Development Tool for Multimodal Human-Computer Interaction Systems},
    year = {2011},
}
Endnote reference [hide]
%0 Conference Proceedings
%T HCI^2 Workbench: A Development Tool for Multimodal Human-Computer Interaction Systems
%A Shen, J.
%A Shi, W.
%A Pantic, M.
%B Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG?11), ECAG?11- 2nd FG Workshop on Facial and Bodily Expressions for Control and Adaptation of Games
%D 2011
%8 March
%C Santa Barbara, CA, USA
%F shen2011hci
%P 766-773

Come and Have an Emotional Workout with Sensitive Artificial Listeners!

M. Schroder, S. Pammi, H. Gunes, M. Pantic, M. F. Valstar, R. Cowie, G. Mckeown, D. Heylen, M. Maat, F. Eyben, B. Schuller, M. Wollmer, E. Bevacqua, C. Pelachaud, E. Sevin. Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG'11). Santa Barbara, CA, USA, pp. 646, March 2011.

Bibtex reference [hide]
@inproceedings{SchroderEtAlFG2011,
    author = {M. Schroder and S. Pammi and H. Gunes and M. Pantic and M. F. Valstar and R. Cowie and G. Mckeown and D. Heylen and M. Maat and F. Eyben and B. Schuller and M. Wollmer and E. Bevacqua and C. Pelachaud and E. Sevin},
    pages = {646},
    address = {Santa Barbara, CA, USA},
    booktitle = {Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG'11)},
    month = {March},
    title = {Come and Have an Emotional Workout with Sensitive Artificial Listeners!},
    year = {2011},
}
Endnote reference [hide]
%0 Conference Proceedings
%T Come and Have an Emotional Workout with Sensitive Artificial Listeners!
%A Schroder, M.
%A Pammi, S.
%A Gunes, H.
%A Pantic, M.
%A Valstar, M. F.
%A Cowie, R.
%A Mckeown, G.
%A Heylen, D.
%A Maat, M.
%A Eyben, F.
%A Schuller, B.
%A Wollmer, M.
%A Bevacqua, E.
%A Pelachaud, C.
%A Sevin, E.
%B Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG?11)
%D 2011
%8 March
%C Santa Barbara, CA, USA
%F SchroderEtAlFG2011
%P 646

Implicit Image Tagging via Facial Information

J. Jiao, M. Pantic. Proceedings ACM Int'l Workshop on Social Signal Processing (SSPW'10), Firenze, Italy. October 2010.

Bibtex reference [hide]
@inproceedings{Jiao2010iitvf,
    author = {J. Jiao and M. Pantic},
    booktitle = {Proceedings ACM Int'l Workshop on Social Signal Processing (SSPW'10), Firenze, Italy},
    month = {October},
    title = {Implicit Image Tagging via Facial Information},
    year = {2010},
}
Endnote reference [hide]
%0 Conference Proceedings
%T Implicit Image Tagging via Facial Information
%A Jiao, J.
%A Pantic, M.
%B Proceedings ACM Int?l Workshop on Social Signal Processing (SSPW?10), Firenze, Italy
%D 2010
%8 October
%F Jiao2010iitvf

The SEMAINE corpus of emotionally coloured character interactions

G. Mckeown, M. F. Valstar, R. Cowie, M. Pantic. Proceedings of IEEE Int'l Conf. Multimedia, Expo (ICME'10), Singapore. pp. 1079 - 1084, July 2010.

Bibtex reference [hide]
@inproceedings{McKeown2010tscoe,
    author = {G. Mckeown and M. F. Valstar and R. Cowie and M. Pantic},
    pages = {1079--1084},
    booktitle = {Proceedings of IEEE Int'l Conf. Multimedia, Expo (ICME'10), Singapore},
    month = {July},
    title = {The SEMAINE corpus of emotionally coloured character interactions},
    year = {2010},
}
Endnote reference [hide]
%0 Conference Proceedings
%T The SEMAINE corpus of emotionally coloured character interactions
%A Mckeown, G.
%A Valstar, M. F.
%A Cowie, R.
%A Pantic, M.
%B Proceedings of IEEE Int?l Conf. Multimedia, Expo (ICME?10), Singapore
%D 2010
%8 July
%F McKeown2010tscoe
%P 1079-1084

A Software Framework for Multimodal Human-Computer Interaction Systems

J. Shen, M. Pantic. Proceedings of IEEE Int'l Conf. Systems, Man, and Cybernetics (SMC'09). San Antonio, USA, pp. 2038 - 2045, October 2009.

Bibtex reference [hide]
@inproceedings{Shen2009asffm,
    author = {J. Shen and M. Pantic},
    pages = {2038--2045},
    address = {San Antonio, USA},
    booktitle = {Proceedings of IEEE Int'l Conf. Systems, Man, and Cybernetics (SMC'09)},
    month = {October},
    title = {A Software Framework for Multimodal Human-Computer Interaction Systems},
    year = {2009},
}
Endnote reference [hide]
%0 Conference Proceedings
%T A Software Framework for Multimodal Human-Computer Interaction Systems
%A Shen, J.
%A Pantic, M.
%B Proceedings of IEEE Int?l Conf. Systems, Man, and Cybernetics (SMC?09)
%D 2009
%8 October
%C San Antonio, USA
%F Shen2009asffm
%P 2038-2045

Implicit Human-Centered Tagging

M. Pantic, A. Vinciarelli. IEEE Signal Processing Magazine. 26(6): pp. 173 - 180, 2009.

Bibtex reference [hide]
@article{Pantic2009iht,
    author = {M. Pantic and A. Vinciarelli},
    pages = {173--180},
    journal = {IEEE Signal Processing Magazine},
    number = {6},
    title = {Implicit Human-Centered Tagging},
    volume = {26},
    year = {2009},
}
Endnote reference [hide]
%0 Journal Article
%T Implicit Human-Centered Tagging
%A Pantic, M.
%A Vinciarelli, A.
%J IEEE Signal Processing Magazine
%D 2009
%V 26
%N 6
%F Pantic2009iht
%P 173-180

A Demonstration of Audiovisual Sensitive Artificial Listeners

M. Schroeder, E. Bevacqua, F. Eyben, H. Gunes, D. Heylen, M. Maat, S. Pammi, M. Pantic, C. Pelachaud, B. Schuller, E. Sevin, M. F. Valstar, M. Woellmer. Proc. of ACII 2009, Proceedings of IEEE Int'l Conf. Affective Computing and Intelligent Interfaces (ACII'09). Amsterdam, Netherlands, 1: pp. 263 - 264, September 2009.

Bibtex reference [hide]
@inproceedings{Schroeder2009adoas,
    author = {M. Schroeder and E. Bevacqua and F. Eyben and H. Gunes and D. Heylen and M. Maat and S. Pammi and M. Pantic and C. Pelachaud and B. Schuller and E. Sevin and M. F. Valstar and M. Woellmer},
    pages = {263--264},
    address = {Amsterdam, Netherlands},
    booktitle = {Proc. of ACII 2009, Proceedings of IEEE Int'l Conf. Affective Computing and Intelligent Interfaces (ACII'09)},
    month = {September},
    title = {A Demonstration of Audiovisual Sensitive Artificial Listeners},
    volume = {1},
    year = {2009},
}
Endnote reference [hide]
%0 Conference Proceedings
%T A Demonstration of Audiovisual Sensitive Artificial Listeners
%A Schroeder, M.
%A Bevacqua, E.
%A Eyben, F.
%A Gunes, H.
%A Heylen, D.
%A Maat, M.
%A Pammi, S.
%A Pantic, M.
%A Pelachaud, C.
%A Schuller, B.
%A Sevin, E.
%A Valstar, M. F.
%A Woellmer, M.
%B Proc. of ACII 2009, Proceedings of IEEE Int?l Conf. Affective Computing and Intelligent Interfaces (ACII?09)
%D 2009
%8 September
%V 1
%C Amsterdam, Netherlands
%F Schroeder2009adoas
%P 263-264

Cost-effective solution to synchronized audio-visual capture using multiple sensors

J. Lichtenauer, M. F. Valstar, J. Shen, M. Pantic. Proceedings of IEEE Int'l Conf. Advanced Video and Signal Based Surveillance (AVSS'09). Genoa, Italy, pp. 324 - 329, September 2009.

Bibtex reference [hide]
@inproceedings{Lichtenauer2009cstsa,
    author = {J. Lichtenauer and M. F. Valstar and J. Shen and M. Pantic},
    pages = {324--329},
    address = {Genoa, Italy},
    booktitle = {Proceedings of IEEE Int'l Conf. Advanced Video and Signal Based Surveillance (AVSS'09)},
    month = {September},
    title = {Cost-effective solution to synchronized audio-visual capture using multiple sensors},
    year = {2009},
}
Endnote reference [hide]
%0 Conference Proceedings
%T Cost-effective solution to synchronized audio-visual capture using multiple sensors
%A Lichtenauer, J.
%A Valstar, M. F.
%A Shen, J.
%A Pantic, M.
%B Proceedings of IEEE Int?l Conf. Advanced Video and Signal Based Surveillance (AVSS?09)
%D 2009
%8 September
%C Genoa, Italy
%F Lichtenauer2009cstsa
%P 324-329

Implicit Human-Centered Tagging

A. Vinciarelli, N. Suditu, M. Pantic. Proceedings of IEEE Int'l Conf. Multimedia, Expo (ICME'09). Cancun, Mexico, pp. 1428 - 1431, July 2009.

Bibtex reference [hide]
@inproceedings{Vinciarelli2009iht,
    author = {A. Vinciarelli and N. Suditu and M. Pantic},
    pages = {1428--1431},
    address = {Cancun, Mexico},
    booktitle = {Proceedings of IEEE Int'l Conf. Multimedia, Expo (ICME'09)},
    month = {July},
    title = {Implicit Human-Centered Tagging},
    year = {2009},
}
Endnote reference [hide]
%0 Conference Proceedings
%T Implicit Human-Centered Tagging
%A Vinciarelli, A.
%A Suditu, N.
%A Pantic, M.
%B Proceedings of IEEE Int?l Conf. Multimedia, Expo (ICME?09)
%D 2009
%8 July
%C Cancun, Mexico
%F Vinciarelli2009iht
%P 1428-1431

Human-Centred Intelligent Human-Computer Interaction (HCI2): How far are we from attaining it?

M. Pantic, A. Nijholt, A. Pentland, T. Huang. Int'l Journal of Autonomous and Adaptive Communications Systems. 1(2): pp. 168 - 187, 2008.

Bibtex reference [hide]
@article{Pantic2008hihi,
    author = {M. Pantic and A. Nijholt and A. Pentland and T. Huang},
    pages = {168--187},
    journal = {Int'l Journal of Autonomous and Adaptive Communications Systems},
    number = {2},
    title = {Human-Centred Intelligent Human-Computer Interaction (HCI2): How far are we from attaining it?},
    volume = {1},
    year = {2008},
}
Endnote reference [hide]
%0 Journal Article
%T Human-Centred Intelligent Human-Computer Interaction (HCI2): How far are we from attaining it?
%A Pantic, M.
%A Nijholt, A.
%A Pentland, A.
%A Huang, T.
%J Int?l Journal of Autonomous and Adaptive Communications Systems
%D 2008
%V 1
%N 2
%F Pantic2008hihi
%P 168-187

Emotionally aware automated portrait painting

S. Colton, M. F. Valstar, M. Pantic. Proceedings of ACM Int'l Conf. Digital Interactive Media in Entertainment and Arts (DIMEA'08). Athens, Greece, pp. 304 - 311, September 2008.

Bibtex reference [hide]
@inproceedings{Colton2008eaapp,
    author = {S. Colton and M. F. Valstar and M. Pantic},
    pages = {304--311},
    address = {Athens, Greece},
    booktitle = {Proceedings of ACM Int'l Conf. Digital Interactive Media in Entertainment and Arts (DIMEA'08)},
    month = {September},
    title = {Emotionally aware automated portrait painting},
    year = {2008},
}
Endnote reference [hide]
%0 Conference Proceedings
%T Emotionally aware automated portrait painting
%A Colton, S.
%A Valstar, M. F.
%A Pantic, M.
%B Proceedings of ACM Int?l Conf. Digital Interactive Media in Entertainment and Arts (DIMEA?08)
%D 2008
%8 September
%C Athens, Greece
%F Colton2008eaapp
%P 304-311

Do people emote while engaged in HCI office scenarios?

M. Santos, M. Pantic. Proceedings of Int'l Conf. Interfaces and Human Computer Interaction (IHCI'08). Amsterdam, Netherlands, pp. 101 - 108, July 2008.

Bibtex reference [hide]
@inproceedings{Santos2008dpewe,
    author = {M. Santos and M. Pantic},
    pages = {101--108},
    address = {Amsterdam, Netherlands},
    booktitle = {Proceedings of Int'l Conf. Interfaces and Human Computer Interaction (IHCI'08)},
    month = {July},
    title = {Do people emote while engaged in HCI office scenarios?},
    year = {2008},
}
Endnote reference [hide]
%0 Conference Proceedings
%T Do people emote while engaged in HCI office scenarios?
%A Santos, M.
%A Pantic, M.
%B Proceedings of Int?l Conf. Interfaces and Human Computer Interaction (IHCI?08)
%D 2008
%8 July
%C Amsterdam, Netherlands
%F Santos2008dpewe
%P 101-108

Gaze-X: Adaptive, Affective, Multimodal Interface for Single-User Office Scenarios

L. Maat, M. Pantic. editors: T. Huang, A. Nijholt, M. Pantic, A. Pentland. Lecture Notes in Artificial Intelligence, Special Volume on Artificial Intelligence for Human Computing. vol. 4451, pp. 251 - 271, 2007.

Bibtex reference [hide]
@inbook{Maat2007gaami,
    author = {L. Maat and M. Pantic},
    pages = {251--271},
    booktitle = {Lecture Notes in Artificial Intelligence, Special Volume on Artificial Intelligence for Human Computing},
    editor = {T. Huang and A. Nijholt and A. Pentland and M. Pantic},
    title = {Gaze-X: Adaptive, Affective, Multimodal Interface for Single-User Office Scenarios},
    volume = {4451},
    year = {2007},
}
Endnote reference [hide]
%0 Book Section
%A Maat, L.
%A Pantic, M.
%E Huang, T.
%E Nijholt, A.
%E Pentland, A.
%E Pantic, M.
%B Lecture Notes in Artificial Intelligence, Special Volume on Artificial Intelligence for Human Computing: Adaptive, Affective, Multimodal Interface for Single-User Office Scenarios
%D 2007
%V 4451
%F Maat2007gaami
%P 251-271

Gaze-X: Adaptive affective multimodal interface for single-user office scenarios

L. Maat, M. Pantic. Proceedings of ACM Int'l Conf. Multimodal Interfaces (ICMI'06). Banff, Canada, pp. 171 - 178, November 2006.

Bibtex reference [hide]
@inproceedings{Maat2006gaami,
    author = {L. Maat and M. Pantic},
    pages = {171--178},
    address = {Banff, Canada},
    booktitle = {Proceedings of ACM Int'l Conf. Multimodal Interfaces (ICMI'06)},
    month = {November},
    title = {Gaze-X: Adaptive affective multimodal interface for single-user office scenarios},
    year = {2006},
}
Endnote reference [hide]
%0 Conference Proceedings
%T Gaze-X: Adaptive affective multimodal interface for single-user office scenarios
%A Maat, L.
%A Pantic, M.
%B Proceedings of ACM Int?l Conf. Multimodal Interfaces (ICMI?06)
%D 2006
%8 November
%C Banff, Canada
%F Maat2006gaami
%P 171-178

Face for Ambient Interface

M. Pantic. Key Note Paper, Lecture Notes in Artificial Intelligence, Special Volume on Ambient Intelligence in Everyday Life. vol. 3864, pp. 35 - 66, 2006.

Bibtex reference [hide]
@inbook{Pantic2006ffai,
    author = {M. Pantic},
    pages = {35--66},
    booktitle = {Key Note Paper, Lecture Notes in Artificial Intelligence, Special Volume on Ambient Intelligence in Everyday Life},
    title = {Face for Ambient Interface},
    volume = {3864},
    year = {2006},
}
Endnote reference [hide]
%0 Book Section
%A Pantic, M.
%B Key Note Paper, Lecture Notes in Artificial Intelligence, Special Volume on Ambient Intelligence in Everyday Life
%D 2006
%V 3864
%F Pantic2006ffai
%P 35-66

Towards an Affect-sensitive Multimodal Human-Computer Interaction

M. Pantic, L. Rothkrantz. Invited Paper, Proceedings of the IEEE, Special Issue on Multimodal Human-Computer Interaction (HCI). 91(9): pp. 1370 - 1390, 2003.

Bibtex reference [hide]
@article{Pantic2003taamh,
    author = {M. Pantic and L. Rothkrantz},
    pages = {1370--1390},
    journal = {Invited Paper, Proceedings of the IEEE, Special Issue on Multimodal Human-Computer Interaction (HCI)},
    number = {9},
    title = {Towards an Affect-sensitive Multimodal Human-Computer Interaction},
    volume = {91},
    year = {2003},
}
Endnote reference [hide]
%0 Journal Article
%T Towards an Affect-sensitive Multimodal Human-Computer Interaction
%A Pantic, M.
%A Rothkrantz, L.
%J Invited Paper, Proceedings of the IEEE, Special Issue on Multimodal Human-Computer Interaction (HCI)
%D 2003
%V 91
%N 9
%F Pantic2003taamh
%P 1370-1390

Affect-sensitive Multi-modal Monitoring in Ubiquitous Computing: Advances and Challenges

M. Pantic, L. Rothkrantz. Proceedings of AAAI / IEEE Int'l Conf. Enterprise Information Systems (ICEIS'01). Setubal, Portugal, pp. 466 - 474, July 2001.

Bibtex reference [hide]
@inproceedings{Pantic2001ammiu,
    author = {M. Pantic and L. Rothkrantz},
    pages = {466--474},
    address = {Setubal, Portugal},
    booktitle = {Proceedings of AAAI / IEEE Int'l Conf. Enterprise Information Systems (ICEIS'01)},
    month = {July},
    title = {Affect-sensitive Multi-modal Monitoring in Ubiquitous Computing: Advances and Challenges},
    year = {2001},
}
Endnote reference [hide]
%0 Conference Proceedings
%T Affect-sensitive Multi-modal Monitoring in Ubiquitous Computing: Advances and Challenges
%A Pantic, M.
%A Rothkrantz, L.
%B Proceedings of AAAI / IEEE Int?l Conf. Enterprise Information Systems (ICEIS?01)
%D 2001
%8 July
%C Setubal, Portugal
%F Pantic2001ammiu
%P 466-474