Detection of static geometric facial features

The geometry of a face can tell many things about someone. It harbours information on gender, facial expression, identity and even birth defects. Many contemporary facial expression recognition systems employ state of the art trackers, that track the positions of facial points throughout a video. However, how to initialise these trackers accurately and robustly is still ongoing research.

One of our aims is to perform a fast and reliable location of these facial points in order to perform further face analysis, using both visible and thermal imagery. To this end, we apply Machine Learning techniques (SVRs and boosting) in the Computer Vision field. Our initial efforts to this end resulted in a reliable facial point detector based on the responses of Gabor filters and ensemble learning (Gntleboost). This work is explained in [1]. The compailed version of the detector can be found in the 'Resources'. Our current efforts aim at obtaining a reliable real-time detection, so that the software can be used in real-life applications. Specifically, we use regression techniques to obtain fast estimates of the facial feature location, and combine the predictions of multiple points into probabilistic models that encode the geometric configuration of the face to constrain the solutions and boost the robustness of the method. More information about this research can be found in [2].

A complied version of this work will be available from this website soon for Mac and Windows.

Involved group members

Maja Pantic, James Booth, Baris Gecer, Brais Martinez, Christos Sagonas, Michel Francois Valstar

Related Publications

  1. Local Evidence Aggregation for Regression Based Facial Point Detection

    B. Martinez, M. F. Valstar, X. Binefa, M. Pantic. IEEE Transactions on Pattern Analysis and Machine Intelligence. 35(5): pp. 1149 - 1163, 2013.

    [pdf] [bibtex] [endnote]
    Bibtex reference [hide]
    @article{braisTpami12,
        author = {B. Martinez and M. F. Valstar and X. Binefa and M. Pantic},
        pages = {1149--1163},
        journal = { IEEE Transactions on Pattern Analysis and Machine Intelligence},
        number = {5},
        title = {Local Evidence Aggregation for Regression Based Facial Point Detection},
        volume = {35},
        year = {2013},
    }
    Endnote reference [hide]
    %0 Journal Article
    %T Local Evidence Aggregation for Regression Based Facial Point Detection
    %A Martinez, B.
    %A Valstar, M. F.
    %A Binefa, X.
    %A Pantic, M.
    %J IEEE Transactions on Pattern Analysis and Machine Intelligence
    %D 2013
    %V 35
    %N 5
    %F braisTpami12
    %P 1149-1163

  2. Facial Point Detection using Boosted Regression and Graph Models

    M. F. Valstar, B. Martinez, X. Binefa, M. Pantic. Proceedings of IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR'10). San Francisco, USA, pp. 2729 - 2736, June 2010.

    [pdf] [bibtex] [endnote]
    Bibtex reference [hide]
    @inproceedings{Valstar2010fpdub,
        author = {M. F. Valstar and B. Martinez and X. Binefa and M. Pantic},
        pages = {2729--2736},
        address = {San Francisco, USA},
        booktitle = {Proceedings of IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR'10)},
        month = {June},
        title = {Facial Point Detection using Boosted Regression and Graph Models},
        year = {2010},
    }
    Endnote reference [hide]
    %0 Conference Proceedings
    %T Facial Point Detection using Boosted Regression and Graph Models
    %A Valstar, M. F.
    %A Martinez, B.
    %A Binefa, X.
    %A Pantic, M.
    %B Proceedings of IEEE Int?l Conf. Computer Vision and Pattern Recognition (CVPR?10)
    %D 2010
    %8 June
    %C San Francisco, USA
    %F Valstar2010fpdub
    %P 2729-2736

  3. How to distinguish posed from spontaneous smiles using geometric features

    M. F. Valstar, H. Gunes, M. Pantic. Proceedings of ACM Int'l Conf. Multimodal Interfaces (ICMI'07). Nagoya, Japan, pp. 38 - 45, November 2007.

    [pdf] [bibtex] [endnote]
    Bibtex reference [hide]
    @inproceedings{Valstar2007htdpf,
        author = {M. F. Valstar and H. Gunes and M. Pantic},
        pages = {38--45},
        address = {Nagoya, Japan},
        booktitle = {Proceedings of ACM Int'l Conf. Multimodal Interfaces (ICMI'07)},
        month = {November},
        title = {How to distinguish posed from spontaneous smiles using geometric features},
        year = {2007},
    }
    Endnote reference [hide]
    %0 Conference Proceedings
    %T How to distinguish posed from spontaneous smiles using geometric features
    %A Valstar, M. F.
    %A Gunes, H.
    %A Pantic, M.
    %B Proceedings of ACM Int?l Conf. Multimodal Interfaces (ICMI?07)
    %D 2007
    %8 November
    %C Nagoya, Japan
    %F Valstar2007htdpf
    %P 38-45

  4. Spontaneous vs. posed facial behavior: Automatic analysis of brow actions

    M. F. Valstar, M. Pantic, Z. Ambadar, J. Cohn. Proceedings of ACM Int'l Conf. Multimodal Interfaces (ICMI'06). Banff, Canada, pp. 162 - 170, November 2006.

    [pdf] [bibtex] [endnote]
    Bibtex reference [hide]
    @inproceedings{Valstar2006svpfb,
        author = {M. F. Valstar and M. Pantic and Z. Ambadar and J. Cohn},
        pages = {162--170},
        address = {Banff, Canada},
        booktitle = {Proceedings of ACM Int'l Conf. Multimodal Interfaces (ICMI'06)},
        month = {November},
        title = {Spontaneous vs. posed facial behavior: Automatic analysis of brow actions},
        year = {2006},
    }
    Endnote reference [hide]
    %0 Conference Proceedings
    %T Spontaneous vs. posed facial behavior: Automatic analysis of brow actions
    %A Valstar, M. F.
    %A Pantic, M.
    %A Ambadar, Z.
    %A Cohn, J.
    %B Proceedings of ACM Int?l Conf. Multimodal Interfaces (ICMI?06)
    %D 2006
    %8 November
    %C Banff, Canada
    %F Valstar2006svpfb
    %P 162-170

  5. Fully automatic facial feature point detection using Gabor feature based boosted classifiers

    D. Vukadinovic, M. Pantic. Proceedings of IEEE Int'l Conf. Systems, Man and Cybernetics (SMC'05). Waikoloa, Hawaii, pp. 1692 - 1698, October 2005.

    [pdf] [bibtex] [endnote]
    Bibtex reference [hide]
    @inproceedings{Vukadinovic2005faffp,
        author = {D. Vukadinovic and M. Pantic},
        pages = {1692--1698},
        address = {Waikoloa, Hawaii},
        booktitle = {Proceedings of IEEE Int'l Conf. Systems, Man and Cybernetics (SMC'05)},
        month = {October},
        title = {Fully automatic facial feature point detection using Gabor feature based boosted classifiers},
        year = {2005},
    }
    Endnote reference [hide]
    %0 Conference Proceedings
    %T Fully automatic facial feature point detection using Gabor feature based boosted classifiers
    %A Vukadinovic, D.
    %A Pantic, M.
    %B Proceedings of IEEE Int?l Conf. Systems, Man and Cybernetics (SMC?05)
    %D 2005
    %8 October
    %C Waikoloa, Hawaii
    %F Vukadinovic2005faffp
    %P 1692-1698

  6. A hybrid approach to mouth features detection

    M. Pantic, M. Tomc, L. Rothkrantz. Proceedings of IEEE Int'l Conf. Systems, Man and Cybernetics (SMC'01). Tucson, USA, pp. 1188 - 1193, October 2001.

    [pdf] [bibtex] [endnote]
    Bibtex reference [hide]
    @inproceedings{Pantic2001ahatm,
        author = {M. Pantic and M. Tomc and L. Rothkrantz},
        pages = {1188--1193},
        address = {Tucson, USA},
        booktitle = {Proceedings of IEEE Int'l Conf. Systems, Man and Cybernetics (SMC'01)},
        month = {October},
        title = {A hybrid approach to mouth features detection},
        year = {2001},
    }
    Endnote reference [hide]
    %0 Conference Proceedings
    %T A hybrid approach to mouth features detection
    %A Pantic, M.
    %A Tomc, M.
    %A Rothkrantz, L.
    %B Proceedings of IEEE Int?l Conf. Systems, Man and Cybernetics (SMC?01)
    %D 2001
    %8 October
    %C Tucson, USA
    %F Pantic2001ahatm
    %P 1188-1193