Facial point tracking

Tracking an object in an image sequence (video) is the process of dynamically computing moving objects’ location over time. Tracking has important applications which span several disciplines such as human-computer interaction, security and surveillance, augmented reality, video editing etc.  In most of the tracking systems a set of parameters related to the location of the object (state) are dynamically computed (e.g., translation parameters, affine transform parameters, deformation parameters etc). The lab has considerable experience in developing robust tracking algorithms and possesses a number of tools:

Tracking using particle filters frameworks.

The past decade, particle filtering has been the dominant paradigm for tracking the state of an object given a set of noisy observations. The lab proposed and developed various tracking algorithm within a particle filter framework ([1], [2]):

  • The lab has developed tracking algorithms by combining an observation model that explicitly addresses appearance changes caused by local deformations with an auxiliary particle filtering framework. The observation model can also deal with changing lighting conditions and shadows.
  • The lab has developed a novel particle filtering framework (the so-called Particle Filtering with Factorized Likelihoods) which deals with the problem of the high dimensionality of the state space.

The above algorithms have been successfully applied to for multiple independent objects tracking such as tracking of facial features (e.g., mouth and eye corners).

 

Hierarchical Incremental Active Appearance models.

Hierarchical face and gaze tracking is a wise-combination of Appearance-Based Trackers capable of estimating predefined facial features in monocular video sequences. This tracker describes the non-rigid facial movements of Eyelids, Irises, Eyebrows and Lips. At the same time, the rigid Head motion is estimated in 3D (three angles, image plane translation and in-deep scaling). This method is person independent and does not require prior training neither with facial textures, shapes nor facial actions. The tracker learns temporally On-Line the facial textures and smooth illumination changes. Therefore, this method is robust to short-time occlusions, drifting problems and out-of-plane movements. Consequently, unusual faces can be also tracked either to describe 3D head pose and location or the still visual features suitable of cognitive interpretation.

Online Kernel Subspace Learning for Robust Tracking

Currently, we are developing tracking algorithms based on an online learning approach for kernel-based Principal Component Analysis which incrementally updates the Eigenspace of the tracking features. We devise kernels which are robust to illumination changes and occlusion [3].
 

 

 

Related Publications

  1. Euler Principal Component Analysis

    S. Liwicki, G. Tzimiropoulos, S. Zafeiriou, M. Pantic. International Journal of Computer Vision. 101(3): pp. 498 - 518, 2013.

    Bibtex reference [hide]
    @article{liwicki2012euler,
        author = {S. Liwicki and G. Tzimiropoulos and S. Zafeiriou and M. Pantic},
        pages = {498--518},
        journal = {International Journal of Computer Vision},
        number = {3},
        title = {Euler Principal Component Analysis},
        volume = {101},
        year = {2013},
    }
    Endnote reference [hide]
    %0 Journal Article
    %T Euler Principal Component Analysis
    %A Liwicki, S.
    %A Tzimiropoulos, G.
    %A Zafeiriou, S.
    %A Pantic, M.
    %J International Journal of Computer Vision
    %D 2013
    %V 101
    %N 3
    %F liwicki2012euler
    %P 498-518

  2. Efficient Online Subspace Learning with an Indefinite Kernel for Visual Tracking and Recognition

    S. Liwicki, G. Tzimiropoulos, S. Zafeiriou, M. Pantic. IEEE Transactions on Neural Networks and Learning Systems. 23: pp. 1624 - 1636, October 2012.

    Bibtex reference [hide]
    @article{liwicki2012efficient,
        author = {S. Liwicki and G. Tzimiropoulos and S. Zafeiriou and M. Pantic},
        pages = {1624--1636},
        journal = {IEEE Transactions on Neural Networks and Learning Systems},
        month = {October},
        title = {Efficient Online Subspace Learning with an Indefinite Kernel for Visual Tracking and Recognition},
        volume = {23},
        year = {2012},
    }
    Endnote reference [hide]
    %0 Journal Article
    %T Efficient Online Subspace Learning with an Indefinite Kernel for Visual Tracking and Recognition
    %A Liwicki, S.
    %A Tzimiropoulos, G.
    %A Zafeiriou, S.
    %A Pantic, M.
    %J IEEE Transactions on Neural Networks and Learning Systems
    %D 2012
    %8 October
    %V 23
    %F liwicki2012efficient
    %P 1624-1636

  3. Subspace Analysis Of Arbitrarily Many Linear Filter Responses with an application to Face Tracking

    S. Zafeiriou, G. Tzimiropoulos, M. Pantic. Proceedings of IEEE Int’l Conf. Computer Vision and Pattern Recognition (CVPR-W’11), Workshop on Computer Vision for Computer Games. Colorado Springs, USA, pp. 37 - 42, June 2011.

    Bibtex reference [hide]
    @inproceedings{ZaferiouEtAlCVPRW2011,
        author = {S. Zafeiriou and G. Tzimiropoulos and M. Pantic},
        pages = {37--42},
        address = {Colorado Springs, USA},
        booktitle = {Proceedings of IEEE Int’l Conf. Computer Vision and Pattern Recognition (CVPR-W’11), Workshop on Computer Vision for Computer Games},
        month = {June},
        title = {Subspace Analysis Of Arbitrarily Many Linear Filter Responses with an application to Face Tracking},
        year = {2011},
    }
    Endnote reference [hide]
    %0 Conference Proceedings
    %T Subspace Analysis Of Arbitrarily Many Linear Filter Responses with an application to Face Tracking
    %A Zafeiriou, S.
    %A Tzimiropoulos, G.
    %A Pantic, M.
    %B Proceedings of IEEE Int’l Conf. Computer Vision and Pattern Recognition (CVPR-W’11), Workshop on Computer Vision for Computer Games
    %D 2011
    %8 June
    %C Colorado Springs, USA
    %F ZaferiouEtAlCVPRW2011
    %P 37-42

  4. Sparse Representations of Image Gradient Orientations for Visual Recognition and Tracking

    G. Tzimiropoulos, S. Zafeiriou, M. Pantic. Proceedings of IEEE Int’l Conf. Computer Vision and Pattern Recognition (CVPR-W’11), Workshop on CVPR for Human Behaviour Analysis. Colorado Springs, USA, pp. 26 - 33, June 2011.

    Bibtex reference [hide]
    @inproceedings{TzimiropoulosEtAlCVPR2011,
        author = {G. Tzimiropoulos and S. Zafeiriou and M. Pantic},
        pages = {26--33},
        address = {Colorado Springs, USA},
        booktitle = {Proceedings of IEEE Int’l Conf. Computer Vision and Pattern Recognition (CVPR-W’11), Workshop on CVPR for Human Behaviour Analysis},
        month = {June},
        title = {Sparse Representations of Image Gradient Orientations for Visual Recognition and Tracking },
        year = {2011},
    }
    Endnote reference [hide]
    %0 Conference Proceedings
    %T Sparse Representations of Image Gradient Orientations for Visual Recognition and Tracking
    %A Tzimiropoulos, G.
    %A Zafeiriou, S.
    %A Pantic, M.
    %B Proceedings of IEEE Int’l Conf. Computer Vision and Pattern Recognition (CVPR-W’11), Workshop on CVPR for Human Behaviour Analysis
    %D 2011
    %8 June
    %C Colorado Springs, USA
    %F TzimiropoulosEtAlCVPR2011
    %P 26-33

  5. Fast and Robust Appearance-based Tracking

    S. Liwicki, S. Zafeiriou, G. Tzimiropoulos, M. Pantic. Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG'11). Santa Barbara, CA, USA, pp. 507 - 513, March 2011.

    Bibtex reference [hide]
    @inproceedings{StephanEtAlFG2011,
        author = {S. Liwicki and S. Zafeiriou and G. Tzimiropoulos and M. Pantic},
        pages = {507--513},
        address = {Santa Barbara, CA, USA},
        booktitle = {Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG'11)},
        month = {March},
        title = {Fast and Robust Appearance-based Tracking},
        year = {2011},
    }
    Endnote reference [hide]
    %0 Conference Proceedings
    %T Fast and Robust Appearance-based Tracking
    %A Liwicki, S.
    %A Zafeiriou, S.
    %A Tzimiropoulos, G.
    %A Pantic, M.
    %B Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG?11)
    %D 2011
    %8 March
    %C Santa Barbara, CA, USA
    %F StephanEtAlFG2011
    %P 507-513

  6. Tracking deformable motion

    I. Patras, M. Pantic. Proceedings of IEEE Int'l Conf. Systems, Man and Cybernetics (SMC'05). Waikoloa, Hawaii, pp. 1066 - 1071, October 2005.

    Bibtex reference [hide]
    @inproceedings{Patras2005tdm,
        author = {I. Patras and M. Pantic},
        pages = {1066--1071},
        address = {Waikoloa, Hawaii},
        booktitle = {Proceedings of IEEE Int'l Conf. Systems, Man and Cybernetics (SMC'05)},
        month = {October},
        title = {Tracking deformable motion},
        year = {2005},
    }
    Endnote reference [hide]
    %0 Conference Proceedings
    %T Tracking deformable motion
    %A Patras, I.
    %A Pantic, M.
    %B Proceedings of IEEE Int?l Conf. Systems, Man and Cybernetics (SMC?05)
    %D 2005
    %8 October
    %C Waikoloa, Hawaii
    %F Patras2005tdm
    %P 1066-1071

  7. Particle Filtering with Factorized Likelihoods for Tracking Facial Features

    I. Patras, M. Pantic. Proceedings of IEEE Int'l Conf. Face and Gesture Recognition (FG'04). Seoul, Korea, pp. 97 - 102, May 2004.

    Bibtex reference [hide]
    @inproceedings{Patras2004pfwfl,
        author = {I. Patras and M. Pantic},
        pages = {97--102},
        address = {Seoul, Korea},
        booktitle = {Proceedings of IEEE Int'l Conf. Face and Gesture Recognition (FG'04)},
        month = {May},
        title = {Particle Filtering with Factorized Likelihoods for Tracking Facial Features},
        year = {2004},
    }
    Endnote reference [hide]
    %0 Conference Proceedings
    %T Particle Filtering with Factorized Likelihoods for Tracking Facial Features
    %A Patras, I.
    %A Pantic, M.
    %B Proceedings of IEEE Int?l Conf. Face and Gesture Recognition (FG?04)
    %D 2004
    %8 May
    %C Seoul, Korea
    %F Patras2004pfwfl
    %P 97-102