Datasets
Code
The Matlab implementation was written by Antonis Oikonomopoulos. The code has been compiled on a Windows machine, hence this downloadable version of our detector will work only on a Windows machine. You will need Matlab Component Runtime installed on your computer for the program to run (Matlab 2010b, equiv. MCR 7.14).
Description: The detector is trained for the Canal9 dataset of political debates.
The gestures for which the detector is trained include: Hand Wag, Fore-Finger Wag, Fore-Finger Raise, Half Fore-Finger Raise, Hand Cross, Hand Scissor and Arm Folding.
Detection is performed in both spatial and temporal domains. In the spatial domain the detector returns an estimate of the gesture center at every frame. In the temporal domain the start/end frames of the gesture are returned.
Finally, the amplitude of the performed gesture is also estimated, with respect to the average face size of the subject that performs the gesture.
The main differences of this detector with respect to the 2010 version are:
1) The inclusion of per-frame spatial estimates of the gesture center.
2) The estimation of the gesture's amplitude with respect to the average face size of the subject that is performing the gesture.
The detector is described in detail in the following works:
[1] A. Oikonomopoulos, I. Patras, M. Pantic., 'Spatiotemporal Localization and Categorization of Human Actions in Unsegmented Image Sequences', IEEE Transactions on Image Processing. 20(4): pp. 1126 - 1140, April 2011.
[2] A. Oikonomopoulos, I. Patras, M. Pantic., 'Discriminative Space-time Voting for Joint Recognition and Localization of Actions', Proceedings ACM Int'l Workshop on Social Signal Processing (SSPW'10), Firenze, Italy. October 2010.
The detector can be downloaded as follows: