FG-2020 Competition: Affective Behavior Analysis in-the-wild Challenge

 

Latest News

The Competition: Affective Behavior Analysis in-the-wild Challenge will be held in conjunction with the IEEE International Conference on Automatic Face and Gesture Recognition (FG) 2020, in Buenos Aires, Argentina, 18-22 May 2020.


For any requests or enquiries, please contact: dimitrios.kollias15@imperial.ac.uk 

 

Organisers

 

Chairs:

Stefanos Zafeiriou, Imperial College London, UK                                s.zafeiriou@imperial.ac.uk

Dimitrios Kollias, Imperial College London, UK                                   dimitrios.kollias15@imperial.ac.uk          

Attila Schulc,  Realeyes - Emotional Intelligence                                attila.schulc@realeyesit.com    

Elnar Hajiyev, Realeyes - Emotional Intelligence                                elnar@realeyesit.com 



Scope


This Competition aims at advancing the state-of-the-art in the problem of analysis of human affective behavior in-the-wild. Representing human emotions has been a basic topic of research. The most frequently used emotion representation is the categorical one, including the seven basic categories, i.e., Anger, Disgust, Fear, Happiness, Sadness, Surprise and Neutral. Discrete emotion representation can also be described in terms of the Facial Action Coding System model, in which all possible facial actions are described in terms of Action Units (AUs). Finally, the dimensional model of affect has been proposed as a means to distinguish between subtly different displays of affect and encode small changes in the intensity of each emotion on a continuous scale. The 2-D Valence and Arousal Space (VA-Space) is the most usual dimensional emotion representation; valence shows how positive or negative an emotional state is, whilst arousal shows how passive or active it is.

 

The Competition

 

The Competition is split into three Challenges-Tracks, which are based, for the first time, on the same database; these target: dimensional and categorical affect recognition. In particular, the 3 Challenges-Tracks are: 

  • valence-arousal estimation
  • seven basic expression classification
  • facial action unit detection

These Challenges will produce a significant step forward when compared to previous events. In particular, they use the Aff-Wild2, the first comprehensive benchmark for all three affect recognition tasks in-the-wild. 

Participants are invited to participate in one or more of these Challenges.

There will be one winner per Challenge-Track; the winners are expected to contribute a paper describing their approach, methodlogy and results; the accepted winning papers will be part of the IEEE FG 2020 proceedings; all other teams will be able to able to submit a paper describing their solutions and final results to the Workshop that we are also organizing at FG2020.

For the purpose of the Challenges and to facilitate training, especially for people that do not have access to face detectors/tracking algorithms, we provide the cropped images and the cropped & aligned ones.

 

A white paper will be created that describes the data used, their annotations, the baselines and the performance measures.

 

Data: Aff-Wild2  

 

Aff-Wild2 is an extension of the Aff-Wild database (both in terms of annotations and videos). Aff-Wild2 is: i) an in-the-wild audiovisual database;  ii) a large scale database consisting of 564 videos of around 2.8M frames (the largest existing one); iii) the first database to contain annotations for all 3 behavior tasks (and also the first audiovisual database with annotations for AUs). 561 videos contain annotations for valence-arousal, 547 videos contain annotations for the 7 basic expressions and 63 videos contain annotations for 8 AUs (AU1,AU2,AU4,AU6,AU12,AU15,AU20,AU25).


How to participate


To participate, you need to register your team.
For this, please send us an email to: 
dimitrios.kollias15@imperial.ac.uk 
with the title "Affective Behavior Analysis in-the-wild Competition: Team Registration".
In this email include the following information:

Team Name
Team Members
Affiliation


There is no maximum number of participants in each team.

As a reply, you will receive access to the dataset's videos, annotations, cropped and cropped-aligned images and other important information.

 

At the end of the Challenges, each team will have to send us: i) their predictions on the test set, ii) a link to a Github repository where their solution/source code will be stored, and iii) a link to an ArXiv paper with 2-6 pages describing their proposed methodology, data used and results. After that, the winner of each Challenge will be announced and all (non-winning) teams will be able to submit a paper describing their solutions and final results to the Workshop that we are also organizing entitled 'Affect Recognition in-the-wild: Uni/Multi-Modal Analysis'.

 

Rules

 

• Participants can contribute to any of the 3 Challenges.

• In order to take part in any Challenge, participants will have to register by sending an email to the organizers containing the following information: Team Name, Team Members, Affiliation.

• Participants can use scene/background/body pose etc. information along with the face information.

• Any face detector whether commercial or academic can be used in the challenge. The paper accompanying the challenge result submission should contain clear details of the detectors/libraries used.

The participants are free to use external data for training along with the Aff-Wild2 partitions. However, this should be clearly discussed in the accompanying paper

The participants are free to use any pre-trained network, even the publicly available ones (CNN, AffWildNet) that displayed the best performance in the (former) Aff-Wild database (part of Aff-Wild2). 

 

Performance Assessment

 

1)  For Challenge-Track 1: Valence-Arousal estimation : the Concordance Correlation Coefficient (CCC) will be the metric to judge the performance of the models.

 

2) For Challenge-Track 2: 7 Basic Expression Classification: the perfromance metric will be:
0.67* F1_Score + 0.33* Accuracy
Note: F1 Score is the unweighted mean and Accuracy is the total accuracy

 

3) For Challenge-Track 3: 8 Action Unit Detection: the perfromance metric will be:
0.5* F1_Score + 0.5* Accuracy

Note: F1 Score is the unweighted mean and Accuracy is the total accuracy

 


References


If you use the above data, you must cite all following papers: 


  • D. Kollias, S. Zafeiriou: "Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace". BMVC, 2019

@article{kollias2019expression, title={Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace}, author={Kollias, Dimitrios and Zafeiriou, Stefanos}, journal={arXiv preprint arXiv:1910.04855}, year={2019} }


  • D. Kollias, S. Zafeiriou: "Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition", 2018

@article{kollias2018aff, title={Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition}, author={Kollias, Dimitrios and Zafeiriou, Stefanos}, journal={arXiv preprint arXiv:1811.07770}, year={2018} }


  • D. Kollias, S. Zafeiriou: "A Multi-Task Learning \& Generation Framework: Valence-Arousal, Action Units \& Primary Expressions", 2018

@article{kollias2018multi, title={A Multi-Task Learning \& Generation Framework: Valence-Arousal, Action Units \& Primary Expressions}, author={Kollias, Dimitrios and Zafeiriou, Stefanos}, journal={arXiv preprint arXiv:1811.07771}, year={2018} }

 

  • D. Kollias, et. al.: "Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond". International Journal of Computer Vision (IJCV), 2019

@article{kollias2019deep, title={Deep affect prediction in-the-wild: Aff-wild database and challenge, deep architectures, and beyond}, author={Kollias, Dimitrios and Tzirakis, Panagiotis and Nicolaou, Mihalis A and Papaioannou, Athanasios and Zhao, Guoying and Schuller, Bj{\"o}rn and Kotsia, Irene and Zafeiriou, Stefanos}, journal={International Journal of Computer Vision}, pages={1--23}, year={2019}, publisher={Springer} }

 

  • S. Zafeiriou, et. al. "Aff-Wild: Valence and Arousal in-the-wild Challenge", CVPRW, 2017

@inproceedings{zafeiriou2017aff, title={Aff-wild: Valence and arousal ‘in-the-wild’challenge}, author={Zafeiriou, Stefanos and Kollias, Dimitrios and Nicolaou, Mihalis A and Papaioannou, Athanasios and Zhao, Guoying and Kotsia, Irene}, booktitle={Computer Vision and Pattern Recognition Workshops (CVPRW), 2017 IEEE Conference on}, pages={1980--1987}, year={2017}, organization={IEEE} }

 

  • D. Kollias, et. al. "Recognition of affect in the wild using deep neural networks", CVPRW, 2017

@inproceedings{kollias2017recognition, title={Recognition of affect in the wild using deep neural networks}, author={Kollias, Dimitrios and Nicolaou, Mihalis A and Kotsia, Irene and Zhao, Guoying and Zafeiriou, Stefanos}, booktitle={Computer Vision and Pattern Recognition Workshops (CVPRW), 2017 IEEE Conference on}, pages={1972--1979}, year={2017}, organization={IEEE} }

 

 

Regarding the database:

 

• The database and annotations are available for non-commercial research purposes only.

• All the training/validation/testing images of the dataset have been obtained from Youtube. We are not responsible for the content nor the meaning of these images.

• Participants will agree not to reproduce, duplicate, copy, sell, trade, resell or exploit for any commercial purposes, any portion of the images and any portion of derived data. They will also agree not to further copy, publish or distribute any portion of annotations of the dataset. Except, for internal use at a single site within the same organization it is allowed to make copies of the dataset.

• We reserve the right to terminate participants’ access to the dataset at any time.

• If a participant’s face is displayed in any video and (s)he wants it to be removed, (s)he can email us at any time

 

 

Important Dates: 

  • Call for participation announced, team registration begins, data available:       
18 November 2019
  • Final submission deadline (Results, Code and ArXiv paper):
  4 February, 2020
  • Winners Announcement:      

  5 February, 2020

  • Final paper submission deadline:                       
 

14 February, 2020

  • Review decisions sent to authors; Notification of acceptance:                     
 

   21 February, 2020

  • Camera ready version deadline:                                                                   
 

   28 February, 2020

 

 

Sumbission Information

More details will follow. To be announced

 

 

Program Committee

More details will follow. To be announced

 

Sponsors:

The Affective Behavior Analysis in-the-wild Challenge has been generously supported by:
Imperial College London
and
Realeyes - Emotional Intelligence