Dr. Jie Shen
Research fellow
Biography
Jie Shen obtained his BEng degree in Information Engineering from Zhejiang University in 2005. He then worked as a research assistant in National Laboratory of Pattern Recognition, Institute of Automation, ChineseAcademy of Sciences until 2007. He received his MSc degree in Advanced Computing and his PhD degree from Department of Computing, Imperial College London in 2008 and 2014, respectively. During his postgraduate study, Jie Shen mainly worked on the software and hardware infrastructure for multi-modal human-computer/robot interaction (HCI/HRI) systems. His work, the HCI^2 Framework, has been used as the system integration framework in EU FP7 FROG and EU FP7 TERESA projects and he has built the data collection system for the MAHNOB-HCI-Tagging database and the SEWA database, Jie Shen’s current research focuses on affect-sensitive human-robot interaction and cloud-based affective computing systems. A demo from his recent work is shown below.
HCI:
Integration platform for development of human-centered interfaces,
Affect-sensitive HCI
Publications
Journal articles
Dilated Convolutions with Lateral Inhibitions for Semantic Image Segmentation
Y. Wang, M. Dong, J. Shen, Y. Lin, M. Pantic. arXiv preprint arXiv:2006.03708. 2020.
author = {Y. Wang and M. Dong and J. Shen and Y. Lin and M. Pantic},
journal = {arXiv preprint arXiv:2006.03708},
title = {Dilated Convolutions with Lateral Inhibitions for Semantic Image Segmentation},
year = {2020},
}
Face Mask Extraction in Video Sequence
Y. Wang, B. Luo, J. Shen, M. Pantic. International Journal of Computer Vision (IJCV). 127: pp. 625 - 641, 2019.
author = {Y. Wang and B. Luo and J. Shen and M. Pantic},
pages = {625--641},
journal = {International Journal of Computer Vision (IJCV)},
publisher = {Springer US},
title = {Face Mask Extraction in Video Sequence},
volume = {127},
year = {2019},
}
The Menpo Benchmark for Multi-pose 2D and 3D Facial Landmark Localisation and Tracking
J. Deng, A. Roussos, G. Chrysos, E. Ververas, I. Kotsia, J. Shen, S. Zafeiriou. IJCV. 2018.
author = {J. Deng and A. Roussos and G. Chrysos and E. Ververas and I. Kotsia and J. Shen and S. Zafeiriou},
journal = {IJCV},
publisher = {IJCV},
title = {The Menpo Benchmark for Multi-pose 2D and 3D Facial Landmark Localisation and Tracking},
year = {2018},
}
A real-time and unsupervised face Re-Identification system for Human-Robot Interaction
Y. Wang, J. Shen, S. Petridis, M. Pantic. Pattern Recognition Letters (Accepted). 2018.
author = {M. Pantic and S. Petridis and J. Shen and Y. Wang},
journal = {Pattern Recognition Letters (Accepted)},
title = {A real-time and unsupervised face Re-Identification system for Human-Robot Interaction},
year = {2018},
}
Blind Audio-Visual Localization and Separation via Low-Rank and Sparsity
J. Pu, Y. Panagakis, S. Petridis, J. Shen, M. Pantic. IEEE transactions on cybernetics. 2018.
author = {J. Pu and Y. Panagakis and S. Petridis and J. Shen and M. Pantic},
journal = {IEEE transactions on cybernetics},
publisher = {IEEE},
title = {Blind Audio-Visual Localization and Separation via Low-Rank and Sparsity},
year = {2018},
}
HCI^2 Framework: A Software Framework for Multimodal Human-Computer Interaction
J. Shen, M. Pantic. IEEE Transactions on Systems, Man, and Cybernetics–Part B. Special issue on Modern Control for Computer Games (Accepted for publication) 2013.
author = {J. Shen and M. Pantic},
journal = {IEEE Transactions on Systems, Man, and Cybernetics–Part B},
month = {Special issue on Modern Control for Computer Games (Accepted for publication)},
title = {HCI^2 Framework: A Software Framework for Multimodal Human-Computer Interaction},
year = {2013},
}
%T HCI^2 Framework: A Software Framework for Multimodal Human-Computer Interaction
%A Shen, J.
%A Pantic, M.
%J IEEE Transactions on Systems, Man, and Cybernetics–Part B
%D 2013
%8 Special issue on Modern Control for Computer Games (Accepted for publication)
%F SMCB-E-2012-04-0349-R3
Cost-effective solution to synchronised audio-visual data capture using multiple sensors
J. Lichtenauer, J. Shen, M. F. Valstar, M. Pantic. Image and Vision Computing. 29: pp. 666 - 680, September 2011.
author = {J. Lichtenauer and J. Shen and M. F. Valstar and M. Pantic},
pages = {666--680},
journal = {Image and Vision Computing},
month = {September},
publisher = {Elsevier},
title = {Cost-effective solution to synchronised audio-visual data capture using multiple sensors},
url = {http://dx.doi.org/10.1016/j.imavis.2011.07.004},
volume = {29},
year = {2011},
}
%T Cost-effective solution to synchronised audio-visual data capture using multiple sensors
%A Lichtenauer, J.
%A Shen, J.
%A Valstar, M. F.
%A Pantic, M.
%J Image and Vision Computing
%D 2011
%8 September
%V 29
%I Elsevier
%F Lichtenauer2011a
%U http://dx.doi.org/10.1016/j.imavis.2011.07.004
%P 666-680
Conference articles
Lip-reading with Densely Connected Temporal Convolutional Networks
P. Ma, Y. Wang, J. Shen, S. Petridis, M. Pantic. The IEEE Winter Conference on Applications of Computer Vision (WACV). 2021.
author = {P. Ma and Y. Wang and J. Shen and S. Petridis and M. Pantic},
booktitle = {The IEEE Winter Conference on Applications of Computer Vision (WACV)},
title = {Lip-reading with Densely Connected Temporal Convolutional Networks},
year = {2021},
}
Dynamic Face Video Segmentation via Reinforcement Learning
Y. Wang, M. Dong, J. Shen, Y. Wu, S. Cheng, M. Pantic. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 6959 - 6969, 2020.
author = {Y. Wang and M. Dong and J. Shen and Y. Wu and S. Cheng and M. Pantic},
pages = {6959--6969},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
title = {Dynamic Face Video Segmentation via Reinforcement Learning},
year = {2020},
}
Shape constrained network for eye segmentation in the wild
B. Luo, J. Shen, S. Cheng, Y. Wang, M. Pantic. The IEEE Winter Conference on Applications of Computer Vision (WACV). pp. 1952 - 1960, 2020.
author = {B. Luo and J. Shen and S. Cheng and Y. Wang and M. Pantic},
pages = {1952--1960},
booktitle = {The IEEE Winter Conference on Applications of Computer Vision (WACV)},
title = {Shape constrained network for eye segmentation in the wild},
year = {2020},
}
FT-RCNN: Real-time Visual Face Tracking with Region-based Convolutional Neural Networks
Y. Lin, J. Shen, S. Cheng, M. Pantic. FG' 2020 (Oral). 2020.
author = {Y. Lin and J. Shen and S. Cheng and M. Pantic},
booktitle = {FG' 2020 (Oral)},
title = {FT-RCNN: Real-time Visual Face Tracking with Region-based Convolutional Neural Networks},
year = {2020},
}
The ibug eye segmentation dataset
B. Luo, J. Shen, Y. Wang, M. Pantic. 2018 Imperial College Computing Student Workshop (ICCSW 2018). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2019.
author = {B. Luo and J. Shen and Y. Wang and M. Pantic},
booktitle = {2018 Imperial College Computing Student Workshop (ICCSW 2018)},
organization = {Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik},
title = {The ibug eye segmentation dataset},
year = {2019},
}
MobiFace: A Novel Dataset for Mobile Face Tracking in the Wild
Y. Lin, S. Cheng, J. Shen, M. Pantic. FG'2019 (Oral). May 2019.
author = {Y. Lin and S. Cheng and J. Shen and M. Pantic},
booktitle = {FG'2019 (Oral)},
month = {May},
title = {MobiFace: A Novel Dataset for Mobile Face Tracking in the Wild},
year = {2019},
}
Visual-Only Recognition of Normal, Whispered and Silent Speech
S. Petridis, J. Shen, D. Cetin, M. Pantic. Accepted to ICASSP . April 2018.
author = {S. Petridis and J. Shen and D. Cetin and M. Pantic},
booktitle = {Accepted to ICASSP },
journal = {Accepted to ICASSP},
month = {April},
title = {Visual-Only Recognition of Normal, Whispered and Silent Speech},
year = {2018},
}
PCSsampler: Sample-based, Private-state Cluster Scheduling
C. Hao, J. Shen, C. Chen, H. Zhang, Y. Wu, M. Li. Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. IEEE Press, pp. 599 - 608, 2017.
author = {C. Hao and J. Shen and C. Chen and H. Zhang and Y. Wu and M. Li},
pages = {599--608},
booktitle = {Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing},
organization = {IEEE Press},
title = {PCSsampler: Sample-based, Private-state Cluster Scheduling},
year = {2017},
}
Enhancing sample-based scheduler with collaborate-state in big data cluster.
C. Hao, C. Chen, J. Shen, M. Li, B. W. Boehm. SEKE. pp. 477 - 480, 2017.
author = {C. Hao and C. Chen and J. Shen and M. Li and B. W. Boehm},
pages = {477--480},
booktitle = {SEKE},
title = {Enhancing sample-based scheduler with collaborate-state in big data cluster.},
year = {2017},
}
Tiresias: low-overhead sample based scheduling with task hopping
C. Hao, J. Shen, H. Zhang, Y. Wu, M. Li. Proceedings of the 2016 IEEE International Conference on Cluster Computing. IEEE, 2016.
author = {C. Hao and J. Shen and H. Zhang and Y. Wu and M. Li},
booktitle = {Proceedings of the 2016 IEEE International Conference on Cluster Computing},
organization = {IEEE},
title = {Tiresias: low-overhead sample based scheduling with task hopping},
year = {2016},
}
The First Facial Landmark Tracking in-the-Wild Challenge: Benchmark and Results
J. Shen, S. Zafeiriou, G. Chrysos, J. Kossaifi, G. Tzimiropoulos, M. Pantic. Proceedings of IEEE International Conference on Computer Vision, 300 Videos in the Wild (300-VW): Facial Landmark Tracking in-the-Wild Challenge & Workshop (ICCVW'15). December 2015.
author = {J. Shen and S. Zafeiriou and G. Chrysos and J. Kossaifi and G. Tzimiropoulos and M. Pantic},
booktitle = {Proceedings of IEEE International Conference on Computer Vision, 300 Videos in the Wild (300-VW): Facial Landmark Tracking in-the-Wild Challenge & Workshop (ICCVW'15)},
month = {December},
title = {The First Facial Landmark Tracking in-the-Wild Challenge: Benchmark and Results},
year = {2015},
}
Sparkle: adaptive sample based scheduling for cluster computing
C. Hao, J. Shen, H. Zhang, X. Zhang, Y. Wu, M. Li. Proceedings of the 5th International Workshop on Cloud Data and Platforms. ACM, 2015.
author = {C. Hao and J. Shen and H. Zhang and X. Zhang and Y. Wu and M. Li},
booktitle = {Proceedings of the 5th International Workshop on Cloud Data and Platforms},
organization = {ACM},
title = {Sparkle: adaptive sample based scheduling for cluster computing},
year = {2015},
}
Sentiment Apprehension in Human-Robot Interaction with NAO
J. Shen, O. Rudovic, S. Cheng, M. Pantic. 1st International Workshop on Automatic Sentiment Analysis in the Wild (WASA'15). Xi'an, China, September 2015.
author = {J. Shen and O. Rudovic and S. Cheng and M. Pantic},
address = {Xi'an, China},
booktitle = {1st International Workshop on Automatic Sentiment Analysis in the Wild (WASA'15)},
month = {September},
title = {Sentiment Apprehension in Human-Robot Interaction with NAO},
year = {2015},
}
TERESA: A Socially Intelligent Semi-autonomous Telepresence System
K. Shiarlis, J. Messias, M. V. Someren, S. Whiteson, J. Kim, J. Vroon, G. Englebienne, K. Truong, V. Evers, N. Perez-higueras, I. Perez-hurtado, R. Ramon-vigo, F. Caballero, L. Merino, J. Shen, S. Petridis, M. Pantic, L. Hedman, M. Scherlund, R. Koster, H. Michel. ICRA 2015: Proceedings of the IEEE International Conference on Robotics and Automation, Workshop on Machine Learning for Social Robotics. May 2015.
author = {K. Shiarlis and J. Messias and M. V. Someren and S. Whiteson and J. Kim and J. Vroon and G. Englebienne and K. Truong and V. Evers and N. Perez-higueras and I. Perez-hurtado and R. Ramon-vigo and F. Caballero and L. Merino and J. Shen and S. Petridis and M. Pantic and L. Hedman and M. Scherlund and R. Koster and H. Michel},
booktitle = {ICRA 2015: Proceedings of the IEEE International Conference on Robotics and Automation, Workshop on Machine Learning for Social Robotics},
month = {May},
title = {TERESA: A Socially Intelligent Semi-autonomous Telepresence System},
year = {2015},
}
Real-time generic face tracking in the wild with CUDA
S. Cheng, A. Asthana, S. Zafeiriou, J. Shen, M. Pantic. Proceedings of the 5th ACM Multimedia Systems Conference. ACM, Singapore, Singapore, pp. 148 - 151, 2014.
author = {S. Cheng and A. Asthana and S. Zafeiriou and J. Shen and M. Pantic},
pages = {148--151},
address = {Singapore, Singapore},
booktitle = {Proceedings of the 5th ACM Multimedia Systems Conference},
publisher = {ACM},
title = {Real-time generic face tracking in the wild with CUDA},
year = {2014},
}
%T Real-time generic face tracking in the wild with CUDA
%A Cheng, S.
%A Asthana, A.
%A Zafeiriou, S.
%A Shen, J.
%A Pantic, M.
%B Proceedings of the 5th ACM Multimedia Systems Conference
%D 2014
%I ACM
%C Singapore, Singapore
%F Cheng2014
%P 148-151
CLM-WILD: Real-Time Generic Face Tracker
A. Asthana, S. Cheng, J. Shen, S. Zafeiriou, M. Pantic. Demo at IEEE International Conference on Computer Vision (ICCV 2013), Sydney, Australia. December 2013.
author = {A. Asthana and S. Cheng and J. Shen and S. Zafeiriou and M. Pantic},
booktitle = {Demo at IEEE International Conference on Computer Vision (ICCV 2013), Sydney, Australia},
month = {December},
title = {CLM-WILD: Real-Time Generic Face Tracker},
year = {2013},
}
%T CLM-WILD: Real-Time Generic Face Tracker
%A Asthana, A.
%A Cheng, S.
%A Shen, J.
%A Zafeiriou, S.
%A Pantic, M.
%B Demo at IEEE International Conference on Computer Vision (ICCV 2013), Sydney, Australia
%D 2013
%8 December
%F iccv13_demo
HCI^2 Workbench: A Development Tool for Multimodal Human-Computer Interaction Systems
J. Shen, W. Shi, M. Pantic. Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG'11), ECAG'11- 2nd FG Workshop on Facial and Bodily Expressions for Control and Adaptation of Games . Santa Barbara, CA, USA, pp. 766 - 773, March 2011.
author = {J. Shen and W. Shi and M. Pantic},
pages = {766--773},
address = {Santa Barbara, CA, USA},
booktitle = {Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG'11), ECAG'11- 2nd FG Workshop on Facial and Bodily Expressions for Control and Adaptation of Games },
month = {March},
title = {HCI^2 Workbench: A Development Tool for Multimodal Human-Computer Interaction Systems},
year = {2011},
}
%T HCI^2 Workbench: A Development Tool for Multimodal Human-Computer Interaction Systems
%A Shen, J.
%A Shi, W.
%A Pantic, M.
%B Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG?11), ECAG?11- 2nd FG Workshop on Facial and Bodily Expressions for Control and Adaptation of Games
%D 2011
%8 March
%C Santa Barbara, CA, USA
%F shen2011hci
%P 766-773
A Software Framework for Multimodal Human-Computer Interaction Systems
J. Shen, M. Pantic. Proceedings of IEEE Int'l Conf. Systems, Man, and Cybernetics (SMC'09). San Antonio, USA, pp. 2038 - 2045, October 2009.
author = {J. Shen and M. Pantic},
pages = {2038--2045},
address = {San Antonio, USA},
booktitle = {Proceedings of IEEE Int'l Conf. Systems, Man, and Cybernetics (SMC'09)},
month = {October},
title = {A Software Framework for Multimodal Human-Computer Interaction Systems},
year = {2009},
}
%T A Software Framework for Multimodal Human-Computer Interaction Systems
%A Shen, J.
%A Pantic, M.
%B Proceedings of IEEE Int?l Conf. Systems, Man, and Cybernetics (SMC?09)
%D 2009
%8 October
%C San Antonio, USA
%F Shen2009asffm
%P 2038-2045
Cost-effective solution to synchronized audio-visual capture using multiple sensors
J. Lichtenauer, M. F. Valstar, J. Shen, M. Pantic. Proceedings of IEEE Int'l Conf. Advanced Video and Signal Based Surveillance (AVSS'09). Genoa, Italy, pp. 324 - 329, September 2009.
author = {J. Lichtenauer and M. F. Valstar and J. Shen and M. Pantic},
pages = {324--329},
address = {Genoa, Italy},
booktitle = {Proceedings of IEEE Int'l Conf. Advanced Video and Signal Based Surveillance (AVSS'09)},
month = {September},
title = {Cost-effective solution to synchronized audio-visual capture using multiple sensors},
year = {2009},
}
%T Cost-effective solution to synchronized audio-visual capture using multiple sensors
%A Lichtenauer, J.
%A Valstar, M. F.
%A Shen, J.
%A Pantic, M.
%B Proceedings of IEEE Int?l Conf. Advanced Video and Signal Based Surveillance (AVSS?09)
%D 2009
%8 September
%C Genoa, Italy
%F Lichtenauer2009cstsa
%P 324-329