Jie Shen

Dr. Jie Shen

Position:

Research fellow

Email:

Biography

Jie Shen obtained his BEng degree in Information Engineering from Zhejiang University in 2005. He then worked as a research assistant in National Laboratory of Pattern Recognition, Institute of Automation, ChineseAcademy of Sciences until 2007. He received his MSc degree in Advanced Computing and his PhD degree from Department of Computing, Imperial College London in 2008 and 2014, respectively. During his postgraduate study, Jie Shen mainly worked on the software and hardware infrastructure for multi-modal human-computer/robot interaction (HCI/HRI) systems. His work, the HCI^2 Framework, has been used as the system integration framework in EU FP7 FROG and EU FP7 TERESA projects and he has built the data collection system for the MAHNOB-HCI-Tagging database and the SEWA database, Jie Shen’s current research focuses on affect-sensitive human-robot interaction and cloud-based affective computing systems. A demo from his recent work is shown below. 

Research Themes:

HCI:
Integration platform for development of human-centered interfaces, Affect-sensitive HCI

Publications

Journal articles

Dilated Convolutions with Lateral Inhibitions for Semantic Image Segmentation

Y. Wang, M. Dong, J. Shen, Y. Lin, M. Pantic. arXiv preprint arXiv:2006.03708. 2020.

Bibtex reference [hide]
@article{wang2020dilated,
    author = {Y. Wang and M. Dong and J. Shen and Y. Lin and M. Pantic},
    journal = {arXiv preprint arXiv:2006.03708},
    title = {Dilated Convolutions with Lateral Inhibitions for Semantic Image Segmentation},
    year = {2020},
}
Endnote reference [hide]

Face Mask Extraction in Video Sequence

Y. Wang, B. Luo, J. Shen, M. Pantic. International Journal of Computer Vision (IJCV). 127: pp. 625 - 641, 2019.

Bibtex reference [hide]
@article{wang2019face,
    author = {Y. Wang and B. Luo and J. Shen and M. Pantic},
    pages = {625--641},
    journal = {International Journal of Computer Vision (IJCV)},
    publisher = {Springer US},
    title = {Face Mask Extraction in Video Sequence},
    volume = {127},
    year = {2019},
}
Endnote reference [hide]

The Menpo Benchmark for Multi-pose 2D and 3D Facial Landmark Localisation and Tracking

J. Deng, A. Roussos, G. Chrysos, E. Ververas, I. Kotsia, J. Shen, S. Zafeiriou. IJCV. 2018.

Bibtex reference [hide]
@article{ijcvmenpo,
    author = {J. Deng and A. Roussos and G. Chrysos and E. Ververas and I. Kotsia and J. Shen and S. Zafeiriou},
    journal = {IJCV},
    publisher = {IJCV},
    title = {The Menpo Benchmark for Multi-pose 2D and 3D Facial Landmark Localisation and Tracking},
    year = {2018},
}
Endnote reference [hide]

A real-time and unsupervised face Re-Identification system for Human-Robot Interaction

Y. Wang, J. Shen, S. Petridis, M. Pantic. Pattern Recognition Letters (Accepted). 2018.

Bibtex reference [hide]
@article{FaceReID,
    author = {M. Pantic and S. Petridis and J. Shen and Y. Wang},
    journal = {Pattern Recognition Letters (Accepted)},
    title = {A real-time and unsupervised face Re-Identification system for Human-Robot Interaction},
    year = {2018},
}
Endnote reference [hide]

Blind Audio-Visual Localization and Separation via Low-Rank and Sparsity

J. Pu, Y. Panagakis, S. Petridis, J. Shen, M. Pantic. IEEE transactions on cybernetics. 2018.

Bibtex reference [hide]
@article{pu2018blind,
    author = {J. Pu and Y. Panagakis and S. Petridis and J. Shen and M. Pantic},
    journal = {IEEE transactions on cybernetics},
    publisher = {IEEE},
    title = {Blind Audio-Visual Localization and Separation via Low-Rank and Sparsity},
    year = {2018},
}
Endnote reference [hide]

HCI^2 Framework: A Software Framework for Multimodal Human-Computer Interaction

J. Shen, M. Pantic. IEEE Transactions on Systems, Man, and Cybernetics–Part B. Special issue on Modern Control for Computer Games (Accepted for publication) 2013.

Bibtex reference [hide]
@article{SMCB-E-2012-04-0349-R3,
    author = {J. Shen and M. Pantic},
    journal = {IEEE Transactions on Systems, Man, and Cybernetics–Part B},
    month = {Special issue on Modern Control for Computer Games (Accepted for publication)},
    title = {HCI^2 Framework: A Software Framework for Multimodal Human-Computer Interaction},
    year = {2013},
}
Endnote reference [hide]
%0 Journal Article
%T HCI^2 Framework: A Software Framework for Multimodal Human-Computer Interaction
%A Shen, J.
%A Pantic, M.
%J IEEE Transactions on Systems, Man, and Cybernetics–Part B
%D 2013
%8 Special issue on Modern Control for Computer Games (Accepted for publication)
%F SMCB-E-2012-04-0349-R3

Cost-effective solution to synchronised audio-visual data capture using multiple sensors

J. Lichtenauer, J. Shen, M. F. Valstar, M. Pantic. Image and Vision Computing. 29: pp. 666 - 680, September 2011.

Bibtex reference [hide]
@article{Lichtenauer2011a,
    author = {J. Lichtenauer and J. Shen and M. F. Valstar and M. Pantic},
    pages = {666--680},
    journal = {Image and Vision Computing},
    month = {September},
    publisher = {Elsevier},
    title = {Cost-effective solution to synchronised audio-visual data capture using multiple sensors},
    url = {http://dx.doi.org/10.1016/j.imavis.2011.07.004},
    volume = {29},
    year = {2011},
}
Endnote reference [hide]
%0 Journal Article
%T Cost-effective solution to synchronised audio-visual data capture using multiple sensors
%A Lichtenauer, J.
%A Shen, J.
%A Valstar, M. F.
%A Pantic, M.
%J Image and Vision Computing
%D 2011
%8 September
%V 29
%I Elsevier
%F Lichtenauer2011a
%U http://dx.doi.org/10.1016/j.imavis.2011.07.004
%P 666-680

Conference articles

Lip-reading with Densely Connected Temporal Convolutional Networks

P. Ma, Y. Wang, J. Shen, S. Petridis, M. Pantic. The IEEE Winter Conference on Applications of Computer Vision (WACV). 2021.

Bibtex reference [hide]
@inproceedings{ma2020lip,
    author = {P. Ma and Y. Wang and J. Shen and S. Petridis and M. Pantic},
    booktitle = {The IEEE Winter Conference on Applications of Computer Vision (WACV)},
    title = {Lip-reading with Densely Connected Temporal Convolutional Networks},
    year = {2021},
}
Endnote reference [hide]

Dynamic Face Video Segmentation via Reinforcement Learning

Y. Wang, M. Dong, J. Shen, Y. Wu, S. Cheng, M. Pantic. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 6959 - 6969, 2020.

Bibtex reference [hide]
@inproceedings{wang2020dynamic,
    author = {Y. Wang and M. Dong and J. Shen and Y. Wu and S. Cheng and M. Pantic},
    pages = {6959--6969},
    booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    title = {Dynamic Face Video Segmentation via Reinforcement Learning},
    year = {2020},
}
Endnote reference [hide]

Shape constrained network for eye segmentation in the wild

B. Luo, J. Shen, S. Cheng, Y. Wang, M. Pantic. The IEEE Winter Conference on Applications of Computer Vision (WACV). pp. 1952 - 1960, 2020.

Bibtex reference [hide]
@inproceedings{luo2020shape,
    author = {B. Luo and J. Shen and S. Cheng and Y. Wang and M. Pantic},
    pages = {1952--1960},
    booktitle = {The IEEE Winter Conference on Applications of Computer Vision (WACV)},
    title = {Shape constrained network for eye segmentation in the wild},
    year = {2020},
}
Endnote reference [hide]

FT-RCNN: Real-time Visual Face Tracking with Region-based Convolutional Neural Networks

Y. Lin, J. Shen, S. Cheng, M. Pantic. FG' 2020 (Oral). 2020.

Bibtex reference [hide]
@inproceedings{FT-RCNN,
    author = {Y. Lin and J. Shen and S. Cheng and M. Pantic},
    booktitle = {FG' 2020 (Oral)},
    title = {FT-RCNN: Real-time Visual Face Tracking with Region-based Convolutional Neural Networks},
    year = {2020},
}
Endnote reference [hide]

The ibug eye segmentation dataset

B. Luo, J. Shen, Y. Wang, M. Pantic. 2018 Imperial College Computing Student Workshop (ICCSW 2018). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2019.

Bibtex reference [hide]
@inproceedings{luo2019ibug,
    author = {B. Luo and J. Shen and Y. Wang and M. Pantic},
    booktitle = {2018 Imperial College Computing Student Workshop (ICCSW 2018)},
    organization = {Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik},
    title = {The ibug eye segmentation dataset},
    year = {2019},
}
Endnote reference [hide]

MobiFace: A Novel Dataset for Mobile Face Tracking in the Wild

Y. Lin, S. Cheng, J. Shen, M. Pantic. FG'2019 (Oral). May 2019.

Bibtex reference [hide]
@inproceedings{mobiface_fg2019,
    author = {Y. Lin and S. Cheng and J. Shen and M. Pantic},
    booktitle = {FG'2019 (Oral)},
    month = {May},
    title = {MobiFace: A Novel Dataset for Mobile Face Tracking in the Wild},
    year = {2019},
}
Endnote reference [hide]

Visual-Only Recognition of Normal, Whispered and Silent Speech

S. Petridis, J. Shen, D. Cetin, M. Pantic. Accepted to ICASSP . April 2018.

Bibtex reference [hide]
@inproceedings{normalWhisperSilentDB,
    author = {S. Petridis and J. Shen and D. Cetin and M. Pantic},
    booktitle = {Accepted to ICASSP },
    journal = {Accepted to ICASSP},
    month = {April},
    title = {Visual-Only Recognition of Normal, Whispered and Silent Speech},
    year = {2018},
}
Endnote reference [hide]

PCSsampler: Sample-based, Private-state Cluster Scheduling

C. Hao, J. Shen, C. Chen, H. Zhang, Y. Wu, M. Li. Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. IEEE Press, pp. 599 - 608, 2017.

Bibtex reference [hide]
@inproceedings{hao2017pcssampler,
    author = {C. Hao and J. Shen and C. Chen and H. Zhang and Y. Wu and M. Li},
    pages = {599--608},
    booktitle = {Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing},
    organization = {IEEE Press},
    title = {PCSsampler: Sample-based, Private-state Cluster Scheduling},
    year = {2017},
}
Endnote reference [hide]

Enhancing sample-based scheduler with collaborate-state in big data cluster.

C. Hao, C. Chen, J. Shen, M. Li, B. W. Boehm. SEKE. pp. 477 - 480, 2017.

Bibtex reference [hide]
@inproceedings{hao2017enhancing,
    author = {C. Hao and C. Chen and J. Shen and M. Li and B. W. Boehm},
    pages = {477--480},
    booktitle = {SEKE},
    title = {Enhancing sample-based scheduler with collaborate-state in big data cluster.},
    year = {2017},
}
Endnote reference [hide]

Tiresias: low-overhead sample based scheduling with task hopping

C. Hao, J. Shen, H. Zhang, Y. Wu, M. Li. Proceedings of the 2016 IEEE International Conference on Cluster Computing. IEEE, 2016.

Bibtex reference [hide]
@inproceedings{hao2016tiresias,
    author = {C. Hao and J. Shen and H. Zhang and Y. Wu and M. Li},
    booktitle = {Proceedings of the 2016 IEEE International Conference on Cluster Computing},
    organization = {IEEE},
    title = {Tiresias: low-overhead sample based scheduling with task hopping},
    year = {2016},
}
Endnote reference [hide]

The First Facial Landmark Tracking in-the-Wild Challenge: Benchmark and Results

J. Shen, S. Zafeiriou, G. Chrysos, J. Kossaifi, G. Tzimiropoulos, M. Pantic. Proceedings of IEEE International Conference on Computer Vision, 300 Videos in the Wild (300-VW): Facial Landmark Tracking in-the-Wild Challenge & Workshop (ICCVW'15). December 2015.

Bibtex reference [hide]
@inproceedings{300VW,
    author = {J. Shen and S. Zafeiriou and G. Chrysos and J. Kossaifi and G. Tzimiropoulos and M. Pantic},
    booktitle = {Proceedings of IEEE International Conference on Computer Vision, 300 Videos in the Wild (300-VW): Facial Landmark Tracking in-the-Wild Challenge & Workshop (ICCVW'15)},
    month = {December},
    title = {The First Facial Landmark Tracking in-the-Wild Challenge: Benchmark and Results},
    year = {2015},
}
Endnote reference [hide]

Sparkle: adaptive sample based scheduling for cluster computing

C. Hao, J. Shen, H. Zhang, X. Zhang, Y. Wu, M. Li. Proceedings of the 5th International Workshop on Cloud Data and Platforms. ACM, 2015.

Bibtex reference [hide]
@inproceedings{hao2015sparkle,
    author = {C. Hao and J. Shen and H. Zhang and X. Zhang and Y. Wu and M. Li},
    booktitle = {Proceedings of the 5th International Workshop on Cloud Data and Platforms},
    organization = {ACM},
    title = {Sparkle: adaptive sample based scheduling for cluster computing},
    year = {2015},
}
Endnote reference [hide]

Sentiment Apprehension in Human-Robot Interaction with NAO

J. Shen, O. Rudovic, S. Cheng, M. Pantic. 1st International Workshop on Automatic Sentiment Analysis in the Wild (WASA'15). Xi'an, China, September 2015.

Bibtex reference [hide]
@inproceedings{Shen2015,
    author = {J. Shen and O. Rudovic and S. Cheng and M. Pantic},
    address = {Xi'an, China},
    booktitle = {1st International Workshop on Automatic Sentiment Analysis in the Wild (WASA'15)},
    month = {September},
    title = {Sentiment Apprehension in Human-Robot Interaction with NAO},
    year = {2015},
}
Endnote reference [hide]

TERESA: A Socially Intelligent Semi-autonomous Telepresence System

K. Shiarlis, J. Messias, M. V. Someren, S. Whiteson, J. Kim, J. Vroon, G. Englebienne, K. Truong, V. Evers, N. Perez-higueras, I. Perez-hurtado, R. Ramon-vigo, F. Caballero, L. Merino, J. Shen, S. Petridis, M. Pantic, L. Hedman, M. Scherlund, R. Koster, H. Michel. ICRA 2015: Proceedings of the IEEE International Conference on Robotics and Automation, Workshop on Machine Learning for Social Robotics. May 2015.

Bibtex reference [hide]
@inproceedings{shiarlisicra15,
    author = {K. Shiarlis and J. Messias and M. V. Someren and S. Whiteson and J. Kim and J. Vroon and G. Englebienne and K. Truong and V. Evers and N. Perez-higueras and I. Perez-hurtado and R. Ramon-vigo and F. Caballero and L. Merino and J. Shen and S. Petridis and M. Pantic and L. Hedman and M. Scherlund and R. Koster and H. Michel},
    booktitle = {ICRA 2015: Proceedings of the IEEE International Conference on Robotics and Automation, Workshop on Machine Learning for Social Robotics},
    month = {May},
    title = {TERESA: A Socially Intelligent Semi-autonomous Telepresence System},
    year = {2015},
}
Endnote reference [hide]

Real-time generic face tracking in the wild with CUDA

S. Cheng, A. Asthana, S. Zafeiriou, J. Shen, M. Pantic. Proceedings of the 5th ACM Multimedia Systems Conference. ACM, Singapore, Singapore, pp. 148 - 151, 2014.

Bibtex reference [hide]
@inproceedings{Cheng2014,
    author = {S. Cheng and A. Asthana and S. Zafeiriou and J. Shen and M. Pantic},
    pages = {148--151},
    address = {Singapore, Singapore},
    booktitle = {Proceedings of the 5th ACM Multimedia Systems Conference},
    publisher = {ACM},
    title = {Real-time generic face tracking in the wild with CUDA},
    year = {2014},
}
Endnote reference [hide]
%0 Conference Proceedings
%T Real-time generic face tracking in the wild with CUDA
%A Cheng, S.
%A Asthana, A.
%A Zafeiriou, S.
%A Shen, J.
%A Pantic, M.
%B Proceedings of the 5th ACM Multimedia Systems Conference
%D 2014
%I ACM
%C Singapore, Singapore
%F Cheng2014
%P 148-151

CLM-WILD: Real-Time Generic Face Tracker

A. Asthana, S. Cheng, J. Shen, S. Zafeiriou, M. Pantic. Demo at IEEE International Conference on Computer Vision (ICCV 2013), Sydney, Australia. December 2013.

Bibtex reference [hide]
@inproceedings{iccv13_demo,
    author = {A. Asthana and S. Cheng and J. Shen and S. Zafeiriou and M. Pantic},
    booktitle = {Demo at IEEE International Conference on Computer Vision (ICCV 2013), Sydney, Australia},
    month = {December},
    title = {CLM-WILD: Real-Time Generic Face Tracker},
    year = {2013},
}
Endnote reference [hide]
%0 Conference Proceedings
%T CLM-WILD: Real-Time Generic Face Tracker
%A Asthana, A.
%A Cheng, S.
%A Shen, J.
%A Zafeiriou, S.
%A Pantic, M.
%B Demo at IEEE International Conference on Computer Vision (ICCV 2013), Sydney, Australia
%D 2013
%8 December
%F iccv13_demo

HCI^2 Workbench: A Development Tool for Multimodal Human-Computer Interaction Systems

J. Shen, W. Shi, M. Pantic. Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG'11), ECAG'11- 2nd FG Workshop on Facial and Bodily Expressions for Control and Adaptation of Games . Santa Barbara, CA, USA, pp. 766 - 773, March 2011.

Bibtex reference [hide]
@inproceedings{shen2011hci,
    author = {J. Shen and W. Shi and M. Pantic},
    pages = {766--773},
    address = {Santa Barbara, CA, USA},
    booktitle = {Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG'11), ECAG'11- 2nd FG Workshop on Facial and Bodily Expressions for Control and Adaptation of Games },
    month = {March},
    title = {HCI^2 Workbench: A Development Tool for Multimodal Human-Computer Interaction Systems},
    year = {2011},
}
Endnote reference [hide]
%0 Conference Proceedings
%T HCI^2 Workbench: A Development Tool for Multimodal Human-Computer Interaction Systems
%A Shen, J.
%A Shi, W.
%A Pantic, M.
%B Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG?11), ECAG?11- 2nd FG Workshop on Facial and Bodily Expressions for Control and Adaptation of Games
%D 2011
%8 March
%C Santa Barbara, CA, USA
%F shen2011hci
%P 766-773

A Software Framework for Multimodal Human-Computer Interaction Systems

J. Shen, M. Pantic. Proceedings of IEEE Int'l Conf. Systems, Man, and Cybernetics (SMC'09). San Antonio, USA, pp. 2038 - 2045, October 2009.

Bibtex reference [hide]
@inproceedings{Shen2009asffm,
    author = {J. Shen and M. Pantic},
    pages = {2038--2045},
    address = {San Antonio, USA},
    booktitle = {Proceedings of IEEE Int'l Conf. Systems, Man, and Cybernetics (SMC'09)},
    month = {October},
    title = {A Software Framework for Multimodal Human-Computer Interaction Systems},
    year = {2009},
}
Endnote reference [hide]
%0 Conference Proceedings
%T A Software Framework for Multimodal Human-Computer Interaction Systems
%A Shen, J.
%A Pantic, M.
%B Proceedings of IEEE Int?l Conf. Systems, Man, and Cybernetics (SMC?09)
%D 2009
%8 October
%C San Antonio, USA
%F Shen2009asffm
%P 2038-2045

Cost-effective solution to synchronized audio-visual capture using multiple sensors

J. Lichtenauer, M. F. Valstar, J. Shen, M. Pantic. Proceedings of IEEE Int'l Conf. Advanced Video and Signal Based Surveillance (AVSS'09). Genoa, Italy, pp. 324 - 329, September 2009.

Bibtex reference [hide]
@inproceedings{Lichtenauer2009cstsa,
    author = {J. Lichtenauer and M. F. Valstar and J. Shen and M. Pantic},
    pages = {324--329},
    address = {Genoa, Italy},
    booktitle = {Proceedings of IEEE Int'l Conf. Advanced Video and Signal Based Surveillance (AVSS'09)},
    month = {September},
    title = {Cost-effective solution to synchronized audio-visual capture using multiple sensors},
    year = {2009},
}
Endnote reference [hide]
%0 Conference Proceedings
%T Cost-effective solution to synchronized audio-visual capture using multiple sensors
%A Lichtenauer, J.
%A Valstar, M. F.
%A Shen, J.
%A Pantic, M.
%B Proceedings of IEEE Int?l Conf. Advanced Video and Signal Based Surveillance (AVSS?09)
%D 2009
%8 September
%C Genoa, Italy
%F Lichtenauer2009cstsa
%P 324-329