Multimodal Face-Pose Estimation With Multitask Manifold Deep Learning

Chaoqun Hong, Jun Yu, Jian Zhang, Xiongnan Jin, Kyong Ho Lee

Research output: Contribution to journalArticlepeer-review

203 Citations (Scopus)


Face-pose estimation aims at estimating the gazing direction with two-dimensional face images. It gives important communicative information and visual saliency. However, it is challenging because of lights, background, face orientations, and appearance visibility. Therefore, a descriptive representation of face images and mapping it to poses are critical. In this paper, we use multimodal data and propose a novel face-pose estimation framework named multitask manifold deep learning (M2DL). It is based on feature extraction with improved convolutional neural networks (CNNs) and multimodal mapping relationship with multitask learning. In the proposed CNNs, manifold regularized convolutional layers learn the relationship between outputs of neurons in a low-rank space. Besides, in the proposed mapping relationship learning method, different modals of face representations are naturally combined by applying multitask learning with incoherent sparse and low-rank learning with a least-squares loss. Experimental results on three challenging benchmark datasets demonstrate the performance of M2DL.

Original languageEnglish
Article number8554134
Pages (from-to)3952-3961
Number of pages10
JournalIEEE Transactions on Industrial Informatics
Issue number7
Publication statusPublished - 2019 Jul

Bibliographical note

Funding Information:
Manuscript received July 19, 2018; revised November 6, 2018; accepted November 25, 2018. Date of publication November 30, 2018; date of current version July 3, 2019.The work was supported in part by the National Natural Science Foundation of China under Grant 61622205 and under Grant 61836002, in part by the Zhejiang Provincial Natural Science Foundation of China under Grant LY17F020009, in part by the Fujian Provincial Natural Science Foundation of China under Grant 2018J01573, in part by the Fujian Provincial High School Natural Science Foundation of China under Grant JZ160472, and in part by the Foundation of Fujian Educational Committee under Grant JAT160357. Paper no. TII-18-1874. (Corresponding author: Jun Yu.) C. Hong is with the College of Computer and Information Engineering, Xiamen University of Technology, Xiamen 361024, China (e-mail:,

Publisher Copyright:
© 2018 IEEE.

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Information Systems
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Multimodal Face-Pose Estimation With Multitask Manifold Deep Learning'. Together they form a unique fingerprint.

Cite this