Towards Face Representation Learning Conditioned on the Soft Biometrics

In this paper, we present a method to leverage soft biometric as a means of conditioning biometrics for better face representation learning. By conditioning, we meant the soft biometric trait (age, gender, etc.) is used as an auxiliary biometric for training along with face modality while it is absent during the inference stage. We propose a two-stream deep neural network consisting of a multilayer perceptron network (MLP) and a convolutional neural network (CNN), which can learn a feature representation from soft biometric vectors and face images, respectively. The two-stream network can be optimized simultaneously and the information can be exploited from both biometrics. The learned conditioning soft biometric representation from the MLP serves as a center prototype of the feature learned from the face network, which is beneficial to contract the intra-class variation of the face feature representation. Due to the lacking of the face dataset that comes along with soft biometrics, we construct a database for evaluation purposes. Extensive experiments are performed on two face datasets that equip with soft biometrics and the results show the superiority of our method compared to the face modality alone.