Hybrid neural network with cost-sensitive support vector machine for class-imbalanced multimodal data

Although deep learning exhibits advantages in various applications involving multimodal data, it cannot effectively solve the class-imbalance problem. Herein, we propose a hybrid neural network with a cost-sensitive support vector machine (hybrid NN-CSSVM) for class-imbalanced multimodal data. We used a fused multiple-network structure obtained by extracting the features of different modality data, and used cost-sensitive support vector machines (SVMs) as a classifier. To alleviate the insufficiency of learning from minority-class data, our proposed cost-sensitive SVM loss function reflects different weights of misclassification errors from both majority and minority classes, by controlling cost parameters. Additionally, we present a theoretical setting of the cost parameters in our model. The proposed model is validated on real datasets that range from low to high imbalance ratios. By exploiting the complementary advantages of two architectures, the hybrid NN-CSSVM performs excellently, even with data having a minor-class proportion of only 2%.