Brightness-based convolutional neural network for thermal image enhancement

In this paper, we propose a convolutional neural network for thermal image enhancement by incorporating the brightness domain with a residual-learning technique, which improves the performance of enhancement and speed of convergence. Typically, the training domain uses the same domain as that of the target image; however, we evaluated several domains to determine the most suitable one for the network. In the analyses, we first compared the performance of networks that were trained by the corresponding regions of color-based and aligned infrared-based images, respectively, including thermal, far, and near spectra. Then, four RGB-based domains, namely, gray, lightness, intensity, and brightness were evaluated. Finally, the proposed network architecture was determined by considering the residual and brightness domains. The results of the analyses indicated that the brightness domain was the best training domain for enhancing the thermal images. The experimental results confirm that the proposed network, which can be trained in approximately one hour, outperforms the conventional learning-based approaches for thermal image enhancement, in terms of several image quality metrics and a qualitative evaluation. Furthermore, the results demonstrate that the brightness domain is effective as the training domain and can be used to increase the performance of existing networks.