Single image shadow removal via detection-free spatially adaptive denormalization

Removing shadows in a single image has been a challenging problem because shadows can appear in various forms due to complex physical situations, influenced by many factors such as light sources and the material's transparency. In order to remove shadows precisely, most previous works utilized shadow mask information, which indicates the shadow region in a given image using binary representation. However, shadow mask utilization inevitably induces multiple problems, including shadow removal performance dependency and additional shadow detection process requirements. To solve these problems, the proposed algorithm is based on an image-to-image translation algorithm, which does not require additional shadow mask information. In this deep neural network, the convergence of fast learning is induced by utilizing various normalization layers. However, in a case that is very sensitive to various spatial features of an input image, such as shadow removal, the normalization process causes a problem of losing a large amount of information existing in the input image data. So, we utilize spatially adaptive denormalization(SPADE) to prevent loss of spatial features of input image data. Therefore, not only does it fundamentally solve the problem that various feature information constituting the input image is lost in the normalization process, but also enables precise shadow region removal by combining the feature map of multi-resolutions with the feature map of the decoder. In evaluation, the proposed algorithm shows that it exceeds the existing approach by about 20∼30% in both PSNR and RMSE based on the ISTD large data set.