Multi-frame super-resolution utilizing spatially adaptive regularization for ToF camera

Recently, 3D time-of-flight cameras have been developed. The development enables utilization of depth images in various fields. However, acquired depth images are corrupted by noise during the image acquisition process and have relatively lower resolution than RGB images due to the limitation of ToF cameras. In this paper, a multi-frame super-resolution reconstruction algorithm is proposed for ToF depth images to overcome such limits. The purpose of the multi-frame super-resolution reconstruction is to reconstruct a high-resolution image from observed multiple low-resolution images through the sequential process of subpixel estimation and restoration. A conventional regularized super-resolution reconstruction algorithm which takes Tikhonov regularization has a major drawback of over-smoothing around edges. To overcome the disadvantage, the spatially adaptive regularization is suggested for preservation of edges. Experimental results show that the image reconstructed by the proposed super-resolution reconstruction algorithm contains significantly higher resolution with less amount of noise and sharper edges than the observed data.