Learning Deeply Aggregated Alternating Minimization for General Inverse Problems

Hyungjoo Jung, Youngjung Kim, Dongbo Min, Hyunsung Jang, Namkoo Ha, Kwanghoon Sohn

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)


Regularization-based image restoration is one of the most powerful tools in image processing and computer vision thanks to its flexibility for handling various inverse problems. However, designing an optimal regularization function still remains unsolved since natural images and related scene types have a complex structure. In this paper, we present a general and principled framework, called deeply aggregated alternating minimization (DeepAM). We design a convolutional neural network (CNN) to implicitly parameterize the regularizer of the alternating minimization (AM) algorithm. Contrary to the conventional AM algorithm based on a point-wise proximal mapping, the DeepAM projects intermediate estimate into a set of natural images via deep aggregation. Since the CNN is fully integrated into the AM procedure, all parameters can be jointly optimized through end-to-end training. These properties enable the DeepAM to converge with a small number of iterations, while maintaining an algorithmic simplicity. We show that the DeepAM outperforms state-of-the-art methods, including nonlocal-based methods, Plug-and-Play regularization, and recent data-driven approaches. The effectiveness of our framework is demonstrated in a variety of image restoration tasks: Guassian denoising, deraining, deblurring, super-resolution, color-guided depth upsampling, and RGB/NIR restoration.

Original languageEnglish
Article number9146780
Pages (from-to)8012-8027
Number of pages16
JournalIEEE Transactions on Image Processing
Publication statusPublished - 2020

Bibliographical note

Publisher Copyright:
© 1992-2012 IEEE.

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Learning Deeply Aggregated Alternating Minimization for General Inverse Problems'. Together they form a unique fingerprint.

Cite this