Low-Light Image Enhancement via a Deep Hybrid Network

Wenqi Ren; Sifei Liu; Lin Ma; Qianqian Xu; Xiangyu Xu; Xiaochun Cao; Junping Du; Ming Hsuan Yang

doi:10.1109/TIP.2019.2910412

Low-Light Image Enhancement via a Deep Hybrid Network

Wenqi Ren, Sifei Liu, Lin Ma, Qianqian Xu, Xiangyu Xu, Xiaochun Cao, Junping Du, Ming Hsuan Yang

College of Computing

Research output: Contribution to journal › Article › peer-review

231 Citations (Scopus)

Abstract

Camera sensors often fail to capture clear images or videos in a poorly lit environment. In this paper, we propose a trainable hybrid network to enhance the visibility of such degraded images. The proposed network consists of two distinct streams to simultaneously learn the global content and the salient structures of the clear image in a unified network. More specifically, the content stream estimates the global content of the low-light input through an encoder-decoder network. However, the encoder in the content stream tends to lose some structure details. To remedy this, we propose a novel spatially variant recurrent neural network (RNN) as an edge stream to model edge details, with the guidance of another auto-encoder. The experimental results show that the proposed network favorably performs against the state-of-the-art low-light image enhancement algorithms.

Original language	English
Article number	8692732
Pages (from-to)	4364-4375
Number of pages	12
Journal	IEEE Transactions on Image Processing
Volume	28
Issue number	9
DOIs	https://doi.org/10.1109/TIP.2019.2910412
Publication status	Published - 2019 Sept

Bibliographical note

Funding Information:
Manuscript received August 19, 2018; revised February 8, 2019; accepted March 28, 2019. Date of publication April 16, 2019; date of current version July 1, 2019. This work was supported in part by the National Natural Science Foundation of China under Grant U1736219, Grant U1605252, Grant U1803264, Grant 61532006, Grant 61772083, and Grant 61802403, in part by the National Key R&D Program of China under Grant 2018YFB0803701, in part by the Beijing Natural Science Foundation under Grant L182057, and in part by the CCF-Tencent Open Fund. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Jana Ehmann. (Corresponding author: Xiaochun Cao.) W. Ren and X. Cao are with the State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100093, China (e-mail: rwq.renwenqi@gmail.com; caoxiaochun@iie.ac.cn).

Publisher Copyright:
© 1992-2012 IEEE.

All Science Journal Classification (ASJC) codes

Software
Computer Graphics and Computer-Aided Design

Access to Document

10.1109/TIP.2019.2910412

Cite this

@article{cef9b678a4a94c4aafa30efcf018c5c2,

title = "Low-Light Image Enhancement via a Deep Hybrid Network",

abstract = "Camera sensors often fail to capture clear images or videos in a poorly lit environment. In this paper, we propose a trainable hybrid network to enhance the visibility of such degraded images. The proposed network consists of two distinct streams to simultaneously learn the global content and the salient structures of the clear image in a unified network. More specifically, the content stream estimates the global content of the low-light input through an encoder-decoder network. However, the encoder in the content stream tends to lose some structure details. To remedy this, we propose a novel spatially variant recurrent neural network (RNN) as an edge stream to model edge details, with the guidance of another auto-encoder. The experimental results show that the proposed network favorably performs against the state-of-the-art low-light image enhancement algorithms.",

author = "Wenqi Ren and Sifei Liu and Lin Ma and Qianqian Xu and Xiangyu Xu and Xiaochun Cao and Junping Du and Yang, {Ming Hsuan}",

note = "Funding Information: Manuscript received August 19, 2018; revised February 8, 2019; accepted March 28, 2019. Date of publication April 16, 2019; date of current version July 1, 2019. This work was supported in part by the National Natural Science Foundation of China under Grant U1736219, Grant U1605252, Grant U1803264, Grant 61532006, Grant 61772083, and Grant 61802403, in part by the National Key R&D Program of China under Grant 2018YFB0803701, in part by the Beijing Natural Science Foundation under Grant L182057, and in part by the CCF-Tencent Open Fund. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Jana Ehmann. (Corresponding author: Xiaochun Cao.) W. Ren and X. Cao are with the State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100093, China (e-mail: rwq.renwenqi@gmail.com; caoxiaochun@iie.ac.cn). Publisher Copyright: {\textcopyright} 1992-2012 IEEE.",

year = "2019",

month = sep,

doi = "10.1109/TIP.2019.2910412",

language = "English",

volume = "28",

pages = "4364--4375",

journal = "IEEE Transactions on Image Processing",

issn = "1057-7149",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "9",

}

TY - JOUR

T1 - Low-Light Image Enhancement via a Deep Hybrid Network

AU - Ren, Wenqi

AU - Liu, Sifei

AU - Ma, Lin

AU - Xu, Qianqian

AU - Xu, Xiangyu

AU - Cao, Xiaochun

AU - Du, Junping

AU - Yang, Ming Hsuan

N1 - Funding Information: Manuscript received August 19, 2018; revised February 8, 2019; accepted March 28, 2019. Date of publication April 16, 2019; date of current version July 1, 2019. This work was supported in part by the National Natural Science Foundation of China under Grant U1736219, Grant U1605252, Grant U1803264, Grant 61532006, Grant 61772083, and Grant 61802403, in part by the National Key R&D Program of China under Grant 2018YFB0803701, in part by the Beijing Natural Science Foundation under Grant L182057, and in part by the CCF-Tencent Open Fund. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Jana Ehmann. (Corresponding author: Xiaochun Cao.) W. Ren and X. Cao are with the State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100093, China (e-mail: rwq.renwenqi@gmail.com; caoxiaochun@iie.ac.cn). Publisher Copyright: © 1992-2012 IEEE.

PY - 2019/9

Y1 - 2019/9

N2 - Camera sensors often fail to capture clear images or videos in a poorly lit environment. In this paper, we propose a trainable hybrid network to enhance the visibility of such degraded images. The proposed network consists of two distinct streams to simultaneously learn the global content and the salient structures of the clear image in a unified network. More specifically, the content stream estimates the global content of the low-light input through an encoder-decoder network. However, the encoder in the content stream tends to lose some structure details. To remedy this, we propose a novel spatially variant recurrent neural network (RNN) as an edge stream to model edge details, with the guidance of another auto-encoder. The experimental results show that the proposed network favorably performs against the state-of-the-art low-light image enhancement algorithms.

AB - Camera sensors often fail to capture clear images or videos in a poorly lit environment. In this paper, we propose a trainable hybrid network to enhance the visibility of such degraded images. The proposed network consists of two distinct streams to simultaneously learn the global content and the salient structures of the clear image in a unified network. More specifically, the content stream estimates the global content of the low-light input through an encoder-decoder network. However, the encoder in the content stream tends to lose some structure details. To remedy this, we propose a novel spatially variant recurrent neural network (RNN) as an edge stream to model edge details, with the guidance of another auto-encoder. The experimental results show that the proposed network favorably performs against the state-of-the-art low-light image enhancement algorithms.

UR - http://www.scopus.com/inward/record.url?scp=85068391906&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068391906&partnerID=8YFLogxK

U2 - 10.1109/TIP.2019.2910412

DO - 10.1109/TIP.2019.2910412

M3 - Article

C2 - 30998467

AN - SCOPUS:85068391906

SN - 1057-7149

VL - 28

SP - 4364

EP - 4375

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

IS - 9

M1 - 8692732

ER -

Low-Light Image Enhancement via a Deep Hybrid Network

Abstract

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this