Deep joint image filtering

Yijun Li; Jia Bin Huang; Narendra Ahuja; Ming Hsuan Yang

doi:10.1007/978-3-319-46493-0_10

Deep joint image filtering

Yijun Li, Jia Bin Huang, Narendra Ahuja, Ming Hsuan Yang

College of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

117 Citations (Scopus)

Abstract

Joint image filters can leverage the guidance image as a prior and transfer the structural details from the guidance image to the target image for suppressing noise or enhancing spatial resolution. Existing methods rely on various kinds of explicit filter construction or handdesigned objective functions. It is thus difficult to understand, improve, and accelerate them in a coherent framework. In this paper, we propose a learning-based approach to construct a joint filter based on Convolutional Neural Networks. In contrast to existing methods that consider only the guidance image, our method can selectively transfer salient structures that are consistent in both guidance and target images. We show that the model trained on a certain type of data, e.g., RGB and depth images, generalizes well for other modalities, e.g., Flash/Non-Flash and RGB/NIR images. We validate the effectiveness of the proposed joint filter through extensive comparisons with state-of-the-art methods.

Original language	English
Title of host publication	Computer Vision - 14th European Conference, ECCV 2016, Proceedings
Editors	Bastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
Publisher	Springer Verlag
Pages	154-169
Number of pages	16
ISBN (Print)	9783319464923
DOIs	https://doi.org/10.1007/978-3-319-46493-0_10
Publication status	Published - 2016
Event	14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands Duration: 2016 Oct 8 → 2016 Oct 16

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9908 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	14th European Conference on Computer Vision, ECCV 2016
Country/Territory	Netherlands
City	Amsterdam
Period	16/10/8 → 16/10/16

Bibliographical note

Funding Information:
This work is supported in part by the NSF CAREER Grant #1149783, gifts from Adobe and Nvidia, and Office of Naval Research under grant N00014-16-1-2314.

Publisher Copyright:
© Springer International Publishing AG 2016.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-319-46493-0_10

Cite this

Li, Y., Huang, J. B., Ahuja, N., & Yang, M. H. (2016). Deep joint image filtering. In B. Leibe, J. Matas, N. Sebe, & M. Welling (Eds.), Computer Vision - 14th European Conference, ECCV 2016, Proceedings (pp. 154-169). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9908 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46493-0_10

Li, Yijun ; Huang, Jia Bin ; Ahuja, Narendra et al. / Deep joint image filtering. Computer Vision - 14th European Conference, ECCV 2016, Proceedings. editor / Bastian Leibe ; Jiri Matas ; Nicu Sebe ; Max Welling. Springer Verlag, 2016. pp. 154-169 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{9f7028ef531d48a98c08d1c23cf03ed2,

title = "Deep joint image filtering",

abstract = "Joint image filters can leverage the guidance image as a prior and transfer the structural details from the guidance image to the target image for suppressing noise or enhancing spatial resolution. Existing methods rely on various kinds of explicit filter construction or handdesigned objective functions. It is thus difficult to understand, improve, and accelerate them in a coherent framework. In this paper, we propose a learning-based approach to construct a joint filter based on Convolutional Neural Networks. In contrast to existing methods that consider only the guidance image, our method can selectively transfer salient structures that are consistent in both guidance and target images. We show that the model trained on a certain type of data, e.g., RGB and depth images, generalizes well for other modalities, e.g., Flash/Non-Flash and RGB/NIR images. We validate the effectiveness of the proposed joint filter through extensive comparisons with state-of-the-art methods.",

author = "Yijun Li and Huang, {Jia Bin} and Narendra Ahuja and Yang, {Ming Hsuan}",

note = "Funding Information: This work is supported in part by the NSF CAREER Grant #1149783, gifts from Adobe and Nvidia, and Office of Naval Research under grant N00014-16-1-2314. Publisher Copyright: {\textcopyright} Springer International Publishing AG 2016.; 14th European Conference on Computer Vision, ECCV 2016 ; Conference date: 08-10-2016 Through 16-10-2016",

year = "2016",

doi = "10.1007/978-3-319-46493-0_10",

language = "English",

isbn = "9783319464923",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "154--169",

editor = "Bastian Leibe and Jiri Matas and Nicu Sebe and Max Welling",

booktitle = "Computer Vision - 14th European Conference, ECCV 2016, Proceedings",

address = "Germany",

}

Li, Y, Huang, JB, Ahuja, N & Yang, MH 2016, Deep joint image filtering. in B Leibe, J Matas, N Sebe & M Welling (eds), Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9908 LNCS, Springer Verlag, pp. 154-169, 14th European Conference on Computer Vision, ECCV 2016, Amsterdam, Netherlands, 16/10/8. https://doi.org/10.1007/978-3-319-46493-0_10

Deep joint image filtering. / Li, Yijun; Huang, Jia Bin; Ahuja, Narendra et al.
Computer Vision - 14th European Conference, ECCV 2016, Proceedings. ed. / Bastian Leibe; Jiri Matas; Nicu Sebe; Max Welling. Springer Verlag, 2016. p. 154-169 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9908 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Deep joint image filtering

AU - Li, Yijun

AU - Huang, Jia Bin

AU - Ahuja, Narendra

AU - Yang, Ming Hsuan

N1 - Funding Information: This work is supported in part by the NSF CAREER Grant #1149783, gifts from Adobe and Nvidia, and Office of Naval Research under grant N00014-16-1-2314. Publisher Copyright: © Springer International Publishing AG 2016.

PY - 2016

Y1 - 2016

N2 - Joint image filters can leverage the guidance image as a prior and transfer the structural details from the guidance image to the target image for suppressing noise or enhancing spatial resolution. Existing methods rely on various kinds of explicit filter construction or handdesigned objective functions. It is thus difficult to understand, improve, and accelerate them in a coherent framework. In this paper, we propose a learning-based approach to construct a joint filter based on Convolutional Neural Networks. In contrast to existing methods that consider only the guidance image, our method can selectively transfer salient structures that are consistent in both guidance and target images. We show that the model trained on a certain type of data, e.g., RGB and depth images, generalizes well for other modalities, e.g., Flash/Non-Flash and RGB/NIR images. We validate the effectiveness of the proposed joint filter through extensive comparisons with state-of-the-art methods.

AB - Joint image filters can leverage the guidance image as a prior and transfer the structural details from the guidance image to the target image for suppressing noise or enhancing spatial resolution. Existing methods rely on various kinds of explicit filter construction or handdesigned objective functions. It is thus difficult to understand, improve, and accelerate them in a coherent framework. In this paper, we propose a learning-based approach to construct a joint filter based on Convolutional Neural Networks. In contrast to existing methods that consider only the guidance image, our method can selectively transfer salient structures that are consistent in both guidance and target images. We show that the model trained on a certain type of data, e.g., RGB and depth images, generalizes well for other modalities, e.g., Flash/Non-Flash and RGB/NIR images. We validate the effectiveness of the proposed joint filter through extensive comparisons with state-of-the-art methods.

UR - http://www.scopus.com/inward/record.url?scp=84990053231&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84990053231&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46493-0_10

DO - 10.1007/978-3-319-46493-0_10

M3 - Conference contribution

AN - SCOPUS:84990053231

SN - 9783319464923

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 154

EP - 169

BT - Computer Vision - 14th European Conference, ECCV 2016, Proceedings

A2 - Leibe, Bastian

A2 - Matas, Jiri

A2 - Sebe, Nicu

A2 - Welling, Max

PB - Springer Verlag

T2 - 14th European Conference on Computer Vision, ECCV 2016

Y2 - 8 October 2016 through 16 October 2016

ER -

Li Y, Huang JB, Ahuja N, Yang MH. Deep joint image filtering. In Leibe B, Matas J, Sebe N, Welling M, editors, Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Springer Verlag. 2016. p. 154-169. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-46493-0_10

Deep joint image filtering

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this