Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks

Seongjong Song; Hyunjung Shim

doi:10.1007/978-3-030-20873-8_41

Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks

Seongjong Song, Hyunjung Shim

School of Integrated Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

4 Citations (Scopus)

Abstract

We propose a novel approach to recovering translucent objects from a single time-of-flight (ToF) depth camera using deep residual networks. When recording translucent objects using the ToF depth camera, their depth values are severely contaminated due to complex light interactions with surrounding environment. While existing methods suggested new capture systems or developed the depth distortion models, their solutions were less practical because of strict assumptions or heavy computational complexity. In this paper, we adopt deep residual networks for modeling the ToF depth distortion caused by translucency. To fully utilize both the local and semantic information of objects, multi-scale patches are used to predict the depth value. Based on the quantitative and qualitative evaluation on our benchmark database, we show the effectiveness and robustness of the proposed algorithm.

Original language	English
Title of host publication	Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers
Editors	Hongdong Li, Konrad Schindler, C.V. Jawahar, Greg Mori
Publisher	Springer Verlag
Pages	641-657
Number of pages	17
ISBN (Print)	9783030208721
DOIs	https://doi.org/10.1007/978-3-030-20873-8_41
Publication status	Published - 2019
Event	14th Asian Conference on Computer Vision, ACCV 2018 - Perth, Australia Duration: 2018 Dec 2 → 2018 Dec 6

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11365 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	14th Asian Conference on Computer Vision, ACCV 2018
Country/Territory	Australia
City	Perth
Period	18/12/2 → 18/12/6

Bibliographical note

Funding Information:
Acknowledgement. This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the University Information Technology Research Center support program (IITP-2016-R2718-16-0014) supervised by the IITP, by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the MSIP (NRF-2016R1A2B4016236), and also by the MIST(Ministry of Science and ICT), Korea, under the “ICT Consilience Creative Program” (IITP-2018-2017-0-01015) supervised by the IITP(Institute for Information & communications Technology Promotion).

Funding Information:
This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the University Information Technology Research Center support program (IITP-2016-R2718-16-0014) supervised by the IITP, by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the MSIP (NRF-2016R1A2B4016236), and also by the MIST(Ministry of Science and ICT), Korea, under the “ICT Consilience Creative Program” (IITP-2018-2017-0-01015) supervised by the IITP(Institute for Information & communications Technology Promotion).

Publisher Copyright:
© Springer Nature Switzerland AG 2019.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-030-20873-8_41

Cite this

Song, S., & Shim, H. (2019). Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks. In H. Li, K. Schindler, C. V. Jawahar, & G. Mori (Eds.), Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers (pp. 641-657). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11365 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-20873-8_41

Song, Seongjong ; Shim, Hyunjung. / Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks. Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. editor / Hongdong Li ; Konrad Schindler ; C.V. Jawahar ; Greg Mori. Springer Verlag, 2019. pp. 641-657 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{9830ecf71f284c23b1995f7782f133b7,

title = "Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks",

abstract = "We propose a novel approach to recovering translucent objects from a single time-of-flight (ToF) depth camera using deep residual networks. When recording translucent objects using the ToF depth camera, their depth values are severely contaminated due to complex light interactions with surrounding environment. While existing methods suggested new capture systems or developed the depth distortion models, their solutions were less practical because of strict assumptions or heavy computational complexity. In this paper, we adopt deep residual networks for modeling the ToF depth distortion caused by translucency. To fully utilize both the local and semantic information of objects, multi-scale patches are used to predict the depth value. Based on the quantitative and qualitative evaluation on our benchmark database, we show the effectiveness and robustness of the proposed algorithm.",

author = "Seongjong Song and Hyunjung Shim",

note = "Funding Information: Acknowledgement. This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the University Information Technology Research Center support program (IITP-2016-R2718-16-0014) supervised by the IITP, by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the MSIP (NRF-2016R1A2B4016236), and also by the MIST(Ministry of Science and ICT), Korea, under the “ICT Consilience Creative Program” (IITP-2018-2017-0-01015) supervised by the IITP(Institute for Information & communications Technology Promotion). Funding Information: This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the University Information Technology Research Center support program (IITP-2016-R2718-16-0014) supervised by the IITP, by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the MSIP (NRF-2016R1A2B4016236), and also by the MIST(Ministry of Science and ICT), Korea, under the “ICT Consilience Creative Program” (IITP-2018-2017-0-01015) supervised by the IITP(Institute for Information & communications Technology Promotion). Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2019.; 14th Asian Conference on Computer Vision, ACCV 2018 ; Conference date: 02-12-2018 Through 06-12-2018",

year = "2019",

doi = "10.1007/978-3-030-20873-8_41",

language = "English",

isbn = "9783030208721",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "641--657",

editor = "Hongdong Li and Konrad Schindler and C.V. Jawahar and Greg Mori",

booktitle = "Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers",

address = "Germany",

}

Song, S & Shim, H 2019, Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks. in H Li, K Schindler, CV Jawahar & G Mori (eds), Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11365 LNCS, Springer Verlag, pp. 641-657, 14th Asian Conference on Computer Vision, ACCV 2018, Perth, Australia, 18/12/2. https://doi.org/10.1007/978-3-030-20873-8_41

Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks. / Song, Seongjong; Shim, Hyunjung.
Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. ed. / Hongdong Li; Konrad Schindler; C.V. Jawahar; Greg Mori. Springer Verlag, 2019. p. 641-657 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11365 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks

AU - Song, Seongjong

AU - Shim, Hyunjung

N1 - Funding Information: Acknowledgement. This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the University Information Technology Research Center support program (IITP-2016-R2718-16-0014) supervised by the IITP, by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the MSIP (NRF-2016R1A2B4016236), and also by the MIST(Ministry of Science and ICT), Korea, under the “ICT Consilience Creative Program” (IITP-2018-2017-0-01015) supervised by the IITP(Institute for Information & communications Technology Promotion). Funding Information: This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the University Information Technology Research Center support program (IITP-2016-R2718-16-0014) supervised by the IITP, by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the MSIP (NRF-2016R1A2B4016236), and also by the MIST(Ministry of Science and ICT), Korea, under the “ICT Consilience Creative Program” (IITP-2018-2017-0-01015) supervised by the IITP(Institute for Information & communications Technology Promotion). Publisher Copyright: © Springer Nature Switzerland AG 2019.

PY - 2019

Y1 - 2019

N2 - We propose a novel approach to recovering translucent objects from a single time-of-flight (ToF) depth camera using deep residual networks. When recording translucent objects using the ToF depth camera, their depth values are severely contaminated due to complex light interactions with surrounding environment. While existing methods suggested new capture systems or developed the depth distortion models, their solutions were less practical because of strict assumptions or heavy computational complexity. In this paper, we adopt deep residual networks for modeling the ToF depth distortion caused by translucency. To fully utilize both the local and semantic information of objects, multi-scale patches are used to predict the depth value. Based on the quantitative and qualitative evaluation on our benchmark database, we show the effectiveness and robustness of the proposed algorithm.

AB - We propose a novel approach to recovering translucent objects from a single time-of-flight (ToF) depth camera using deep residual networks. When recording translucent objects using the ToF depth camera, their depth values are severely contaminated due to complex light interactions with surrounding environment. While existing methods suggested new capture systems or developed the depth distortion models, their solutions were less practical because of strict assumptions or heavy computational complexity. In this paper, we adopt deep residual networks for modeling the ToF depth distortion caused by translucency. To fully utilize both the local and semantic information of objects, multi-scale patches are used to predict the depth value. Based on the quantitative and qualitative evaluation on our benchmark database, we show the effectiveness and robustness of the proposed algorithm.

UR - http://www.scopus.com/inward/record.url?scp=85066789281&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066789281&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-20873-8_41

DO - 10.1007/978-3-030-20873-8_41

M3 - Conference contribution

AN - SCOPUS:85066789281

SN - 9783030208721

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 641

EP - 657

BT - Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers

A2 - Li, Hongdong

A2 - Schindler, Konrad

A2 - Jawahar, C.V.

A2 - Mori, Greg

PB - Springer Verlag

T2 - 14th Asian Conference on Computer Vision, ACCV 2018

Y2 - 2 December 2018 through 6 December 2018

ER -

Song S, Shim H. Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks. In Li H, Schindler K, Jawahar CV, Mori G, editors, Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. Springer Verlag. 2019. p. 641-657. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-20873-8_41

Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this