Unseen Object Segmentation in Videos via Transferable Representations

Yi Wen Chen; Yi Hsuan Tsai; Chu Ya Yang; Yen Yu Lin; Ming Hsuan Yang

doi:10.1007/978-3-030-20870-7_38

Unseen Object Segmentation in Videos via Transferable Representations

Yi Wen Chen, Yi Hsuan Tsai, Chu Ya Yang, Yen Yu Lin, Ming Hsuan Yang

College of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

Abstract

In order to learn object segmentation models in videos, conventional methods require a large amount of pixel-wise ground truth annotations. However, collecting such supervised data is time-consuming and labor-intensive. In this paper, we exploit existing annotations in source images and transfer such visual information to segment videos with unseen object categories. Without using any annotations in the target video, we propose a method to jointly mine useful segments and learn feature representations that better adapt to the target frames. The entire process is decomposed into two tasks: (1) solving a submodular function for selecting object-like segments, and (2) learning a CNN model with a transferable module for adapting seen categories in the source domainÂ to the unseen target video. We present an iterative update scheme between two tasks to self-learn the final solution for object segmentation. Experimental results on numerous benchmark datasets show that the proposed method performs favorably against the state-of-the-art algorithms.

Original language	English
Title of host publication	Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers
Editors	Hongdong Li, C.V. Jawahar, Greg Mori, Konrad Schindler
Publisher	Springer Verlag
Pages	615-631
Number of pages	17
ISBN (Print)	9783030208691
DOIs	https://doi.org/10.1007/978-3-030-20870-7_38
Publication status	Published - 2019
Event	14th Asian Conference on Computer Vision, ACCV 2018 - Perth, Australia Duration: 2018 Dec 2 → 2018 Dec 6

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11364 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	14th Asian Conference on Computer Vision, ACCV 2018
Country/Territory	Australia
City	Perth
Period	18/12/2 → 18/12/6

Bibliographical note

Funding Information:
Acknowledgments. This work is supported in part by Ministry of Science and Technology under grants MOST 105-2221-E-001-030-MY2 and MOST 107-2628-E-001-005-MY3.

Funding Information:
This work is supported in part by Ministry of Science and Technology under grants MOST 105-2221-E-001-030-MY2 and MOST 107-2628-E-001-005-MY3.

Publisher Copyright:
© 2019, Springer Nature Switzerland AG.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-030-20870-7_38

Cite this

Chen, Y. W., Tsai, Y. H., Yang, C. Y., Lin, Y. Y., & Yang, M. H. (2019). Unseen Object Segmentation in Videos via Transferable Representations. In H. Li, C. V. Jawahar, G. Mori, & K. Schindler (Eds.), Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers (pp. 615-631). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11364 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-20870-7_38

Chen, Yi Wen ; Tsai, Yi Hsuan ; Yang, Chu Ya et al. / Unseen Object Segmentation in Videos via Transferable Representations. Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. editor / Hongdong Li ; C.V. Jawahar ; Greg Mori ; Konrad Schindler. Springer Verlag, 2019. pp. 615-631 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{aedc16e1874d46abb328f7a7def050b9,

title = "Unseen Object Segmentation in Videos via Transferable Representations",

abstract = "In order to learn object segmentation models in videos, conventional methods require a large amount of pixel-wise ground truth annotations. However, collecting such supervised data is time-consuming and labor-intensive. In this paper, we exploit existing annotations in source images and transfer such visual information to segment videos with unseen object categories. Without using any annotations in the target video, we propose a method to jointly mine useful segments and learn feature representations that better adapt to the target frames. The entire process is decomposed into two tasks: (1) solving a submodular function for selecting object-like segments, and (2) learning a CNN model with a transferable module for adapting seen categories in the source domain{\^A} to the unseen target video. We present an iterative update scheme between two tasks to self-learn the final solution for object segmentation. Experimental results on numerous benchmark datasets show that the proposed method performs favorably against the state-of-the-art algorithms.",

author = "Chen, {Yi Wen} and Tsai, {Yi Hsuan} and Yang, {Chu Ya} and Lin, {Yen Yu} and Yang, {Ming Hsuan}",

note = "Funding Information: Acknowledgments. This work is supported in part by Ministry of Science and Technology under grants MOST 105-2221-E-001-030-MY2 and MOST 107-2628-E-001-005-MY3. Funding Information: This work is supported in part by Ministry of Science and Technology under grants MOST 105-2221-E-001-030-MY2 and MOST 107-2628-E-001-005-MY3. Publisher Copyright: {\textcopyright} 2019, Springer Nature Switzerland AG.; 14th Asian Conference on Computer Vision, ACCV 2018 ; Conference date: 02-12-2018 Through 06-12-2018",

year = "2019",

doi = "10.1007/978-3-030-20870-7_38",

language = "English",

isbn = "9783030208691",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "615--631",

editor = "Hongdong Li and C.V. Jawahar and Greg Mori and Konrad Schindler",

booktitle = "Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers",

address = "Germany",

}

Chen, YW, Tsai, YH, Yang, CY, Lin, YY & Yang, MH 2019, Unseen Object Segmentation in Videos via Transferable Representations. in H Li, CV Jawahar, G Mori & K Schindler (eds), Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11364 LNCS, Springer Verlag, pp. 615-631, 14th Asian Conference on Computer Vision, ACCV 2018, Perth, Australia, 18/12/2. https://doi.org/10.1007/978-3-030-20870-7_38

Unseen Object Segmentation in Videos via Transferable Representations. / Chen, Yi Wen; Tsai, Yi Hsuan; Yang, Chu Ya et al.
Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. ed. / Hongdong Li; C.V. Jawahar; Greg Mori; Konrad Schindler. Springer Verlag, 2019. p. 615-631 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11364 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Unseen Object Segmentation in Videos via Transferable Representations

AU - Chen, Yi Wen

AU - Tsai, Yi Hsuan

AU - Yang, Chu Ya

AU - Lin, Yen Yu

AU - Yang, Ming Hsuan

N1 - Funding Information: Acknowledgments. This work is supported in part by Ministry of Science and Technology under grants MOST 105-2221-E-001-030-MY2 and MOST 107-2628-E-001-005-MY3. Funding Information: This work is supported in part by Ministry of Science and Technology under grants MOST 105-2221-E-001-030-MY2 and MOST 107-2628-E-001-005-MY3. Publisher Copyright: © 2019, Springer Nature Switzerland AG.

PY - 2019

Y1 - 2019

N2 - In order to learn object segmentation models in videos, conventional methods require a large amount of pixel-wise ground truth annotations. However, collecting such supervised data is time-consuming and labor-intensive. In this paper, we exploit existing annotations in source images and transfer such visual information to segment videos with unseen object categories. Without using any annotations in the target video, we propose a method to jointly mine useful segments and learn feature representations that better adapt to the target frames. The entire process is decomposed into two tasks: (1) solving a submodular function for selecting object-like segments, and (2) learning a CNN model with a transferable module for adapting seen categories in the source domainÂ to the unseen target video. We present an iterative update scheme between two tasks to self-learn the final solution for object segmentation. Experimental results on numerous benchmark datasets show that the proposed method performs favorably against the state-of-the-art algorithms.

AB - In order to learn object segmentation models in videos, conventional methods require a large amount of pixel-wise ground truth annotations. However, collecting such supervised data is time-consuming and labor-intensive. In this paper, we exploit existing annotations in source images and transfer such visual information to segment videos with unseen object categories. Without using any annotations in the target video, we propose a method to jointly mine useful segments and learn feature representations that better adapt to the target frames. The entire process is decomposed into two tasks: (1) solving a submodular function for selecting object-like segments, and (2) learning a CNN model with a transferable module for adapting seen categories in the source domainÂ to the unseen target video. We present an iterative update scheme between two tasks to self-learn the final solution for object segmentation. Experimental results on numerous benchmark datasets show that the proposed method performs favorably against the state-of-the-art algorithms.

UR - http://www.scopus.com/inward/record.url?scp=85066875844&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066875844&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-20870-7_38

DO - 10.1007/978-3-030-20870-7_38

M3 - Conference contribution

AN - SCOPUS:85066875844

SN - 9783030208691

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 615

EP - 631

BT - Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers

A2 - Li, Hongdong

A2 - Jawahar, C.V.

A2 - Mori, Greg

A2 - Schindler, Konrad

PB - Springer Verlag

T2 - 14th Asian Conference on Computer Vision, ACCV 2018

Y2 - 2 December 2018 through 6 December 2018

ER -

Chen YW, Tsai YH, Yang CY, Lin YY, Yang MH. Unseen Object Segmentation in Videos via Transferable Representations. In Li H, Jawahar CV, Mori G, Schindler K, editors, Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. Springer Verlag. 2019. p. 615-631. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-20870-7_38

Unseen Object Segmentation in Videos via Transferable Representations

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this