Modeling Artistic Workflows for Image Generation and Editing

Hung Yu Tseng; Matthew Fisher; Jingwan Lu; Yijun Li; Vladimir Kim; Ming Hsuan Yang

doi:10.1007/978-3-030-58523-5_10

Modeling Artistic Workflows for Image Generation and Editing

Hung Yu Tseng, Matthew Fisher, Jingwan Lu, Yijun Li, Vladimir Kim, Ming Hsuan Yang

College of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

7 Citations (Scopus)

Abstract

People often create art by following an artistic workflow involving multiple stages that inform the overall design. If an artist wishes to modify an earlier decision, significant work may be required to propagate this new decision forward to the final artwork. Motivated by the above observations, we propose a generative model that follows a given artistic workflow, enabling both multi-stage image generation as well as multi-stage image editing of an existing piece of art. Furthermore, for the editing scenario, we introduce an optimization process along with learning-based regularization to ensure the edited image produced by the model closely aligns with the originally provided image. Qualitative and quantitative results on three different artistic datasets demonstrate the effectiveness of the proposed framework on both image generation and editing tasks.

Original language	English
Title of host publication	Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings
Editors	Andrea Vedaldi, Horst Bischof, Thomas Brox, Jan-Michael Frahm
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	158-174
Number of pages	17
ISBN (Print)	9783030585228
DOIs	https://doi.org/10.1007/978-3-030-58523-5_10
Publication status	Published - 2020
Event	16th European Conference on Computer Vision, ECCV 2020 - Glasgow, United Kingdom Duration: 2020 Aug 23 → 2020 Aug 28

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12363 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	16th European Conference on Computer Vision, ECCV 2020
Country/Territory	United Kingdom
City	Glasgow
Period	20/8/23 → 20/8/28

Bibliographical note

Funding Information:
supported in part by the NSF CAREER Grant

Funding Information:
This work is supported in part by the NSF CAREER Grant #1149783.

Publisher Copyright:
© 2020, Springer Nature Switzerland AG.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-030-58523-5_10

Cite this

Tseng, H. Y., Fisher, M., Lu, J., Li, Y., Kim, V., & Yang, M. H. (2020). Modeling Artistic Workflows for Image Generation and Editing. In A. Vedaldi, H. Bischof, T. Brox, & J-M. Frahm (Eds.), Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings (pp. 158-174). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12363 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-58523-5_10

Tseng, Hung Yu ; Fisher, Matthew ; Lu, Jingwan et al. / Modeling Artistic Workflows for Image Generation and Editing. Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings. editor / Andrea Vedaldi ; Horst Bischof ; Thomas Brox ; Jan-Michael Frahm. Springer Science and Business Media Deutschland GmbH, 2020. pp. 158-174 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{1120beaa09834e67b9b55ea1df306a80,

title = "Modeling Artistic Workflows for Image Generation and Editing",

abstract = "People often create art by following an artistic workflow involving multiple stages that inform the overall design. If an artist wishes to modify an earlier decision, significant work may be required to propagate this new decision forward to the final artwork. Motivated by the above observations, we propose a generative model that follows a given artistic workflow, enabling both multi-stage image generation as well as multi-stage image editing of an existing piece of art. Furthermore, for the editing scenario, we introduce an optimization process along with learning-based regularization to ensure the edited image produced by the model closely aligns with the originally provided image. Qualitative and quantitative results on three different artistic datasets demonstrate the effectiveness of the proposed framework on both image generation and editing tasks.",

author = "Tseng, {Hung Yu} and Matthew Fisher and Jingwan Lu and Yijun Li and Vladimir Kim and Yang, {Ming Hsuan}",

note = "Funding Information: supported in part by the NSF CAREER Grant Funding Information: This work is supported in part by the NSF CAREER Grant #1149783. Publisher Copyright: {\textcopyright} 2020, Springer Nature Switzerland AG.; 16th European Conference on Computer Vision, ECCV 2020 ; Conference date: 23-08-2020 Through 28-08-2020",

year = "2020",

doi = "10.1007/978-3-030-58523-5_10",

language = "English",

isbn = "9783030585228",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "158--174",

editor = "Andrea Vedaldi and Horst Bischof and Thomas Brox and Jan-Michael Frahm",

booktitle = "Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings",

address = "Germany",

}

Tseng, HY, Fisher, M, Lu, J, Li, Y, Kim, V & Yang, MH 2020, Modeling Artistic Workflows for Image Generation and Editing. in A Vedaldi, H Bischof, T Brox & J-M Frahm (eds), Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12363 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 158-174, 16th European Conference on Computer Vision, ECCV 2020, Glasgow, United Kingdom, 20/8/23. https://doi.org/10.1007/978-3-030-58523-5_10

Modeling Artistic Workflows for Image Generation and Editing. / Tseng, Hung Yu; Fisher, Matthew; Lu, Jingwan et al.
Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings. ed. / Andrea Vedaldi; Horst Bischof; Thomas Brox; Jan-Michael Frahm. Springer Science and Business Media Deutschland GmbH, 2020. p. 158-174 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12363 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Modeling Artistic Workflows for Image Generation and Editing

AU - Tseng, Hung Yu

AU - Fisher, Matthew

AU - Lu, Jingwan

AU - Li, Yijun

AU - Kim, Vladimir

AU - Yang, Ming Hsuan

N1 - Funding Information: supported in part by the NSF CAREER Grant Funding Information: This work is supported in part by the NSF CAREER Grant #1149783. Publisher Copyright: © 2020, Springer Nature Switzerland AG.

PY - 2020

Y1 - 2020

N2 - People often create art by following an artistic workflow involving multiple stages that inform the overall design. If an artist wishes to modify an earlier decision, significant work may be required to propagate this new decision forward to the final artwork. Motivated by the above observations, we propose a generative model that follows a given artistic workflow, enabling both multi-stage image generation as well as multi-stage image editing of an existing piece of art. Furthermore, for the editing scenario, we introduce an optimization process along with learning-based regularization to ensure the edited image produced by the model closely aligns with the originally provided image. Qualitative and quantitative results on three different artistic datasets demonstrate the effectiveness of the proposed framework on both image generation and editing tasks.

AB - People often create art by following an artistic workflow involving multiple stages that inform the overall design. If an artist wishes to modify an earlier decision, significant work may be required to propagate this new decision forward to the final artwork. Motivated by the above observations, we propose a generative model that follows a given artistic workflow, enabling both multi-stage image generation as well as multi-stage image editing of an existing piece of art. Furthermore, for the editing scenario, we introduce an optimization process along with learning-based regularization to ensure the edited image produced by the model closely aligns with the originally provided image. Qualitative and quantitative results on three different artistic datasets demonstrate the effectiveness of the proposed framework on both image generation and editing tasks.

UR - http://www.scopus.com/inward/record.url?scp=85097817979&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85097817979&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-58523-5_10

DO - 10.1007/978-3-030-58523-5_10

M3 - Conference contribution

AN - SCOPUS:85097817979

SN - 9783030585228

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 158

EP - 174

BT - Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings

A2 - Vedaldi, Andrea

A2 - Bischof, Horst

A2 - Brox, Thomas

A2 - Frahm, Jan-Michael

PB - Springer Science and Business Media Deutschland GmbH

T2 - 16th European Conference on Computer Vision, ECCV 2020

Y2 - 23 August 2020 through 28 August 2020

ER -

Tseng HY, Fisher M, Lu J, Li Y, Kim V, Yang MH. Modeling Artistic Workflows for Image Generation and Editing. In Vedaldi A, Bischof H, Brox T, Frahm J-M, editors, Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings. Springer Science and Business Media Deutschland GmbH. 2020. p. 158-174. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-58523-5_10

Modeling Artistic Workflows for Image Generation and Editing

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this