TREND: Truncated Generalized Normal Density Estimation of Inception Embeddings for GAN Evaluation

Junghyuk Lee; Jong Seok Lee

doi:10.1007/978-3-031-20050-2_6

TREND: Truncated Generalized Normal Density Estimation of Inception Embeddings for GAN Evaluation

Junghyuk Lee, Jong Seok Lee

School of Integrated Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

Evaluating image generation models such as generative adversarial networks (GANs) is a challenging problem. A common approach is to compare the distributions of the set of ground truth images and the set of generated test images. The Frechét Inception distance is one of the most widely used metrics for evaluation of GANs, which assumes that the features from a trained Inception model for a set of images follow a normal distribution. In this paper, we argue that this is an over-simplified assumption, which may lead to unreliable evaluation results, and more accurate density estimation can be achieved using a truncated generalized normal distribution. Based on this, we propose a novel metric for accurate evaluation of GANs, named TREND (TRuncated gEneralized Normal Density estimation of inception embeddings). We demonstrate that our approach significantly reduces errors of density estimation, which consequently eliminates the risk of faulty evaluation results. Furthermore, the proposed metric significantly improves robustness of evaluation results against variation of the number of image samples.

Original language	English
Title of host publication	Computer Vision – ECCV 2022 - 17th European Conference, 2022, Proceedings
Editors	Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, Tal Hassner
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	87-103
Number of pages	17
ISBN (Print)	9783031200496
DOIs	https://doi.org/10.1007/978-3-031-20050-2_6
Publication status	Published - 2022
Event	17th European Conference on Computer Vision, ECCV 2022 - Tel Aviv, Israel Duration: 2022 Oct 23 → 2022 Oct 27

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13683 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	17th European Conference on Computer Vision, ECCV 2022
Country/Territory	Israel
City	Tel Aviv
Period	22/10/23 → 22/10/27

Bibliographical note

Publisher Copyright:
© 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-031-20050-2_6

Cite this

Lee, J., & Lee, J. S. (2022). TREND: Truncated Generalized Normal Density Estimation of Inception Embeddings for GAN Evaluation. In S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, & T. Hassner (Eds.), Computer Vision – ECCV 2022 - 17th European Conference, 2022, Proceedings (pp. 87-103). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13683 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-20050-2_6

Lee, Junghyuk ; Lee, Jong Seok. / TREND : Truncated Generalized Normal Density Estimation of Inception Embeddings for GAN Evaluation. Computer Vision – ECCV 2022 - 17th European Conference, 2022, Proceedings. editor / Shai Avidan ; Gabriel Brostow ; Moustapha Cissé ; Giovanni Maria Farinella ; Tal Hassner. Springer Science and Business Media Deutschland GmbH, 2022. pp. 87-103 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{d44edf63e81243febbcb933af9a25b71,

title = "TREND: Truncated Generalized Normal Density Estimation of Inception Embeddings for GAN Evaluation",

abstract = "Evaluating image generation models such as generative adversarial networks (GANs) is a challenging problem. A common approach is to compare the distributions of the set of ground truth images and the set of generated test images. The Frech{\'e}t Inception distance is one of the most widely used metrics for evaluation of GANs, which assumes that the features from a trained Inception model for a set of images follow a normal distribution. In this paper, we argue that this is an over-simplified assumption, which may lead to unreliable evaluation results, and more accurate density estimation can be achieved using a truncated generalized normal distribution. Based on this, we propose a novel metric for accurate evaluation of GANs, named TREND (TRuncated gEneralized Normal Density estimation of inception embeddings). We demonstrate that our approach significantly reduces errors of density estimation, which consequently eliminates the risk of faulty evaluation results. Furthermore, the proposed metric significantly improves robustness of evaluation results against variation of the number of image samples.",

keywords = "Generative adversarial networks, Image generation, Image quality, Performance evaluation metrics",

author = "Junghyuk Lee and Lee, {Jong Seok}",

note = "Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 17th European Conference on Computer Vision, ECCV 2022 ; Conference date: 23-10-2022 Through 27-10-2022",

year = "2022",

doi = "10.1007/978-3-031-20050-2_6",

language = "English",

isbn = "9783031200496",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "87--103",

editor = "Shai Avidan and Gabriel Brostow and Moustapha Ciss{\'e} and Farinella, {Giovanni Maria} and Tal Hassner",

booktitle = "Computer Vision – ECCV 2022 - 17th European Conference, 2022, Proceedings",

address = "Germany",

}

Lee, J & Lee, JS 2022, TREND: Truncated Generalized Normal Density Estimation of Inception Embeddings for GAN Evaluation. in S Avidan, G Brostow, M Cissé, GM Farinella & T Hassner (eds), Computer Vision – ECCV 2022 - 17th European Conference, 2022, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13683 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 87-103, 17th European Conference on Computer Vision, ECCV 2022, Tel Aviv, Israel, 22/10/23. https://doi.org/10.1007/978-3-031-20050-2_6

TREND: Truncated Generalized Normal Density Estimation of Inception Embeddings for GAN Evaluation. / Lee, Junghyuk; Lee, Jong Seok.
Computer Vision – ECCV 2022 - 17th European Conference, 2022, Proceedings. ed. / Shai Avidan; Gabriel Brostow; Moustapha Cissé; Giovanni Maria Farinella; Tal Hassner. Springer Science and Business Media Deutschland GmbH, 2022. p. 87-103 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13683 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - TREND

T2 - 17th European Conference on Computer Vision, ECCV 2022

AU - Lee, Junghyuk

AU - Lee, Jong Seok

PY - 2022

Y1 - 2022

N2 - Evaluating image generation models such as generative adversarial networks (GANs) is a challenging problem. A common approach is to compare the distributions of the set of ground truth images and the set of generated test images. The Frechét Inception distance is one of the most widely used metrics for evaluation of GANs, which assumes that the features from a trained Inception model for a set of images follow a normal distribution. In this paper, we argue that this is an over-simplified assumption, which may lead to unreliable evaluation results, and more accurate density estimation can be achieved using a truncated generalized normal distribution. Based on this, we propose a novel metric for accurate evaluation of GANs, named TREND (TRuncated gEneralized Normal Density estimation of inception embeddings). We demonstrate that our approach significantly reduces errors of density estimation, which consequently eliminates the risk of faulty evaluation results. Furthermore, the proposed metric significantly improves robustness of evaluation results against variation of the number of image samples.

AB - Evaluating image generation models such as generative adversarial networks (GANs) is a challenging problem. A common approach is to compare the distributions of the set of ground truth images and the set of generated test images. The Frechét Inception distance is one of the most widely used metrics for evaluation of GANs, which assumes that the features from a trained Inception model for a set of images follow a normal distribution. In this paper, we argue that this is an over-simplified assumption, which may lead to unreliable evaluation results, and more accurate density estimation can be achieved using a truncated generalized normal distribution. Based on this, we propose a novel metric for accurate evaluation of GANs, named TREND (TRuncated gEneralized Normal Density estimation of inception embeddings). We demonstrate that our approach significantly reduces errors of density estimation, which consequently eliminates the risk of faulty evaluation results. Furthermore, the proposed metric significantly improves robustness of evaluation results against variation of the number of image samples.

KW - Generative adversarial networks

KW - Image generation

KW - Image quality

KW - Performance evaluation metrics

UR - http://www.scopus.com/inward/record.url?scp=85142694537&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85142694537&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-20050-2_6

DO - 10.1007/978-3-031-20050-2_6

M3 - Conference contribution

AN - SCOPUS:85142694537

SN - 9783031200496

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 87

EP - 103

BT - Computer Vision – ECCV 2022 - 17th European Conference, 2022, Proceedings

A2 - Avidan, Shai

A2 - Brostow, Gabriel

A2 - Cissé, Moustapha

A2 - Farinella, Giovanni Maria

A2 - Hassner, Tal

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 23 October 2022 through 27 October 2022

ER -

Lee J, Lee JS. TREND: Truncated Generalized Normal Density Estimation of Inception Embeddings for GAN Evaluation. In Avidan S, Brostow G, Cissé M, Farinella GM, Hassner T, editors, Computer Vision – ECCV 2022 - 17th European Conference, 2022, Proceedings. Springer Science and Business Media Deutschland GmbH. 2022. p. 87-103. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-20050-2_6