Context driven scene parsing with attention to rare classes

Jimei Yang; Brian Price; Scott Cohen; Ming Hsuan Yang

doi:10.1109/CVPR.2014.415

Context driven scene parsing with attention to rare classes

Jimei Yang, Brian Price, Scott Cohen, Ming Hsuan Yang

College of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

130 Citations (Scopus)

Abstract

This paper presents a scalable scene parsing algorithm based on image retrieval and superpixel matching. We focus on rare object classes, which play an important role in achieving richer semantic understanding of visual scenes, compared to common background classes. Towards this end, we make two novel contributions: rare class expansion and semantic context description. First, considering the long-tailed nature of the label distribution, we expand the retrieval set by rare class exemplars and thus achieve more balanced superpixel classification results. Second, we incorporate both global and local semantic context information through a feedback based mechanism to refine image retrieval and superpixel matching. Results on the SIFTflow and LMSun datasets show the superior performance of our algorithm, especially on the rare classes, without sacrificing overall labeling accuracy.

Original language	English
Title of host publication	Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Publisher	IEEE Computer Society
Pages	3294-3301
Number of pages	8
ISBN (Electronic)	9781479951178, 9781479951178
DOIs	https://doi.org/10.1109/CVPR.2014.415
Publication status	Published - 2014 Sept 24
Event	27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014 - Columbus, United States Duration: 2014 Jun 23 → 2014 Jun 28

Publication series

Name	Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
ISSN (Print)	1063-6919

Other

Other	27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014
Country/Territory	United States
City	Columbus
Period	14/6/23 → 14/6/28

Bibliographical note

Publisher Copyright:
© 2014 IEEE.

All Science Journal Classification (ASJC) codes

Software
Computer Vision and Pattern Recognition

Access to Document

10.1109/CVPR.2014.415

Cite this

Yang, J., Price, B., Cohen, S., & Yang, M. H. (2014). Context driven scene parsing with attention to rare classes. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 3294-3301). Article 6909817 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition). IEEE Computer Society. https://doi.org/10.1109/CVPR.2014.415

@inproceedings{9c9e252b6e824735913e13eb774845aa,

title = "Context driven scene parsing with attention to rare classes",

abstract = "This paper presents a scalable scene parsing algorithm based on image retrieval and superpixel matching. We focus on rare object classes, which play an important role in achieving richer semantic understanding of visual scenes, compared to common background classes. Towards this end, we make two novel contributions: rare class expansion and semantic context description. First, considering the long-tailed nature of the label distribution, we expand the retrieval set by rare class exemplars and thus achieve more balanced superpixel classification results. Second, we incorporate both global and local semantic context information through a feedback based mechanism to refine image retrieval and superpixel matching. Results on the SIFTflow and LMSun datasets show the superior performance of our algorithm, especially on the rare classes, without sacrificing overall labeling accuracy.",

author = "Jimei Yang and Brian Price and Scott Cohen and Yang, {Ming Hsuan}",

note = "Publisher Copyright: {\textcopyright} 2014 IEEE.; 27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014 ; Conference date: 23-06-2014 Through 28-06-2014",

year = "2014",

month = sep,

day = "24",

doi = "10.1109/CVPR.2014.415",

language = "English",

series = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",

publisher = "IEEE Computer Society",

pages = "3294--3301",

booktitle = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",

address = "United States",

}

Yang, J, Price, B, Cohen, S & Yang, MH 2014, Context driven scene parsing with attention to rare classes. in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition., 6909817, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, pp. 3294-3301, 27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, United States, 14/6/23. https://doi.org/10.1109/CVPR.2014.415

Context driven scene parsing with attention to rare classes. / Yang, Jimei; Price, Brian; Cohen, Scott et al.
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 2014. p. 3294-3301 6909817 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Context driven scene parsing with attention to rare classes

AU - Yang, Jimei

AU - Price, Brian

AU - Cohen, Scott

AU - Yang, Ming Hsuan

PY - 2014/9/24

Y1 - 2014/9/24

N2 - This paper presents a scalable scene parsing algorithm based on image retrieval and superpixel matching. We focus on rare object classes, which play an important role in achieving richer semantic understanding of visual scenes, compared to common background classes. Towards this end, we make two novel contributions: rare class expansion and semantic context description. First, considering the long-tailed nature of the label distribution, we expand the retrieval set by rare class exemplars and thus achieve more balanced superpixel classification results. Second, we incorporate both global and local semantic context information through a feedback based mechanism to refine image retrieval and superpixel matching. Results on the SIFTflow and LMSun datasets show the superior performance of our algorithm, especially on the rare classes, without sacrificing overall labeling accuracy.

AB - This paper presents a scalable scene parsing algorithm based on image retrieval and superpixel matching. We focus on rare object classes, which play an important role in achieving richer semantic understanding of visual scenes, compared to common background classes. Towards this end, we make two novel contributions: rare class expansion and semantic context description. First, considering the long-tailed nature of the label distribution, we expand the retrieval set by rare class exemplars and thus achieve more balanced superpixel classification results. Second, we incorporate both global and local semantic context information through a feedback based mechanism to refine image retrieval and superpixel matching. Results on the SIFTflow and LMSun datasets show the superior performance of our algorithm, especially on the rare classes, without sacrificing overall labeling accuracy.

UR - http://www.scopus.com/inward/record.url?scp=84911380286&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84911380286&partnerID=8YFLogxK

U2 - 10.1109/CVPR.2014.415

DO - 10.1109/CVPR.2014.415

M3 - Conference contribution

AN - SCOPUS:84911380286

T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

SP - 3294

EP - 3301

BT - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

PB - IEEE Computer Society

T2 - 27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014

Y2 - 23 June 2014 through 28 June 2014

ER -

Yang J, Price B, Cohen S, Yang MH. Context driven scene parsing with attention to rare classes. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society. 2014. p. 3294-3301. 6909817. (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition). doi: 10.1109/CVPR.2014.415

Context driven scene parsing with attention to rare classes

Abstract

Publication series

Other

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this