Overcoming catastrophic forgetting with unlabeled data in the wild

Kibok Lee; Kimin Lee; Jinwoo Shin; Honglak Lee

doi:10.1109/ICCV.2019.00040

Overcoming catastrophic forgetting with unlabeled data in the wild

Kibok Lee, Kimin Lee, Jinwoo Shin, Honglak Lee

Department of Applied Statistics

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

78 Citations (Scopus)

Abstract

Lifelong learning with deep neural networks is well-known to suffer from catastrophic forgetting: The performance on previous tasks drastically degrades when learning a new task. To alleviate this effect, we propose to leverage a large stream of unlabeled data easily obtainable in the wild. In particular, we design a novel class-incremental learning scheme with (a) a new distillation loss, termed global distillation, (b) a learning strategy to avoid overfitting to the most recent task, and (c) a confidence-based sampling method to effectively leverage unlabeled external data. Our experimental results on various datasets, including CIFAR and ImageNet, demonstrate the superiority of the proposed methods over prior methods, particularly when a stream of unlabeled data is accessible: Our method shows up to 15.8% higher accuracy and 46.5% less forgetting compared to the state-of-the-art method. The code is available at https://github.com/kibok90/iccv2019-inc.

Original language	English
Title of host publication	Proceedings - 2019 International Conference on Computer Vision, ICCV 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	312-321
Number of pages	10
ISBN (Electronic)	9781728148038
DOIs	https://doi.org/10.1109/ICCV.2019.00040
Publication status	Published - 2019 Oct
Event	17th IEEE/CVF International Conference on Computer Vision, ICCV 2019 - Seoul, Korea, Republic of Duration: 2019 Oct 27 → 2019 Nov 2

Publication series

Name	Proceedings of the IEEE International Conference on Computer Vision
Volume	2019-October
ISSN (Print)	1550-5499

Conference

Conference	17th IEEE/CVF International Conference on Computer Vision, ICCV 2019
Country/Territory	Korea, Republic of
City	Seoul
Period	19/10/27 → 19/11/2

Bibliographical note

Funding Information:
This work was supported in part by Kwanjeong Educational Foundation Scholarship, NSF CAREER IIS-1453651, and Sloan Research Fellowship. We also thank Lajanugen Logeswaran, Jongwook Choi, Yijie Guo, Wilka Carvalho, and Yunseok Jang for helpful discussions.

Publisher Copyright:
© 2019 IEEE.

All Science Journal Classification (ASJC) codes

Software
Computer Vision and Pattern Recognition

Access to Document

10.1109/ICCV.2019.00040

Cite this

Lee, K., Lee, K., Shin, J., & Lee, H. (2019). Overcoming catastrophic forgetting with unlabeled data in the wild. In Proceedings - 2019 International Conference on Computer Vision, ICCV 2019 (pp. 312-321). Article 9010368 (Proceedings of the IEEE International Conference on Computer Vision; Vol. 2019-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCV.2019.00040

@inproceedings{f22afe3a7e7944cc8166cbe05d9aec8a,

title = "Overcoming catastrophic forgetting with unlabeled data in the wild",

abstract = "Lifelong learning with deep neural networks is well-known to suffer from catastrophic forgetting: The performance on previous tasks drastically degrades when learning a new task. To alleviate this effect, we propose to leverage a large stream of unlabeled data easily obtainable in the wild. In particular, we design a novel class-incremental learning scheme with (a) a new distillation loss, termed global distillation, (b) a learning strategy to avoid overfitting to the most recent task, and (c) a confidence-based sampling method to effectively leverage unlabeled external data. Our experimental results on various datasets, including CIFAR and ImageNet, demonstrate the superiority of the proposed methods over prior methods, particularly when a stream of unlabeled data is accessible: Our method shows up to 15.8% higher accuracy and 46.5% less forgetting compared to the state-of-the-art method. The code is available at https://github.com/kibok90/iccv2019-inc.",

author = "Kibok Lee and Kimin Lee and Jinwoo Shin and Honglak Lee",

note = "Funding Information: This work was supported in part by Kwanjeong Educational Foundation Scholarship, NSF CAREER IIS-1453651, and Sloan Research Fellowship. We also thank Lajanugen Logeswaran, Jongwook Choi, Yijie Guo, Wilka Carvalho, and Yunseok Jang for helpful discussions. Publisher Copyright: {\textcopyright} 2019 IEEE.; 17th IEEE/CVF International Conference on Computer Vision, ICCV 2019 ; Conference date: 27-10-2019 Through 02-11-2019",

year = "2019",

month = oct,

doi = "10.1109/ICCV.2019.00040",

language = "English",

series = "Proceedings of the IEEE International Conference on Computer Vision",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "312--321",

booktitle = "Proceedings - 2019 International Conference on Computer Vision, ICCV 2019",

address = "United States",

}

Lee, K, Lee, K, Shin, J & Lee, H 2019, Overcoming catastrophic forgetting with unlabeled data in the wild. in Proceedings - 2019 International Conference on Computer Vision, ICCV 2019., 9010368, Proceedings of the IEEE International Conference on Computer Vision, vol. 2019-October, Institute of Electrical and Electronics Engineers Inc., pp. 312-321, 17th IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea, Republic of, 19/10/27. https://doi.org/10.1109/ICCV.2019.00040

Overcoming catastrophic forgetting with unlabeled data in the wild. / Lee, Kibok; Lee, Kimin; Shin, Jinwoo et al.
Proceedings - 2019 International Conference on Computer Vision, ICCV 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 312-321 9010368 (Proceedings of the IEEE International Conference on Computer Vision; Vol. 2019-October).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Overcoming catastrophic forgetting with unlabeled data in the wild

AU - Lee, Kibok

AU - Lee, Kimin

AU - Shin, Jinwoo

AU - Lee, Honglak

N1 - Funding Information: This work was supported in part by Kwanjeong Educational Foundation Scholarship, NSF CAREER IIS-1453651, and Sloan Research Fellowship. We also thank Lajanugen Logeswaran, Jongwook Choi, Yijie Guo, Wilka Carvalho, and Yunseok Jang for helpful discussions. Publisher Copyright: © 2019 IEEE.

PY - 2019/10

Y1 - 2019/10

N2 - Lifelong learning with deep neural networks is well-known to suffer from catastrophic forgetting: The performance on previous tasks drastically degrades when learning a new task. To alleviate this effect, we propose to leverage a large stream of unlabeled data easily obtainable in the wild. In particular, we design a novel class-incremental learning scheme with (a) a new distillation loss, termed global distillation, (b) a learning strategy to avoid overfitting to the most recent task, and (c) a confidence-based sampling method to effectively leverage unlabeled external data. Our experimental results on various datasets, including CIFAR and ImageNet, demonstrate the superiority of the proposed methods over prior methods, particularly when a stream of unlabeled data is accessible: Our method shows up to 15.8% higher accuracy and 46.5% less forgetting compared to the state-of-the-art method. The code is available at https://github.com/kibok90/iccv2019-inc.

AB - Lifelong learning with deep neural networks is well-known to suffer from catastrophic forgetting: The performance on previous tasks drastically degrades when learning a new task. To alleviate this effect, we propose to leverage a large stream of unlabeled data easily obtainable in the wild. In particular, we design a novel class-incremental learning scheme with (a) a new distillation loss, termed global distillation, (b) a learning strategy to avoid overfitting to the most recent task, and (c) a confidence-based sampling method to effectively leverage unlabeled external data. Our experimental results on various datasets, including CIFAR and ImageNet, demonstrate the superiority of the proposed methods over prior methods, particularly when a stream of unlabeled data is accessible: Our method shows up to 15.8% higher accuracy and 46.5% less forgetting compared to the state-of-the-art method. The code is available at https://github.com/kibok90/iccv2019-inc.

UR - http://www.scopus.com/inward/record.url?scp=85081886304&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85081886304&partnerID=8YFLogxK

U2 - 10.1109/ICCV.2019.00040

DO - 10.1109/ICCV.2019.00040

M3 - Conference contribution

AN - SCOPUS:85081886304

T3 - Proceedings of the IEEE International Conference on Computer Vision

SP - 312

EP - 321

BT - Proceedings - 2019 International Conference on Computer Vision, ICCV 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 17th IEEE/CVF International Conference on Computer Vision, ICCV 2019

Y2 - 27 October 2019 through 2 November 2019

ER -

Overcoming catastrophic forgetting with unlabeled data in the wild

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this