Simplifying deep neural networks for FPGA-like neuromorphic systems

Jaeyong Chung; Taehwan Shin; Joon Sung Yang

doi:10.1109/TCAD.2018.2877016

Simplifying deep neural networks for FPGA-like neuromorphic systems

Jaeyong Chung, Taehwan Shin, Joon Sung Yang

College of Engineering

Research output: Contribution to journal › Article › peer-review

3 Citations (Scopus)

Abstract

Deep learning using deep neural networks is taking machine intelligence to the next level in computer vision, speech recognition, natural language processing, etc. Brain-like hardware platforms for the brain-inspired computational models are being studied, but the maximum size of neural networks they can evaluate is often limited by the number of neurons and synapses equipped with the hardware. This paper presents two techniques, factorization and pruning, that not only compress the models but also maintain the form of the models for the execution on neuromorphic architectures. We also propose a novel method to combine the two techniques. The proposed method shows significant improvements in reducing the number of model parameters over standalone use of each method while maintaining the performance. Our experimental results show that the proposed method can achieve 30 × reduction rate within 1% budget of accuracy for the largest layer of AlexNet.

Original language	English
Article number	8500327
Pages (from-to)	2032-2042
Number of pages	11
Journal	IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Volume	38
Issue number	11
DOIs	https://doi.org/10.1109/TCAD.2018.2877016
Publication status	Published - 2019 Nov

Bibliographical note

Funding Information:
Manuscript received January 30, 2018; revised April 18, 2018; accepted May 31, 2018. Date of publication October 19, 2018; date of current version October 16, 2019. This work was supported in part by the Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Education under Grant NRF-2017R1D1A1B03029103, and in part by the Institute for Information and Communications Technology Promotion funded by the Korea Government under Grant 1711073912. Preliminary results of this study were presented at DAC 2016. This paper was recommended by Associate Editor Y. Wang.

Funding Information:
ACKNOWLEDGMENT The authors would like to thank EDA tools used in this paper were supported by IDEC.

Publisher Copyright:
© 1982-2012 IEEE.

All Science Journal Classification (ASJC) codes

Software
Computer Graphics and Computer-Aided Design
Electrical and Electronic Engineering

Access to Document

10.1109/TCAD.2018.2877016

Cite this

@article{ee7901b01a34427cb8488ce15a2947c6,

title = "Simplifying deep neural networks for FPGA-like neuromorphic systems",

abstract = "Deep learning using deep neural networks is taking machine intelligence to the next level in computer vision, speech recognition, natural language processing, etc. Brain-like hardware platforms for the brain-inspired computational models are being studied, but the maximum size of neural networks they can evaluate is often limited by the number of neurons and synapses equipped with the hardware. This paper presents two techniques, factorization and pruning, that not only compress the models but also maintain the form of the models for the execution on neuromorphic architectures. We also propose a novel method to combine the two techniques. The proposed method shows significant improvements in reducing the number of model parameters over standalone use of each method while maintaining the performance. Our experimental results show that the proposed method can achieve 30 × reduction rate within 1% budget of accuracy for the largest layer of AlexNet.",

author = "Jaeyong Chung and Taehwan Shin and Yang, {Joon Sung}",

note = "Funding Information: Manuscript received January 30, 2018; revised April 18, 2018; accepted May 31, 2018. Date of publication October 19, 2018; date of current version October 16, 2019. This work was supported in part by the Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Education under Grant NRF-2017R1D1A1B03029103, and in part by the Institute for Information and Communications Technology Promotion funded by the Korea Government under Grant 1711073912. Preliminary results of this study were presented at DAC 2016. This paper was recommended by Associate Editor Y. Wang. Funding Information: ACKNOWLEDGMENT The authors would like to thank EDA tools used in this paper were supported by IDEC. Publisher Copyright: {\textcopyright} 1982-2012 IEEE.",

year = "2019",

month = nov,

doi = "10.1109/TCAD.2018.2877016",

language = "English",

volume = "38",

pages = "2032--2042",

journal = "IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems",

issn = "0278-0070",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "11",

}

TY - JOUR

T1 - Simplifying deep neural networks for FPGA-like neuromorphic systems

AU - Chung, Jaeyong

AU - Shin, Taehwan

AU - Yang, Joon Sung

N1 - Funding Information: Manuscript received January 30, 2018; revised April 18, 2018; accepted May 31, 2018. Date of publication October 19, 2018; date of current version October 16, 2019. This work was supported in part by the Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Education under Grant NRF-2017R1D1A1B03029103, and in part by the Institute for Information and Communications Technology Promotion funded by the Korea Government under Grant 1711073912. Preliminary results of this study were presented at DAC 2016. This paper was recommended by Associate Editor Y. Wang. Funding Information: ACKNOWLEDGMENT The authors would like to thank EDA tools used in this paper were supported by IDEC. Publisher Copyright: © 1982-2012 IEEE.

PY - 2019/11

Y1 - 2019/11

N2 - Deep learning using deep neural networks is taking machine intelligence to the next level in computer vision, speech recognition, natural language processing, etc. Brain-like hardware platforms for the brain-inspired computational models are being studied, but the maximum size of neural networks they can evaluate is often limited by the number of neurons and synapses equipped with the hardware. This paper presents two techniques, factorization and pruning, that not only compress the models but also maintain the form of the models for the execution on neuromorphic architectures. We also propose a novel method to combine the two techniques. The proposed method shows significant improvements in reducing the number of model parameters over standalone use of each method while maintaining the performance. Our experimental results show that the proposed method can achieve 30 × reduction rate within 1% budget of accuracy for the largest layer of AlexNet.

AB - Deep learning using deep neural networks is taking machine intelligence to the next level in computer vision, speech recognition, natural language processing, etc. Brain-like hardware platforms for the brain-inspired computational models are being studied, but the maximum size of neural networks they can evaluate is often limited by the number of neurons and synapses equipped with the hardware. This paper presents two techniques, factorization and pruning, that not only compress the models but also maintain the form of the models for the execution on neuromorphic architectures. We also propose a novel method to combine the two techniques. The proposed method shows significant improvements in reducing the number of model parameters over standalone use of each method while maintaining the performance. Our experimental results show that the proposed method can achieve 30 × reduction rate within 1% budget of accuracy for the largest layer of AlexNet.

UR - http://www.scopus.com/inward/record.url?scp=85055194261&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055194261&partnerID=8YFLogxK

U2 - 10.1109/TCAD.2018.2877016

DO - 10.1109/TCAD.2018.2877016

M3 - Article

AN - SCOPUS:85055194261

SN - 0278-0070

VL - 38

SP - 2032

EP - 2042

JO - IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems

JF - IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems

IS - 11

M1 - 8500327

ER -

Simplifying deep neural networks for FPGA-like neuromorphic systems

Abstract

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this