Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU

Dongsu Lee; Hyunyul Lim; Tae Hyun Kim; Sungho Kang

doi:10.1109/TENCON.2018.8650184

Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU

Dongsu Lee, Hyunyul Lim, Tae Hyun Kim, Sungho Kang

Department of Electrical and Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Recently, the researches on DNN (deep neural network) using GPUs has been actively conducted. The reason for using GPUs in DNN is that it reduces the learning time by using many computational cores. However, GPUs have no implements to support the reliable computing operations. It can exacerbate the reliability of the deep neural network. To ensure the reliability of the deep neural network, the proposed approach is to utilize the dropout technique used in MLP (Multi-Layer Perceptron) learning. In case of the dropout method, some threads in GPUs do not participate in the calculation and it causes GPUs underutilization. The proposed method uses the GPUs underutilization to support a reliable deep neural network. The proposed approach is available through using the idle neurons with adjacent calculating neurons in the dropout process. The experiment results show that the proposed approach is able to support the reliability issues in GPUs while executing deep neural network algorithms.

Original language	English
Title of host publication	Proceedings of TENCON 2018 - 2018 IEEE Region 10 Conference
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2281-2286
Number of pages	6
ISBN (Electronic)	9781538654576
DOIs	https://doi.org/10.1109/TENCON.2018.8650184
Publication status	Published - 2019 Feb 22
Event	2018 IEEE Region 10 Conference, TENCON 2018 - Jeju, Korea, Republic of Duration: 2018 Oct 28 → 2018 Oct 31

Publication series

Name	IEEE Region 10 Annual International Conference, Proceedings/TENCON
Volume	2018-October
ISSN (Print)	2159-3442
ISSN (Electronic)	2159-3450

Conference

Conference	2018 IEEE Region 10 Conference, TENCON 2018
Country/Territory	Korea, Republic of
City	Jeju
Period	18/10/28 → 18/10/31

Bibliographical note

Funding Information:
This work was supported by Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government(MSIP) (No.2016-0-00140, Development of Application Program Optimization Tools for High Performance Computing Systems).

Publisher Copyright:
© 2018 IEEE.

All Science Journal Classification (ASJC) codes

Computer Science Applications
Electrical and Electronic Engineering

Access to Document

10.1109/TENCON.2018.8650184

Cite this

Lee, D., Lim, H., Kim, T. H., & Kang, S. (2019). Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU. In Proceedings of TENCON 2018 - 2018 IEEE Region 10 Conference (pp. 2281-2286). Article 8650184 (IEEE Region 10 Annual International Conference, Proceedings/TENCON; Vol. 2018-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/TENCON.2018.8650184

@inproceedings{ee2007d1737241dfa353175fd5db692c,

title = "Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU",

abstract = "Recently, the researches on DNN (deep neural network) using GPUs has been actively conducted. The reason for using GPUs in DNN is that it reduces the learning time by using many computational cores. However, GPUs have no implements to support the reliable computing operations. It can exacerbate the reliability of the deep neural network. To ensure the reliability of the deep neural network, the proposed approach is to utilize the dropout technique used in MLP (Multi-Layer Perceptron) learning. In case of the dropout method, some threads in GPUs do not participate in the calculation and it causes GPUs underutilization. The proposed method uses the GPUs underutilization to support a reliable deep neural network. The proposed approach is available through using the idle neurons with adjacent calculating neurons in the dropout process. The experiment results show that the proposed approach is able to support the reliability issues in GPUs while executing deep neural network algorithms.",

author = "Dongsu Lee and Hyunyul Lim and Kim, {Tae Hyun} and Sungho Kang",

note = "Funding Information: This work was supported by Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government(MSIP) (No.2016-0-00140, Development of Application Program Optimization Tools for High Performance Computing Systems). Publisher Copyright: {\textcopyright} 2018 IEEE.; 2018 IEEE Region 10 Conference, TENCON 2018 ; Conference date: 28-10-2018 Through 31-10-2018",

year = "2019",

month = feb,

day = "22",

doi = "10.1109/TENCON.2018.8650184",

language = "English",

series = "IEEE Region 10 Annual International Conference, Proceedings/TENCON",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2281--2286",

booktitle = "Proceedings of TENCON 2018 - 2018 IEEE Region 10 Conference",

address = "United States",

}

Lee, D, Lim, H, Kim, TH & Kang, S 2019, Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU. in Proceedings of TENCON 2018 - 2018 IEEE Region 10 Conference., 8650184, IEEE Region 10 Annual International Conference, Proceedings/TENCON, vol. 2018-October, Institute of Electrical and Electronics Engineers Inc., pp. 2281-2286, 2018 IEEE Region 10 Conference, TENCON 2018, Jeju, Korea, Republic of, 18/10/28. https://doi.org/10.1109/TENCON.2018.8650184

Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU. / Lee, Dongsu; Lim, Hyunyul; Kim, Tae Hyun et al.
Proceedings of TENCON 2018 - 2018 IEEE Region 10 Conference. Institute of Electrical and Electronics Engineers Inc., 2019. p. 2281-2286 8650184 (IEEE Region 10 Annual International Conference, Proceedings/TENCON; Vol. 2018-October).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU

AU - Lee, Dongsu

AU - Lim, Hyunyul

AU - Kim, Tae Hyun

AU - Kang, Sungho

N1 - Funding Information: This work was supported by Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government(MSIP) (No.2016-0-00140, Development of Application Program Optimization Tools for High Performance Computing Systems). Publisher Copyright: © 2018 IEEE.

PY - 2019/2/22

Y1 - 2019/2/22

N2 - Recently, the researches on DNN (deep neural network) using GPUs has been actively conducted. The reason for using GPUs in DNN is that it reduces the learning time by using many computational cores. However, GPUs have no implements to support the reliable computing operations. It can exacerbate the reliability of the deep neural network. To ensure the reliability of the deep neural network, the proposed approach is to utilize the dropout technique used in MLP (Multi-Layer Perceptron) learning. In case of the dropout method, some threads in GPUs do not participate in the calculation and it causes GPUs underutilization. The proposed method uses the GPUs underutilization to support a reliable deep neural network. The proposed approach is available through using the idle neurons with adjacent calculating neurons in the dropout process. The experiment results show that the proposed approach is able to support the reliability issues in GPUs while executing deep neural network algorithms.

AB - Recently, the researches on DNN (deep neural network) using GPUs has been actively conducted. The reason for using GPUs in DNN is that it reduces the learning time by using many computational cores. However, GPUs have no implements to support the reliable computing operations. It can exacerbate the reliability of the deep neural network. To ensure the reliability of the deep neural network, the proposed approach is to utilize the dropout technique used in MLP (Multi-Layer Perceptron) learning. In case of the dropout method, some threads in GPUs do not participate in the calculation and it causes GPUs underutilization. The proposed method uses the GPUs underutilization to support a reliable deep neural network. The proposed approach is available through using the idle neurons with adjacent calculating neurons in the dropout process. The experiment results show that the proposed approach is able to support the reliability issues in GPUs while executing deep neural network algorithms.

UR - http://www.scopus.com/inward/record.url?scp=85063187356&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063187356&partnerID=8YFLogxK

U2 - 10.1109/TENCON.2018.8650184

DO - 10.1109/TENCON.2018.8650184

M3 - Conference contribution

AN - SCOPUS:85063187356

T3 - IEEE Region 10 Annual International Conference, Proceedings/TENCON

SP - 2281

EP - 2286

BT - Proceedings of TENCON 2018 - 2018 IEEE Region 10 Conference

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2018 IEEE Region 10 Conference, TENCON 2018

Y2 - 28 October 2018 through 31 October 2018

ER -

Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this