SIF-NPU: A 28nm 3.48 TOPS/W 0.25 TOPS/mm2CNN Accelerator with Spatially Independent Fusion for Real-Time UHD Super-Resolution

Sumin Lee; Ki Beom Lee; Sunghwan Joo; Hong Keun Ahn; Junghyup Lee; Dohyung Kim; Bumsub Ham; Seong Ook Jung

doi:10.1109/ESSCIRC55480.2022.9911509

SIF-NPU: A 28nm 3.48 TOPS/W 0.25 TOPS/mm²CNN Accelerator with Spatially Independent Fusion for Real-Time UHD Super-Resolution

Sumin Lee, Ki Beom Lee, Sunghwan Joo, Hong Keun Ahn, Junghyup Lee, Dohyung Kim, Bumsub Ham, Seong Ook Jung

Department of Electrical and Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Citations (Scopus)

Abstract

This paper proposes a convolutional neural network (CNN)-based super-resolution accelerator for up-scaling to ultra-HD (UHD) resolution in real-Time in edge devices. A novel error-compensated bit quantization is adopted to reduce bit depth in the SR task. Spatially independent layer fusion is exploited to satisfy high throughput requirements at UHD resolution by increasing parallelism. Burst operation with write mask in the dual-port SRAM increases the process element utilization by allowing the concurrent multi-Access without exploiting additional memory. The accelerator is implemented in the 28nm technology and shows at least 4.3 times higher {FoM}({TOPS}/{mm}{2}\times {TOPS/W)} of 0.87 than the state-of-Art CNN accelerators. The implemented accelerator supports up-scaling up to 96 frames-per-seconds in UHD resolution.

Original language	English
Title of host publication	ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	97-100
Number of pages	4
ISBN (Electronic)	9781665484947
DOIs	https://doi.org/10.1109/ESSCIRC55480.2022.9911509
Publication status	Published - 2022
Event	48th IEEE European Solid State Circuits Conference, ESSCIRC 2022 - Milan, Italy Duration: 2022 Sept 19 → 2022 Sept 22

Publication series

Name	ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings

Conference

Conference	48th IEEE European Solid State Circuits Conference, ESSCIRC 2022
Country/Territory	Italy
City	Milan
Period	22/9/19 → 22/9/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

All Science Journal Classification (ASJC) codes

Hardware and Architecture
Electrical and Electronic Engineering
Electronic, Optical and Magnetic Materials
Instrumentation
Artificial Intelligence

Access to Document

10.1109/ESSCIRC55480.2022.9911509

Cite this

Lee, S., Lee, K. B., Joo, S., Ahn, H. K., Lee, J., Kim, D., Ham, B., & Jung, S. O. (2022). SIF-NPU: A 28nm 3.48 TOPS/W 0.25 TOPS/mm²CNN Accelerator with Spatially Independent Fusion for Real-Time UHD Super-Resolution. In ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings (pp. 97-100). (ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ESSCIRC55480.2022.9911509

Lee, Sumin ; Lee, Ki Beom ; Joo, Sunghwan et al. / SIF-NPU : A 28nm 3.48 TOPS/W 0.25 TOPS/mm²CNN Accelerator with Spatially Independent Fusion for Real-Time UHD Super-Resolution. ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 97-100 (ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings).

@inproceedings{1bd57cdb2f20478ebd70ef4747ae9c18,

title = "SIF-NPU: A 28nm 3.48 TOPS/W 0.25 TOPS/mm2CNN Accelerator with Spatially Independent Fusion for Real-Time UHD Super-Resolution",

abstract = "This paper proposes a convolutional neural network (CNN)-based super-resolution accelerator for up-scaling to ultra-HD (UHD) resolution in real-Time in edge devices. A novel error-compensated bit quantization is adopted to reduce bit depth in the SR task. Spatially independent layer fusion is exploited to satisfy high throughput requirements at UHD resolution by increasing parallelism. Burst operation with write mask in the dual-port SRAM increases the process element utilization by allowing the concurrent multi-Access without exploiting additional memory. The accelerator is implemented in the 28nm technology and shows at least 4.3 times higher {FoM}({TOPS}/{mm}{2}\times {TOPS/W)} of 0.87 than the state-of-Art CNN accelerators. The implemented accelerator supports up-scaling up to 96 frames-per-seconds in UHD resolution.",

keywords = "AI accelerator, Bit quantization, Convolutional neural network, Dataflow, Super-resolution",

author = "Sumin Lee and Lee, {Ki Beom} and Sunghwan Joo and Ahn, {Hong Keun} and Junghyup Lee and Dohyung Kim and Bumsub Ham and Jung, {Seong Ook}",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 48th IEEE European Solid State Circuits Conference, ESSCIRC 2022 ; Conference date: 19-09-2022 Through 22-09-2022",

year = "2022",

doi = "10.1109/ESSCIRC55480.2022.9911509",

language = "English",

series = "ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "97--100",

booktitle = "ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings",

address = "United States",

}

Lee, S, Lee, KB, Joo, S, Ahn, HK, Lee, J, Kim, D, Ham, B & Jung, SO 2022, SIF-NPU: A 28nm 3.48 TOPS/W 0.25 TOPS/mm²CNN Accelerator with Spatially Independent Fusion for Real-Time UHD Super-Resolution. in ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings. ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings, Institute of Electrical and Electronics Engineers Inc., pp. 97-100, 48th IEEE European Solid State Circuits Conference, ESSCIRC 2022, Milan, Italy, 22/9/19. https://doi.org/10.1109/ESSCIRC55480.2022.9911509

SIF-NPU: A 28nm 3.48 TOPS/W 0.25 TOPS/mm²CNN Accelerator with Spatially Independent Fusion for Real-Time UHD Super-Resolution. / Lee, Sumin; Lee, Ki Beom; Joo, Sunghwan et al.
ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc., 2022. p. 97-100 (ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - SIF-NPU

T2 - 48th IEEE European Solid State Circuits Conference, ESSCIRC 2022

AU - Lee, Sumin

AU - Lee, Ki Beom

AU - Joo, Sunghwan

AU - Ahn, Hong Keun

AU - Lee, Junghyup

AU - Kim, Dohyung

AU - Ham, Bumsub

AU - Jung, Seong Ook

PY - 2022

Y1 - 2022

N2 - This paper proposes a convolutional neural network (CNN)-based super-resolution accelerator for up-scaling to ultra-HD (UHD) resolution in real-Time in edge devices. A novel error-compensated bit quantization is adopted to reduce bit depth in the SR task. Spatially independent layer fusion is exploited to satisfy high throughput requirements at UHD resolution by increasing parallelism. Burst operation with write mask in the dual-port SRAM increases the process element utilization by allowing the concurrent multi-Access without exploiting additional memory. The accelerator is implemented in the 28nm technology and shows at least 4.3 times higher {FoM}({TOPS}/{mm}{2}\times {TOPS/W)} of 0.87 than the state-of-Art CNN accelerators. The implemented accelerator supports up-scaling up to 96 frames-per-seconds in UHD resolution.

AB - This paper proposes a convolutional neural network (CNN)-based super-resolution accelerator for up-scaling to ultra-HD (UHD) resolution in real-Time in edge devices. A novel error-compensated bit quantization is adopted to reduce bit depth in the SR task. Spatially independent layer fusion is exploited to satisfy high throughput requirements at UHD resolution by increasing parallelism. Burst operation with write mask in the dual-port SRAM increases the process element utilization by allowing the concurrent multi-Access without exploiting additional memory. The accelerator is implemented in the 28nm technology and shows at least 4.3 times higher {FoM}({TOPS}/{mm}{2}\times {TOPS/W)} of 0.87 than the state-of-Art CNN accelerators. The implemented accelerator supports up-scaling up to 96 frames-per-seconds in UHD resolution.

KW - AI accelerator

KW - Bit quantization

KW - Convolutional neural network

KW - Dataflow

KW - Super-resolution

UR - http://www.scopus.com/inward/record.url?scp=85141491844&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85141491844&partnerID=8YFLogxK

U2 - 10.1109/ESSCIRC55480.2022.9911509

DO - 10.1109/ESSCIRC55480.2022.9911509

M3 - Conference contribution

AN - SCOPUS:85141491844

T3 - ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings

SP - 97

EP - 100

BT - ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 19 September 2022 through 22 September 2022

ER -

Lee S, Lee KB, Joo S, Ahn HK, Lee J, Kim D et al. SIF-NPU: A 28nm 3.48 TOPS/W 0.25 TOPS/mm²CNN Accelerator with Spatially Independent Fusion for Real-Time UHD Super-Resolution. In ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc. 2022. p. 97-100. (ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings). doi: 10.1109/ESSCIRC55480.2022.9911509