Real-Time Object Detection System with Multi-Path Neural Networks

Seonyeong Heo, Sungjun Cho, Youngsok Kim, Hanjun Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

39 Citations (Scopus)

Abstract

Thanks to the recent advances in Deep Neural Networks (DNNs), DNN-based object detection systems become highly accurate and widely used in real-time environments such as autonomous vehicles, drones and security robots. Although the systems should detect objects within a certain time limit that can vary depending on their execution environments such as vehicle speeds, existing systems blindly execute the entire long-latency DNNs without reflecting the time-varying time limits, and thus they cannot guarantee real-time constraints. This work proposes a novel real-time object detection system that employs multipath neural networks based on a new worst-case execution time (WCET) model for DNNs on a GPU. This work designs the WCET model for a single DNN layer analyzing processor and memory contention on GPUs, and extends the WCET model to the end-to-end networks. This work also designs the multipath networks with three new operators such as skip, switch, and dynamic generate proposals that dynamically change their execution paths and the number of target objects. Finally, this work proposes a path decision model that chooses the optimal execution path at run-time reflecting dynamically changing environments and time constraints. Our detailed evaluation using widely-used driving datasets shows that the proposed real-time object detection system performs as good as a baseline object detection system without violating the time-varying time limits. Moreover, the WCET model predicts the worst-case execution latency of convolutional and group normalization layers with only 27% and 81% errors on average, respectively.

Original languageEnglish
Title of host publicationProceedings - 2020 IEEE Real-Time and Embedded Technology and Applications Symposium, RTAS 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages174-187
Number of pages14
ISBN (Electronic)9781728154992
DOIs
Publication statusPublished - 2020 Apr
Event26th IEEE Real-Time and Embedded Technology and Applications Symposium, RTAS 2020 - Sydney, Australia
Duration: 2020 Apr 212020 Apr 24

Publication series

NameProceedings of the IEEE Real-Time and Embedded Technology and Applications Symposium, RTAS
Volume2020-April
ISSN (Print)1545-3421

Conference

Conference26th IEEE Real-Time and Embedded Technology and Applications Symposium, RTAS 2020
Country/TerritoryAustralia
CitySydney
Period20/4/2120/4/24

Bibliographical note

Publisher Copyright:
© 2020 IEEE.

All Science Journal Classification (ASJC) codes

  • General Engineering

Fingerprint

Dive into the research topics of 'Real-Time Object Detection System with Multi-Path Neural Networks'. Together they form a unique fingerprint.

Cite this