End-to-End Steering Controller with CNN-based Closed-loop Feedback for Autonomous Vehicles

Junekyo Jhung, Il Bae, Jaeyoung Moon, Taewoo Kim, Jincheol Kim, Shiho Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

27 Citations (Scopus)


Many significant research achievements in the past few decades have demonstrated that convolutional neural networks (CNNs) could be capable of steering wheel control which is the basic and essential maneuver of autonomous vehicles. We propose an end-to-end steering controller with CNN-based closed-loop feedback for autonomous vehicles that improves driving performance compared to traditional CNN-based approaches. This paper demonstrates that the proposed neural network, DAVE-2SKY, is able to learn to inference steering wheel angles for the lateral control of self-driving vehicles through initial supervised pre-training and subsequent reinforced closed-loop post-training with images from a camera mounted on the vehicle. We used the PreScan simulator and Caffe deep learning framework for training under diversified circumstances in a software in the loop (SIL) simulation environment. We used DRIVE TM PX2 computer to implement a self-driving car for an experimental validation of the proposed end-to-end controller. The performance of the proposed system has been investigated under simulations and on-road tests as well. This work shows that the CNN-based end-to-end controller performs robust steering control even under partially observable road conditions, which indicates the possibility of fully self-driving vehicles controlled by CNN-based end-to-end steering controllers.

Original languageEnglish
Title of host publication2018 IEEE Intelligent Vehicles Symposium, IV 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9781538644522
Publication statusPublished - 2018 Oct 18
Event2018 IEEE Intelligent Vehicles Symposium, IV 2018 - Changshu, Suzhou, China
Duration: 2018 Sept 262018 Sept 30

Publication series

NameIEEE Intelligent Vehicles Symposium, Proceedings


Other2018 IEEE Intelligent Vehicles Symposium, IV 2018
CityChangshu, Suzhou

Bibliographical note

Funding Information:
This work was supported by the MSIT (Ministry of Science and ICT), Korea, under the ICT Consilience Creative program (IITP-2017-2017-0-01015) supervised by the IITP (Institute for Information & communications Technology Promotion). The authors performed this work as a part of research projects of SKT-Yonsei Cooperative Autonomous Driving Research Center under the SKT-Yonsei Global Talent Fostering Program supported by the SK Telecom ICT R&D Center.

Publisher Copyright:
© 2018 IEEE.

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Automotive Engineering
  • Modelling and Simulation


Dive into the research topics of 'End-to-End Steering Controller with CNN-based Closed-loop Feedback for Autonomous Vehicles'. Together they form a unique fingerprint.

Cite this