High-Precision Depth Estimation Using Uncalibrated LiDAR and Stereo Fusion

Kihong Park, Seungryong Kim, Kwanghoon Sohn

Research output: Contribution to journalArticlepeer-review

45 Citations (Scopus)


We address the problem of 3D reconstruction from uncalibrated LiDAR point cloud and stereo images. Since the usage of each sensor alone for 3D reconstruction has weaknesses in terms of density and accuracy, we propose a deep sensor fusion framework for high-precision depth estimation. The proposed architecture consists of calibration network and depth fusion network, where both networks are designed considering the trade-off between accuracy and efficiency for mobile devices. The calibration network first corrects an initial extrinsic parameter to align the input sensor coordinate systems. The accuracy of calibration is markedly improved by formulating the calibration in the depth domain. In the depth fusion network, complementary characteristics of sparse LiDAR and dense stereo depth are then encoded in a boosting manner. Since training data for the LiDAR and stereo depth fusion are rather limited, we introduce a simple but effective approach to generate pseudo ground truth labels from the raw KITTI dataset. The experimental evaluation verifies that the proposed method outperforms current state-of-the-art methods on the KITTI benchmark. We also collect data using our proprietary multi-sensor acquisition platform and verify that the proposed method generalizes across different sensor settings and scenes.

Original languageEnglish
Article number8624583
Pages (from-to)321-335
Number of pages15
JournalIEEE Transactions on Intelligent Transportation Systems
Issue number1
Publication statusPublished - 2020 Jan

Bibliographical note

Publisher Copyright:
© 2000-2011 IEEE.

All Science Journal Classification (ASJC) codes

  • Automotive Engineering
  • Mechanical Engineering
  • Computer Science Applications


Dive into the research topics of 'High-Precision Depth Estimation Using Uncalibrated LiDAR and Stereo Fusion'. Together they form a unique fingerprint.

Cite this