Detailed real-time urban 3D reconstruction from video

M. Pollefeys, D. Nistér, J. M. Frahm, A. Akbarzadeh, P. Mordohai, B. Clipp, C. Engels, D. Gallup, S. J. Kim, P. Merrell, C. Salmi, S. Sinha, B. Talton, L. Wang, Q. Yang, H. Stewénius, R. Yang, G. Welch, H. Towles

Research output: Contribution to journalArticlepeer-review

615 Citations (Scopus)


The paper presents a system for automatic, geo-registered, real-time 3D reconstruction from video of urban scenes. The system collects video streams, as well as GPS and inertia measurements in order to place the reconstructed models in geo-registered coordinates. It is designed using current state of the art real-time modules for all processing steps. It employs commodity graphics hardware and standard CPU's to achieve real-time performance. We present the main considerations in designing the system and the steps of the processing pipeline. Our system extends existing algorithms to meet the robustness and variability necessary to operate out of the lab. To account for the large dynamic range of outdoor videos the processing pipeline estimates global camera gain changes in the feature tracking stage and efficiently compensates for these in stereo estimation without impacting the real-time performance. The required accuracy for many applications is achieved with a two-step stereo reconstruction process exploiting the redundancy across frames. We show results on real video sequences comprising hundreds of thousands of frames.

Original languageEnglish
Pages (from-to)143-167
Number of pages25
JournalInternational Journal of Computer Vision
Issue number2-3
Publication statusPublished - 2008 Jul

Bibliographical note

Funding Information:
Acknowledgements We gratefully acknowledge the support of the DARPA UrbanScape project as well as the support of the DTO VACE project “3D Content Extraction from Video Streams”.

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence


Dive into the research topics of 'Detailed real-time urban 3D reconstruction from video'. Together they form a unique fingerprint.

Cite this