Hierarchical depth estimation for image synthesis in mixed reality

Han Sung Kim, Kwanghoon Sohn

Research output: Contribution to journalConference articlepeer-review

11 Citations (Scopus)


Mixed reality is different from the virtual reality in that users can feel immersed in a space which is composed of not only virtual but also real objects. Thus, it is essential to realize seamless integration and mutual occlusion of the virtual and real worlds. Therefore, we need depth information of the real scene to perform the synthesis. We propose the depth estimation algorithm with sharp object boundaries for mixed reality system based on hierarchical disparity estimation. Initial disparity vectors are obtained from downsampled stereo images using region-dividing disparity estimation technique. Then, background region is detected and flattened. With these initial vectors, dense disparities are estimated and regularized with shape-adaptive window in full resolution images. Finally, depth values are calculated by stereo geometry and camera parameters. As a result, virtual objects can be mixed into the image of real world by comparing the calculated depth values with the depth information of generated virtual objects. Experimental results show that occlusion between the virtual and real objects are correctly established with sharp boundaries in the synthesized images, so that user can observe the mixed scene with considerably natural sensation.

Original languageEnglish
Pages (from-to)544-553
Number of pages10
JournalProceedings of SPIE - The International Society for Optical Engineering
Publication statusPublished - 2003
EventStereoscopic Displays and Virtual Reality Systems X - Santa Clara, CA, United States
Duration: 2003 Jan 212003 Jan 24

All Science Journal Classification (ASJC) codes

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering


Dive into the research topics of 'Hierarchical depth estimation for image synthesis in mixed reality'. Together they form a unique fingerprint.

Cite this