Local spherical harmonics for facial shape and albedo estimation

Suwoong Heo, Hyewon Song, Jiwoo Kang, Sanghoon Lee

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)

Abstract

In this paper, we present a novel facial albedo and 3D shape recovery method with a local spherical harmonic illumination model. From a face in an image, the proposed method can produce a high-quality 3D shape and albedo using a novel parameterization of local illuminations. Because a facial shape is partially convex, a single spherical harmonics is generally used for the illumination of a face within a constrained illumination environment. However, when a facial image is captured in an unconstrained scene, the illumination is inappropriately estimated due to the presence of shadow and specular reflections. To address this issue, we propose a novel local spherical harmonic illumination model for representing the illumination of a face. Unlike the existing parameterization of local illumination, our local spherical harmonic illumination model utilizes a smooth weight function for the seamless representation of natural illumination. Therefore, the albedo and shape information in an image can be precisely estimated using the first-order spherical harmonics. For accurate estimation of albedo, we also utilize facial albedo statistics to prevent the estimated albedo from becoming biased toward input image. Furthermore, we developed an accurate and reliable 3D shape reconstruction method from a normal map based on tetrahedron-based deformation. Comparing to the Laplacian deformation based method, our method is applicable to any mesh regardless of its structure. Through rigorous experiments, we demonstrate that the proposed local spherical harmonic illumination model is effective in estimating the complex illumination and can recover a high-quality facial albedo and 3D shape.

Original languageEnglish
Pages (from-to)177424-177436
Number of pages13
JournalIEEE Access
Volume8
DOIs
Publication statusPublished - 2020

Bibliographical note

Publisher Copyright:
© 2020 Institute of Electrical and Electronics Engineers Inc.. All rights reserved.

All Science Journal Classification (ASJC) codes

  • General Computer Science
  • General Materials Science
  • General Engineering
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Local spherical harmonics for facial shape and albedo estimation'. Together they form a unique fingerprint.

Cite this