Technical Report MSC-2017-11

Title: Learning to Reconstruct Face Geometries
Authors: Elad Richardson
Supervisors: Ron Kimmel
Abstract: Facial geometry reconstruction is a long researched problem with numerous applications in computer vision and graphics. While the problem might appear simple at a first glance, it has proved to be challenging due to the extensive variation human faces exhibit when considering expressions, poses, textures, and intrinsic geometry. Many approaches tackle this complexity by using multiple sources as input, thus introducing more constraints to the problem. Still, the ill-posed scenario of extracting the facial geometry from a single image remains a difficult problem, where current methods can generally provide only coarse estimations of the facial geometry. In this thesis, we propose to leverage the power of convolutional neural networks (CNNs) to produce a highly detailed face reconstruction from a single image. To that end, we introduce a CNN framework which derives the geometry in a coarse-to-fine fashion. The proposed architecture is composed of two blocks, a network that recovers the coarse facial geometry (CoarseNet), followed by a CNN that refines the facial features of that geometry (FineNet), where the networks are connected using a novel rendering layer. While deep neural networks have proved themselves over complex computer vision tasks, they are heavily dependent on the data, usually requiring an extremely large dataset of annotated examples. Unfortunately, unlike object recognition and detection problems, there are no suitable datasets for training CNNs to perform face geometry reconstruction. To overcome that, we turn to our understanding of image formation and facial geometry models to propose a specialized training regime that uses only synthetic images and unlabeled in-the-wild facial images. The accuracy and robustness of the proposed model are demonstrated by both qualitative and quantitative evaluation tests.

CopyrightThe above paper is copyright by the Technion, Author(s), or others. Please contact the author(s) for more information

Remark: Any link to this technical report should be to this page (, rather than to the URL of the PDF or PS files directly. The latter URLs may change without notice.

To the list of the MSC technical reports of 2017
To the main CS technical reports page

Computer science department, Technion