Semi-supervised adversarial learning to generate photorealistic face images of new identities from 3D morphable model

Baris Gecer, Binod Bhattarai, Josef Kittler, Tae-Kyun Kim

Research output: Chapter in Book/Report/Conference proceedingPublished conference contribution


We propose a novel end-to-end semi-supervised adversarial framework to generate photorealistic face images of new identities with a wide range of expressions, poses, and illuminations conditioned by synthetic images sampled from a 3D morphable model. Previous adversarial style-transfer methods either supervise their networks with a large volume of paired data or train highly under-constrained two-way generative networks in an unsupervised fashion. We propose a semi-supervised adversarial learning framework to constrain the two-way networks by a small number of paired real and synthetic images, along with a large volume of unpaired data. A set-based loss is also proposed to preserve identity coherence of generated images. Qualitative results show that generated face images of new identities contain pose, lighting and expression diversity. They are also highly constrained by the synthetic input images while adding photorealism and retaining identity information. We combine face images generated by the proposed method with a real data set to train face recognition algorithms and evaluate the model quantitatively on two challenging data sets: LFW and IJB-A. The generated images by our framework consistently improve the performance of deep face recognition networks trained with the Oxford VGG Face dataset, and achieve comparable results to the state-of-the-art.
Original languageEnglish
Title of host publicationECCV 2018; European Conference on Computer Vision
Number of pages18
Publication statusPublished - 2018

Publication series

NameLecture Notes in Computer Science

Bibliographical note

This work was supported by the EPSRC Programme Grant ‘FACER2VM’ (EP/N007743/1). We would like to thank Microsoft Research for their support with Microsoft Azure Research Award. Baris Gecer is funded by the Turkish Ministry of National Education. This study is morally motivated to improve face recognition to help prediction of genetic disorders visible on human face in earlier stages.


Dive into the research topics of 'Semi-supervised adversarial learning to generate photorealistic face images of new identities from 3D morphable model'. Together they form a unique fingerprint.

Cite this