Abstract
Manipulating facial expressions is a challenging task due to fine-grained shape changes produced by facial muscles and the lack of input-output pairs for supervised learning. Unlike previous methods using Generative Adversarial Networks (GAN), which rely on cycle-consistency loss or sparse geometry (landmarks) loss for expression synthesis, we propose a novel GAN framework to exploit 3D dense (depth and surface normals) information for expression manipulation. However, a large-scale dataset containing RGB images with expression annotations and their corresponding depth maps is not available. To this end, we propose to use an off-the-shelf state-of-the-art 3D reconstruction model to estimate the depth and create a large-scale RGB-Depth dataset after a manual data clean-up process. We utilise this dataset to minimise the novel depth consistency loss via adversarial learning (note we do not have ground truth depth maps for generated face images) and the depth categorical loss of synthetic data on the discriminator. In addition, to improve the generalisation and lower the bias of the depth parameters, we propose to use a novel confidence regulariser on the discriminator side of the framework. We extensively performed both quantitative and qualitative evaluations on two publicly available challenging facial expression benchmarks: AffectNet and RaFD. Our experiments demonstrate that the proposed method outperforms the competitive baseline and existing arts by a large margin.
Original language | English |
---|---|
Title of host publication | 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) |
Publisher | IEEE Explore |
Number of pages | 10 |
ISBN (Electronic) | 978-1-6654-0477-8 |
ISBN (Print) | 978-1-6654-4640-2 |
DOIs | |
Publication status | Published - 14 Jun 2021 |
Event | 2021 IEEE Winter Conference on Applications of Computer Vision - Virtual Duration: 3 Jan 2021 → 8 Jan 2021 https://wacv2021.thecvf.com/home |
Publication series
Name | IEEE Workshop on Applications of Computer Vision |
---|---|
Publisher | IEEE |
ISSN (Print) | 2472-6737 |
ISSN (Electronic) | 2642-9381 |
Conference
Conference | 2021 IEEE Winter Conference on Applications of Computer Vision |
---|---|
Abbreviated title | WACV |
Period | 3/01/21 → 8/01/21 |
Internet address |