Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance
In this work we introduce Deforming Autoencoders, a generative model for
images that disentangles shape from appearance in an unsupervised manner. As in
the deformable template paradigm, shape is represented as a deformation between
a canonical coordinate system (template') and an observed image, while
appearance is modeled in canonical', template, coordinates, thus discarding
variability due to deformations. We introduce novel techniques that allow this
approach to be deployed in the setting of autoencoders and show that this
method can be used for unsupervised group-wise image alignment. We show
experiments with expression morphing in humans, hands, and digits, face
manipulation, such as shape and appearance interpolation, as well as
unsupervised landmark localization. A more powerful form of unsupervised
disentangling becomes possible in template coordinates, allowing us to
successfully decompose face images into shading and albedo, and further
manipulate face images.
Code (2)
Tasks
Unsupervised Facial Landmark DetectionSimilar Papers 제목 키워드 기반
Unsupervised Part-Based Disentangling of Object Shape and Appearance
Large intra-class variation is the result of changes in multiple object characteristics. Images, however, only show the superposition of different variable factors such as appearance or shape. Therefore, learning to dise…
Appearance TransferImage GenerationObjectPose Prediction+3Unsupervised Part Segmentation through Disentangling Appearance and Shape
We study the problem of unsupervised discovery and segmentation of object parts, which, as an intermediate local representation, are capable of finding intrinsic object structure and providing more explainable recognitio…
DisentanglementObjectSegmentationSemantic Segmentation+1EndoSurf: Neural Surface Reconstruction of Deformable Tissues with Stereo Endoscope Videos
Reconstructing soft tissues from stereo endoscope videos is an essential prerequisite for many medical applications. Previous methods struggle to produce high-quality geometry and appearance due to their inadequate repre…
Surface ReconstructionTransfer Learning using Spectral Convolutional Autoencoders on Semi-Regular Surface Meshes
The underlying dynamics and patterns of 3D surface meshes deforming over time can be discovered by unsupervised learning, especially autoencoders, which calculate low-dimensional embeddings of the surfaces. To study the …
Transfer LearningRealistic Face Reenactment via Self-Supervised Disentangling of Identity and Pose
Recent works have shown how realistic talking face images can be obtained under the supervision of geometry guidance, e.g., facial landmark or boundary. To alleviate the demand for manual annotations, in this paper, we p…
Face Reenactment