Text-Free Learning of a Natural Language Interface for Pretrained Face Generators
We propose Fast text2StyleGAN, a natural language interface that adapts pre-trained GANs for text-guided human face synthesis. Leveraging the recent advances in Contrastive Language-Image Pre-training (CLIP), no text data is required during training. Fast text2StyleGAN is formulated as a conditional variational autoencoder (CVAE) that provides extra control and diversity to the generated images at test time. Our model does not require re-training or fine-tuning of the GANs or CLIP when encountering new text prompts. In contrast to prior work, we do not rely on optimization at test time, making our method orders of magnitude faster than prior work. Empirically, on FFHQ dataset, our method offers faster and more accurate generation of images from natural language descriptions with varying levels of detail compared to prior work.
Code (1)
Tasks
DiversityFace GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
On joint training with interfaces for spoken language understanding
Spoken language understanding (SLU) systems extract both text transcripts and semantics associated with intents and slots from input speech utterances. SLU systems usually consist of (1) an automatic speech recognition (…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Natural Language Understandingspeech-recognition+2Hands-Free VR
The paper introduces Hands-Free VR, a voice-based natural-language interface for VR. The user gives a command using their voice, the speech audio data is converted to text using a speech-to-text deep learning model that …
DiversityLanguage ModellingLarge Language ModelSpeech-to-TextTowards Neural Programming Interfaces
It is notoriously difficult to control the behavior of artificial neural networks such as generative neural language models. We recast the problem of controlling natural language generation as that of learning to interfa…
Language ModelingLanguage ModellingText GenerationGestureGPT: Toward Zero-Shot Free-Form Hand Gesture Understanding with Large Language Model Agents
Existing gesture interfaces only work with a fixed set of gestures defined either by interface designers or by users themselves, which introduces learning or demonstration efforts that diminish their naturalness. Humans,…
Common Sense ReasoningFormGesture RecognitionLanguage Modeling+3When Machines Speak: A Unified Generative Framework for Integrating Machine-Native Symbols into Pretrained Large Language Models
Many real-world AI systems represent entities, behaviors, and structured information using discrete machine-native symbols rather than natural language. While these representations are compact and preserve task-relevant …
Sequential RecommendationStructured Prediction