Generating Diverse and Informative Natural Language Fashion Feedback
Recent advances in multi-modal vision and language tasks enable a new set of applications. In this paper, we consider the task of generating natural language fashion feedback on outfit images. We collect a unique dataset, which contains outfit images and corresponding positive and constructive fashion feedback. We treat each feedback type separately, and train deep generative encoder-decoder models with visual attention, similar to the standard image captioning pipeline. Following this approach, the generated sentences tend to be too general and non-informative. We propose an alternative decoding technique based on the Maximum Mutual Information objective function, which leads to more diverse and detailed responses. We evaluate our model with common language metrics, and also show human evaluation results. This technology is applied within the ``Alexa, how do I look?'' feature, publicly available in Echo Look devices.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderImage CaptioningSimilar Papers 제목 키워드 기반
Sub-GAN: An Unsupervised Generative Model via Subspaces
The recent years have witnessed significant growth in constructing robust generative models to capture informative distributions of natural data. However, it is difficult to fully exploit the distribution of complex data…
Generative Adversarial NetworkGenerating Long and Informative Reviews with Aspect-Aware Coarse-to-Fine Decoding
Generating long and informative review text is a challenging natural language generation task. Previous work focuses on word-level generation, neglecting the importance of topical and syntactic characteristics from natur…
DecoderReview GenerationSentenceText GenerationAn Equal-Size Hard EM Algorithm for Diverse Dialogue Generation
Open-domain dialogue systems aim to interact with humans through natural language texts in an open-ended fashion. Despite the recent success of super large dialogue systems such as ChatGPT, using medium-to-small-sized di…
DecoderDialogue GenerationFashionVQA: A Domain-Specific Visual Question Answering System
Humans apprehend the world through various sensory modalities, yet language is their predominant communication channel. Machine learning systems need to draw on the same multimodal richness to have informed discourses wi…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)Neural Language Generation: Formulation, Methods, and Evaluation
Recent advances in neural network-based generative modeling have reignited the hopes in having computer systems capable of seamlessly conversing with humans and able to understand natural language. Neural architectures h…
SurveyText Generation