Scones: Towards Conversational Authoring of Sketches
Iteratively refining and critiquing sketches are crucial steps to developing effective designs. We introduce Scones, a mixed-initiative, machine-learning-driven system that enables users to iteratively author sketches from text instructions. Scones is a novel deep-learning-based system that iteratively generates scenes of sketched objects composed with semantic specifications from natural language. Scones exceeds state-of-the-art performance on a text-based scene modification task, and introduces a mask-conditioned sketching model that can generate sketches with poses specified by high-level scene information. In an exploratory user evaluation of Scones, participants reported enjoying an iterative drawing task with Scones, and suggested additional features for further applications. We believe Scones is an early step towards automated, intelligent systems that support human-in-the-loop applications for communicating ideas through sketching in art and design.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Sketch-based Creativity Support Tools using Deep Learning
Sketching is a natural and effective visual communication medium commonly used in creative processes. Recent developments in deep-learning models drastically improved machines' ability in understanding and generating vis…
Deep LearningRetrievalJam or Cream First? Modeling Ambiguity in Neural Machine Translation with SCONES
The softmax layer in neural machine translation is designed to model the distribution over mutually exclusive tokens. Machine translation, however, is intrinsically uncertain: the same source sentence can have multiple s…
Machine TranslationMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATIONNMT+2Efficient network-guided multi-locus association mapping with graph cuts
As an increasing number of genome-wide association studies reveal the limitations of attempting to explain phenotypic heritability by single genetic loci, there is growing interest for associating complex phenotypes with…
Exploring Multimodal Prompt for Visualization Authoring with Large Language Models
Recent advances in large language models (LLMs) have shown great potential in automating the process of visualization authoring through simple natural language utterances. However, instructing LLMs using natural language…
Agentic Authoring of Interactive Multiview Visualizations in Genomics
Diverse genomics data, scientific questions, and analysis tasks typically demand highly specialized visualizations. Therefore, users often must customize or author new ones tailored to their data. Existing tools are usua…