3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows

2022-10-20 · Vivian Liu, Jo Vermeulen, George Fitzmaurice, Justin Matejka

Text-to-image AI are capable of generating novel images for inspiration, but their applications for 3D design workflows and how designers can build 3D models using AI-provided inspiration have not yet been explored. To investigate this, we integrated DALL-E, GPT-3, and CLIP within a CAD software in 3DALL-E, a plugin that generates 2D image inspiration for 3D design. 3DALL-E allows users to construct text and image prompts based on what they are modeling. In a study with 13 designers, we found that designers saw great potential in 3DALL-E within their workflows and could use text-to-image AI to produce reference images, prevent design fixation, and inspire design considerations. We elaborate on prompting patterns observed across 3D modeling tasks and provide measures of prompt complexity observed across participants. From our findings, we discuss how 3DALL-E can merge with existing generative design workflows and propose prompt bibliographies as a form of human-AI design history.

📄 PDF Abstract BibTeX arXiv:2210.11603

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

15 Ways to Contact How can i speak to someone at Delta Airlines 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics

2022-10-05 · Ivan Kapelyukh, Vitalis Vosylius, Edward Johns

We introduce the first work to explore web-scale diffusion models for robotics. DALL-E-Bot enables a robot to rearrange objects in a scene, by first inferring a text description of those objects, then generating an image…

DALL-M: Context-Aware Clinical Data Augmentation with LLMs

2024-07-11 · Chihcheng Hsieh, Catarina Moreira, Isabel Blanco Nobre, Sandra Costa Sousa 외

X-ray images are vital in medical diagnostics, but their effectiveness is limited without clinical context. Radiologists often find chest X-rays insufficient for diagnosing underlying diseases, necessitating the integrat…

Data Augmentation Data Integration

DEsignBench: Exploring and Benchmarking DALL-E 3 for Imagining Visual Design

2023-10-23 · Kevin Lin, Zhengyuan Yang, Linjie Li, JianFeng Wang 외

We introduce DEsignBench, a text-to-image (T2I) generation benchmark tailored for visual design scenarios. Recent T2I models like DALL-E 3 and others, have demonstrated remarkable capabilities in generating photorealisti…

Benchmarking Image Generation

A very preliminary analysis of DALL-E 2

2022-04-25 · Gary Marcus, Ernest Davis, Scott Aaronson

The DALL-E 2 system generates original synthetic images corresponding to an input text as caption. We report here on the outcome of fourteen tests of this system designed to assess its common sense, reasoning and ability…

Common Sense Reasoning

DiffKendall: A Novel Approach for Few-Shot Learning with Differentiable Kendall's Rank Correlation

2023-07-28 · NeurIPS 2023 11

Few-shot learning aims to adapt models trained on the base dataset to novel tasks where the categories were not seen by the model before. This often leads to a relatively uniform distribution of feature values across cha…

Few-Shot Image Classification Few-Shot Learning