Vector Grimoire: Codebook-based Shape Generation under Raster Image Supervision

2024-10-08 · Moritz Feuerpfeil, Marco Cipriano, Gerard de Melo

Scalable Vector Graphics (SVG) is a popular format on the web and in the design industry. However, despite the great strides made in generative modeling, SVG has remained underexplored due to the discrete and complex nature of such data. We introduce GRIMOIRE, a text-guided SVG generative model that is comprised of two modules: A Visual Shape Quantizer (VSQ) learns to map raster images onto a discrete codebook by reconstructing them as vector shapes, and an Auto-Regressive Transformer (ART) models the joint probability distribution over shape tokens, positions and textual descriptions, allowing us to generate vector graphics from natural language. Unlike existing models that require direct supervision from SVG data, GRIMOIRE learns shape image patches using only raster image supervision which opens up vector generative modeling to significantly more data. We demonstrate the effectiveness of our method by fitting GRIMOIRE for closed filled shapes on the MNIST and for outline strokes on icon and font data, surpassing previous image-supervised methods in generative quality and vector-supervised approach in flexibility.

📄 PDF Abstract BibTeX arXiv:2410.05991

Code (0)

등록된 구현이 없습니다.

Tasks

Vector Graphics

Methods 이 논문이 사용한 방법론

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Adam 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Residual Connection 설명 없음

Position-Wise Feed-Forward Layer 설명 없음

Attention 설명 없음

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Vector Quantization for Deep-Learning-Based CSI Feedback in Massive MIMO Systems

2024-03-12 · Junyong Shin, Yujin Kang, Yo-Seb Jeon

This paper presents a finite-rate deep-learning (DL)-based channel state information (CSI) feedback method for massive multiple-input multiple-output (MIMO) systems. The presented method provides a finite-bit representat…

Quantization

TAR3D: Creating High-Quality 3D Assets via Next-Part Prediction

2024-12-22 · Xuying Zhang, Yutong Liu, Yangguang Li, Renrui Zhang 외

We present TAR3D, a novel framework that consists of a 3D-aware Vector Quantized-Variational AutoEncoder (VQ-VAE) and a Generative Pre-trained Transformer (GPT) to generate high-quality 3D assets. The core insight of thi…

Image to 3D Text to 3D

Class-Partitioned VQ-VAE and Latent Flow Matching for Point Cloud Scene Generation

2026-01-18 · Dasith de Silva Edirimuni, Ajmal Saeed Mian arxiv

Most 3D scene generation methods are limited to only generating object bounding box parameters while newer diffusion methods also generate class labels and latent features. Using object size or latent feature, they then …

Point Cloud Generation Scene Generation

Scalable Training for Vector-Quantized Networks with 100% Codebook Utilization

2025-09-12 · Yifan Chang, Jie Qin, Limeng Qiao, Xiaofeng Wang 외 arxiv

Vector quantization (VQ) is a key component in discrete tokenizers for image generation, but its training is often unstable due to straight-through estimation bias, one-step-behind updates, and sparse codebook gradients,…

Image Generation

3DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Process

2023-03-18 · Yuhan Li, Yishun Dou, Xuanhong Chen, Bingbing Ni 외

We develop a generalized 3D shape generation prior model, tailored for multiple 3D tasks including unconditional shape generation, point cloud completion, and cross-modality shape generation, etc. On one hand, to precise…

3D Shape Generation Point Cloud Completion