"Stack It Up!": 3D Stable Structure Generation from 2D Hand-drawn Sketch

2025-08-04 · Yiqing Xu, Linfeng Li, Cunjun Yu, David Hsu arxiv

Imagine a child sketching the Eiffel Tower and asking a robot to bring it to life. Today's robot manipulation systems can't act on such sketches directly-they require precise 3D block poses as goals, which in turn demand structural analysis and expert tools like CAD. We present StackItUp, a system that enables non-experts to specify complex 3D structures using only 2D front-view hand-drawn sketches. StackItUp introduces an abstract relation graph to bridge the gap between rough sketches and accurate 3D block arrangements, capturing the symbolic geometric relations (e.g., left-of) and stability patterns (e.g., two-pillar-bridge) while discarding noisy metric details from sketches. It then grounds this graph to 3D poses using compositional diffusion models and iteratively updates it by predicting hidden internal and rear supports-critical for stability but absent from the sketch. Evaluated on sketches of iconic landmarks and modern house designs, StackItUp consistently produces stable, multilevel 3D structures and outperforms all baselines in both stability and visual resemblance.

📄 PDF Abstract BibTeX arXiv:2508.02093

Code (0)

등록된 구현이 없습니다.

Tasks

Robot Manipulation

MRReP: Mixed Reality-based Hand-drawn Reference Path Editing Interface for Mobile Robot Navigation

2026-03-31 · Takumi Taki, Masato Kobayashi, Yuki Uranishi arxiv

Autonomous mobile robots operating in human-shared indoor environments often require paths that reflect human spatial intentions, such as avoiding interference with pedestrian flow or maintaining comfortable clearance. H…

Robot Navigation

Sketch2MinSurf: Vision-Language Guided Generation of Editable Minimal Surfaces from Hand-Drawn Sketches

2026-05-20 · Wenda Wang, Anqi Liu, Junqi Yang, Lei He 외 arxiv

Converting hand-drawn sketches into structured 3D geometries remains challenging due to the difficulty of representing non-Euclidean surfaces and maintaining topological consistency. Existing generative models such as GA…

VISTA: An End-to-End Benchmark for Visual Spec-to-Web-App Coding Agents

2026-05-22 · JunJia Guo, Yuhang Yao, Jiawei, Zhou 외 arxiv

We present VISTA (VIsual Spec-To-App Benchmark), a benchmark for evaluating the end-to-end web-app generation capabilities of LLM-based agents. Unlike prior code generation benchmarks that focus on algorithmic tasks, VIS…

Code Generation

SketchAgent: Generating Structured Diagrams from Hand-Drawn Sketches

2025-08-02 · Cheng Tan, Qi Chen, Jingxuan Wei, Gaowei Wu 외 arxiv

Hand-drawn sketches are a natural and efficient medium for capturing and conveying ideas. Despite significant advancements in controllable natural image generation, translating freehand sketches into structured, machine-…

Image Generation

StableSketcher: Enhancing Diffusion Model for Pixel-based Sketch Generation via Visual Question Answering Feedback

2025-10-23 · Jiho Park, Sieun Choi, Jaeyoon Seo, Jihie Kim arxiv

Although recent advancements in diffusion models have significantly enriched the quality of generated images, challenges remain in synthesizing pixel-based human-drawn sketches, a representative example of abstract expre…

Visual Question Answering Reinforcement Learning