PlankAssembly: Robust 3D Reconstruction from Three Orthographic Views with Learnt Shape Programs
In this paper, we develop a new method to automatically convert 2D line drawings from three orthographic views into 3D CAD models. Existing methods for this problem reconstruct 3D models by back-projecting the 2D observations into 3D space while maintaining explicit correspondence between the input and output. Such methods are sensitive to errors and noises in the input, thus often fail in practice where the input drawings created by human designers are imperfect. To overcome this difficulty, we leverage the attention mechanism in a Transformer-based sequence generation model to learn flexible mappings between the input and output. Further, we design shape programs which are suitable for generating the objects of interest to boost the reconstruction accuracy and facilitate CAD modeling applications. Experiments on a new benchmark dataset show that our method significantly outperforms existing ones when the inputs are noisy or incomplete.
Code (1)
Tasks
3D ReconstructionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
GaussianCAD: Robust Self-Supervised CAD Reconstruction from Three Orthographic Views Using 3D Gaussian Splatting
The automatic reconstruction of 3D computer-aided design (CAD) models from CAD sketches has recently gained significant attention in the computer vision community. Most existing methods, however, rely on vector CAD sketc…
3D ReconstructionCAD ReconstructionCamera Pose EstimationPose EstimationReconstructing vehicles from orthographic drawings using deep neural networks
This paper explores the current state-of-the-art of object reconstruction from multiple orthographic drawings using deep neural networks. It proposes two algorithms to extract multiple views from a single image. The pape…
Object ReconstructionregressionFront2Back: Single View 3D Shape Reconstruction via Front to Back Prediction
Reconstruction of a 3D shape from a single 2D image is a classical computer vision problem, whose difficulty stems from the inherent ambiguity of recovering occluded or only partially observed surfaces. Recent methods ad…
3D Shape ReconstructionSurface Reconstruction3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models
Current Large Language Models have achieved Olympiad-level logic, yet Vision-Language Models paradoxically falter on elementary spatial tasks like block counting. This capability mismatch reveals a critical ``spatial int…
Spatial ReasoningLearning variable length units for SMT between related languages via Byte Pair Encoding
We explore the use of segments learnt using Byte Pair Encoding (referred to as BPE units) as basic units for statistical machine translation between related languages and compare it with orthographic syllables, which are…
Machine TranslationTranslation