SnakeVoxFormer: Transformer-based Single Image\\Voxel Reconstruction with Run Length Encoding
Deep learning-based 3D object reconstruction has achieved unprecedented results. Among those, the transformer deep neural model showed outstanding performance in many applications of computer vision. We introduce SnakeVoxFormer, a novel, 3D object reconstruction in voxel space from a single image using the transformer. The input to SnakeVoxFormer is a 2D image, and the result is a 3D voxel model. The key novelty of our approach is in using the run-length encoding that traverses (like a snake) the voxel space and encodes wide spatial differences into a 1D structure that is suitable for transformer encoding. We then use dictionary encoding to convert the discovered RLE blocks into tokens that are used for the transformer. The 1D representation is a lossless 3D shape data compression method that converts to 1D data that use only about 1% of the original data size. We show how different voxel traversing strategies affect the effect of encoding and reconstruction. We compare our method with the state-of-the-art for 3D voxel reconstruction from images and our method improves the state-of-the-art methods by at least 2.8% and up to 19.8%.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Object ReconstructionData CompressionObject ReconstructionSimilar Papers 제목 키워드 기반
R3D-SWIN:Use Shifted Window Attention for Single-View 3D Reconstruction
Recently, vision transformers have performed well in various computer vision tasks, including voxel 3D reconstruction. However, the windows of the vision transformer are not multi-scale, and there is no connection betwee…
3D ReconstructionSingle-View 3D Reconstruction3D-RETR: End-to-End Single and Multi-View 3D Reconstruction with Transformers
3D reconstruction aims to reconstruct 3D objects from 2D views. Previous works for 3D reconstruction mainly focus on feature matching between views or using CNNs as backbones. Recently, Transformers have been shown effec…
3D ReconstructionDecoderMulti-View 3D ReconstructionVoxel-based 3D Detection and Reconstruction of Multiple Objects from a Single Image
Inferring 3D locations and shapes of multiple objects from a single 2D image is a long-standing objective of computer vision. Most of the existing works either predict one of these 3D properties or focus on solving both …
Keypoint DetectionObjectBUOL: A Bottom-Up Framework with Occupancy-aware Lifting for Panoptic 3D Scene Reconstruction From A Single Image
Understanding and modeling the 3D scene from a single image is a practical problem. A recent advance proposes a panoptic 3D scene reconstruction task that performs both 3D reconstruction and 3D panoptic segmentation from…
3D Panoptic Segmentation3D Reconstruction3D Scene ReconstructionPanoptic SegmentationVoxelFormer: Parameter-Efficient Multi-Subject Visual Decoding from fMRI
Recent advances in fMRI-based visual decoding have enabled compelling reconstructions of perceived images. However, most approaches rely on subject-specific training, limiting scalability and practical deployment. We int…