Structure-Aware Sparse-View X-ray 3D Reconstruction

2023-11-18 · CVPR 2024 1 · Yuanhao Cai, Jiahao Wang, Alan Yuille, Zongwei Zhou, Angtian Wang

X-ray, known for its ability to reveal internal structures of objects, is expected to provide richer information for 3D reconstruction than visible light. Yet, existing neural radiance fields (NeRF) algorithms overlook this important nature of X-ray, leading to their limitations in capturing structural contents of imaged objects. In this paper, we propose a framework, Structure-Aware X-ray Neural Radiodensity Fields (SAX-NeRF), for sparse-view X-ray 3D reconstruction. Firstly, we design a Line Segment-based Transformer (Lineformer) as the backbone of SAX-NeRF. Linefomer captures internal structures of objects in 3D space by modeling the dependencies within each line segment of an X-ray. Secondly, we present a Masked Local-Global (MLG) ray sampling strategy to extract contextual and geometric information in 2D projection. Plus, we collect a larger-scale dataset X3D covering wider X-ray applications. Experiments on X3D show that SAX-NeRF surpasses previous NeRF-based methods by 12.56 and 2.49 dB on novel view synthesis and CT reconstruction. Code, models, and data are released at https://github.com/caiyuanhao1998/SAX-NeRF

📄 PDF Abstract BibTeX arXiv:2311.10959

Code (2)

caiyuanhao1998/sax-nerf 공식 구현 pytorch

cmhungsteve/Awesome-Transformer-Attention pytorch

Tasks

3D ReconstructionCT ReconstructionLow-Dose X-Ray Ct ReconstructionNeRFNovel View Synthesis

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Are Pixel-Wise Metrics Reliable for Sparse-View Computed Tomography Reconstruction?

2025-06-02 · Tianyu Lin, Xinran Li, Chuntung Zhuang, Qi Chen 외

Widely adopted evaluation metrics for sparse-view CT reconstruction--such as Structural Similarity Index Measure and Peak Signal-to-Noise Ratio--prioritize pixel-wise fidelity but often fail to capture the completeness o…

Anatomy CT Reconstruction

DGSfM: Depth-Guided Scale-Aware Global Structure-from-Motion

2026-07-10 · Sithu Aung, Viktor Kocur, Yaqing Ding, Torsten Sattler 외 arxiv

Global Structure-from-Motion (SfM) is an efficient paradigm for recovering camera poses and sparse 3D structure from unordered images. However, its reliance on scale-ambiguous epipolar geometry makes global positioning s…

Urban Neural Surface Reconstruction from Constrained Sparse Aerial Imagery with 3D SAR Fusion

2026-01-29 · Da Li, Chen Yao, Tong Mao, Jiacheng Bao 외 arxiv

Neural surface reconstruction (NSR) has recently shown strong potential for urban 3D reconstruction from multi-view aerial imagery. However, existing NSR methods often suffer from geometric ambiguity and instability, par…

3D Reconstruction Point Clouds

Sparse-View Surface Reconstruction using Gaussian Splatting through High-Confidence Depth Propagation with Normal Priors

2026-07-04 · Liang Han, Bangcai Wei, Junsheng Zhou, Yu-Shen Liu 외 arxiv

3D reconstruction from sparse views is a challenging task in 3D computer vision. Recent studies on 3D Gaussian Splatting (3DGS) have achieved remarkable results with sparse views in novel view synthesis, yet reconstructi…

Novel View Synthesis 3D Reconstruction

GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstruction

2025-12-31 · Yi-Chuan Huang, Hao-Jen Chien, Chin-Yang Lin, Ying-Huan Chen 외 arxiv

Recent 3D reconstruction methods achieve impressive results with dense multi-view imagery but struggle when only a few views are available. Various approaches, including regularization techniques, semantic priors, and ge…

3D Reconstruction