paper-with-me

Papers

CTformer: Convolution-free Token2Token Dilated Vision Transformer for Low-dose CT Denoising

2022-02-28 · Dayang Wang, Fenglei Fan, Zhan Wu, Rui Liu, Fei Wang, Hengyong Yu

Low-dose computed tomography (LDCT) denoising is an important problem in CT research. Compared to the normal dose CT (NDCT), LDCT images are subjected to severe noise and artifacts. Recently in many studies, vision transformers have shown superior feature representation ability over convolutional neural networks (CNNs). However, unlike CNNs, the potential of vision transformers in LDCT denoising was little explored so far. To fill this gap, we propose a Convolution-free Token2Token Dilated Vision Transformer for low-dose CT denoising. The CTformer uses a more powerful token rearrangement to encompass local contextual information and thus avoids convolution. It also dilates and shifts feature maps to capture longer-range interaction. We interpret the CTformer by statically inspecting patterns of its internal attention maps and dynamically tracing the hierarchical attention flow with an explanatory graph. Furthermore, an overlapped inference mechanism is introduced to effectively eliminate the boundary artifacts that are common for encoder-decoder-based denoising models. Experimental results on Mayo LDCT dataset suggest that the CTformer outperforms the state-of-the-art denoising methods with a low computation overhead.

📄 PDF Abstract BibTeX arXiv:2202.13517

Code (2)

wdayang/ctformer 공식 구현 pytorch
wdayang/TED-net pytorch

Tasks

DecoderDenoising

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Position-Wise Feed-Forward Layer 설명 없음
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Residual Connection 설명 없음
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Similar Papers 제목 키워드 기반

Behavior Structformer: Learning Players Representations with Structured Tokenization

2024-06-07 · Oleg Smirnov, Labinot Polisi

In this paper, we introduce the Behavior Structformer, a method for modeling user behavior using structured tokenization within a Transformer-based architecture. By converting tracking events into dense tokens, this appr…

Benchmarking

Multi-class Token Transformer for Weakly Supervised Semantic Segmentation

2022-03-06 · CVPR 2022 1 · Lian Xu, Wanli Ouyang, Mohammed Bennamoun, Farid Boussaid 외

This paper proposes a new transformer-based framework to learn class-specific object localization maps as pseudo labels for weakly supervised semantic segmentation (WSSS). Inspired by the fact that the attended regions o…

ObjectObject LocalizationSemantic SegmentationWeakly supervised Semantic Segmentation+1

OctFormer: Octree-based Transformers for 3D Point Clouds

2023-05-04 · Peng-Shuai Wang

We propose octree-based transformers, named OctFormer, for 3D point cloud learning. OctFormer can not only serve as a general and effective backbone for 3D point cloud segmentation and object detection but also have line…

3D Object Detection3D Semantic SegmentationGPUobject-detection+3

XCTFormer: Leveraging Cross-Channel and Cross-Time Dependencies for Enhanced Time-Series Analysis

2026-05-18 · Israel Zexer, Omri Azencot arxiv

Multivariate time-series analysis involves extracting informative representations from sequences of multiple interdependent variables, supporting tasks such as forecasting, imputation, and anomaly detection. In real-worl…

Anomaly Detection

MCTformer+: Multi-Class Token Transformer for Weakly Supervised Semantic Segmentation

2023-08-06 · Lian Xu, Mohammed Bennamoun, Farid Boussaid, Hamid Laga 외

This paper proposes a novel transformer-based framework that aims to enhance weakly supervised semantic segmentation (WSSS) by generating accurate class-specific object localization maps as pseudo labels. Building upon t…

Object LocalizationSemantic SegmentationWeakly supervised Semantic SegmentationWeakly-Supervised Semantic Segmentation