MultiScale Spectral-Spatial Convolutional Transformer for Hyperspectral Image Classification

2023-10-28 · Zhiqiang Gong, Xian Zhou, Wen Yao

Due to the powerful ability in capturing the global information, Transformer has become an alternative architecture of CNNs for hyperspectral image classification. However, general Transformer mainly considers the global spectral information while ignores the multiscale spatial information of the hyperspectral image. In this paper, we propose a multiscale spectral-spatial convolutional Transformer (MultiscaleFormer) for hyperspectral image classification. First, the developed method utilizes multiscale spatial patches as tokens to formulate the spatial Transformer and generates multiscale spatial representation of each band in each pixel. Second, the spatial representation of all the bands in a given pixel are utilized as tokens to formulate the spectral Transformer and generate the multiscale spectral-spatial representation of each pixel. Besides, a modified spectral-spatial CAF module is constructed in the MultiFormer to fuse cross-layer spectral and spatial information. Therefore, the proposed MultiFormer can capture the multiscale spectral-spatial information and provide better performance than most of other architectures for hyperspectral image classification. Experiments are conducted over commonly used real-world datasets and the comparison results show the superiority of the proposed method.

📄 PDF Abstract BibTeX arXiv:2310.18550

Code (0)

등록된 구현이 없습니다.

Tasks

Classification Hyperspectral Image Classification image-classification Image Classification

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Multi-Head Attention 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Residual Connection 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Multiscale Clustering of Hyperspectral Images Through Spectral-Spatial Diffusion Geometry

2021-03-29 · Sam L. Polk, James M. Murphy

Clustering algorithms partition a dataset into groups of similar points. The primary contribution of this article is the Multiscale Spatially-Regularized Diffusion Learning (M-SRDL) clustering algorithm, which uses spati…

Clustering

ConvVitMamba: Efficient Multiscale Convolution, Transformer, and Mamba-Based Sequence modelling for Hyperspectral Image Classification

2026-04-20 · Mohammed Q. Alkhatib arxiv

Hyperspectral image (HSI) classification remains challenging due to high spectral dimensionality, redundancy, and limited labeled data. Although convolutional neural networks (CNNs) and Vision Transformers (ViTs) achieve…

Hyperspectral Image Classification

A Cross-Hierarchical Difference Feature Fusion Network Based on Multiscale Encoder-Decoder for Hyperspectral Change Detection

2025-09-21 · Mingshuai Sheng, Bhatti Uzair Aslam, Junfeng Zhang, Siling Feng 외 arxiv

Hyperspectral change detection (HCD) is one of the core applications of remote sensing images, holding significant research value in fields like environmental monitoring and disaster assessment. However, existing methods…

Change Detection

Multiscale Convolutional Transformer with Center Mask Pretraining for Hyperspectral Image Classification

2022-03-09 · Sen Jia, Yifan Wang

Hyperspectral images (HSI) not only have a broad macroscopic field of view but also contain rich spectral information, and the types of surface objects can be identified through spectral information, which is one of the …

Classification Decoder Hyperspectral Image Classification image-classification+1

Hierarchical Homogeneity-Based Superpixel Segmentation: Application to Hyperspectral Image Analysis

2024-07-22 · Luciano Carvalho Ayres, Sérgio José Melo de Almeida, José Carlos Moreira Bermudez, Ricardo Augusto Borsoi

Hyperspectral image (HI) analysis approaches have recently become increasingly complex and sophisticated. Recently, the combination of spectral-spatial information and superpixel techniques have addressed some hyperspect…

Hyperspectral image analysis Superpixels