paper-with-me

홈 › Papers

Raw Produce Quality Detection with Shifted Window Self-Attention

2021-12-24 · Oh Joon Kwon, Byungsoo Kim, Youngduck Choi

Global food insecurity is expected to worsen in the coming decades with the accelerated rate of climate change and the rapidly increasing population. In this vein, it is important to remove inefficiencies at every level of food production. The recent advances in deep learning can help reduce such inefficiencies, yet their application has not yet become mainstream throughout the industry, inducing economic costs at a massive scale. To this point, modern techniques such as CNNs (Convolutional Neural Networks) have been applied to RPQD (Raw Produce Quality Detection) tasks. On the other hand, Transformer's successful debut in the vision among other modalities led us to expect a better performance with these Transformer-based models in RPQD. In this work, we exclusively investigate the recent state-of-the-art Swin (Shifted Windows) Transformer which computes self-attention in both intra- and inter-window fashion. We compare Swin Transformer against CNN models on four RPQD image datasets, each containing different kinds of raw produce: fruits and vegetables, fish, pork, and beef. We observe that Swin Transformer not only achieves better or competitive performance but also is data- and compute-efficient, making it ideal for actual deployment in real-world setting. To the best of our knowledge, this is the first large-scale empirical study on RPQD task, which we hope will gain more attention in future works.

📄 PDF Abstract BibTeX arXiv:2112.13845

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Stochastic Depth Stochastic Depth aims to shrink the depth of a network during training, while keeping it unchanged during testing. This is achieved by randomly dropping entire…
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Residual Connection 설명 없음
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Similar Papers 제목 키워드 기반

SwinShadow: Shifted Window for Ambiguous Adjacent Shadow Detection

2024-08-07 · Yonghui Wang, Shaokai Liu, Li Li, Wengang Zhou 외

Shadow detection is a fundamental and challenging task in many computer vision applications. Intuitively, most shadows come from the occlusion of light by the object itself, resulting in the object and its shadow being c…

Shadow Detection

AgileIR: Memory-Efficient Group Shifted Windows Attention for Agile Image Restoration

2024-09-10 · Hongyi Cai, Mohammad Mahdinur Rahman, Mohammad Shahid Akhtar, Jie Li 외

Image Transformers show a magnificent success in Image Restoration tasks. Nevertheless, most of transformer-based models are strictly bounded by exorbitant memory occupancy. Our goal is to reduce the memory consumption o…

Image RestorationQuantization

What Makes for Hierarchical Vision Transformer?

2021-07-05 · Yuxin Fang, Xinggang Wang, Rui Wu, Wenyu Liu

Recent studies indicate that hierarchical Vision Transformer with a macro architecture of interleaved non-overlapped window-based self-attention \& shifted-window operation is able to achieve state-of-the-art performance…

Instance Segmentationobject-detectionObject DetectionSemantic Segmentation+1

Degenerate Swin to Win: Plain Window-based Transformer without Sophisticated Operations

2022-11-25 · Tan Yu, Ping Li

The formidable accomplishment of Transformers in natural language processing has motivated the researchers in the computer vision community to build Vision Transformers. Compared with the Convolution Neural Networks (CNN…

object-detectionObject DetectionSemantic Segmentation

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

2021-03-25 · ICCV 2021 10 · Ze Liu, Yutong Lin, Yue Cao, Han Hu 외

This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision. Challenges in adapting Transformer from language to vision arise from differen…

image-classificationImage ClassificationInstance SegmentationObject Detection+3