Transfer Learning with Self-Supervised Vision Transformers for Snake Identification

2024-07-08 · Anthony Miyaguchi, Murilo Gustineli, Austin Fischer, Ryan Lundqvist

We present our approach for the SnakeCLEF 2024 competition to predict snake species from images. We explore and use Meta's DINOv2 vision transformer model for feature extraction to tackle species' high variability and visual similarity in a dataset of 182,261 images. We perform exploratory analysis on embeddings to understand their structure, and train a linear classifier on the embeddings to predict species. Despite achieving a score of 39.69, our results show promise for DINOv2 embeddings in snake identification. All code for this project is available at https://github.com/dsgt-kaggle-clef/snakeclef-2024.

📄 PDF Abstract BibTeX arXiv:2407.06178

Code (1)

dsgt-kaggle-clef/snakeclef-2024 공식 구현 pytorch

Tasks

Transfer Learning

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Residual Connection 설명 없음

Multi-Head Attention 설명 없음

Vision Transformer The Vision Transformer, or ViT, is a model for image classification that employs a Transformer-like architecture over…

Entropy Reveals Block Importance in Masked Self-Supervised Vision Transformers

2026-02-03 · Peihao Xiang, Kaida Wu, Ou Bai arxiv

Masked self-supervised vision transformers have become a dominant pretraining paradigm, yet their substantial model size poses significant challenges for resource-constrained deployment and efficient transfer learning. A…

Transfer Learning Model Compression

Solutions for Fine-grained and Long-tailed Snake Species Recognition in SnakeCLEF 2022

2022-07-04 · Cheng Zou, Furong Xu, Meng Wang, Wen Li 외

Automatic snake species recognition is important because it has vast potential to help lower deaths and disabilities caused by snakebites. We introduce our solution in SnakeCLEF 2022 for fine-grained snake species recogn…

Self-Supervised Learning

Self-supervised Models are Good Teaching Assistants for Vision Transformers

2021-09-29 · Haiyan Wu, Yuting Gao, Ke Li, Yinqi Zhang 외

Transformers have shown remarkable progress on computer vision tasks in the past year. Compared to their CNN counterparts, transformers usually need the help of distillation to achieve comparable results on middle or sma…

Image Classification Knowledge Distillation

RePre: Improving Self-Supervised Vision Transformer with Reconstructive Pre-training

2022-01-18 · Luya Wang, Feng Liang, Yangguang Li, Honggang Zhang 외

Recently, self-supervised vision transformers have attracted unprecedented attention for their impressive representation learning ability. However, the dominant method, contrastive learning, mainly relies on an instance …

Contrastive Learning Decoder Representation Learning

HiViT: Hierarchical Vision Transformer Meets Masked Image Modeling

2022-05-30 · Xiaosong Zhang, Yunjie Tian, Wei Huang, Qixiang Ye 외

Recently, masked image modeling (MIM) has offered a new methodology of self-supervised pre-training of vision transformers. A key idea of efficient implementation is to discard the masked image patches (or tokens) throug…