Billion-scale semi-supervised learning for image classification

2019-05-02 · I. Zeki Yalniz, Hervé Jégou, Kan Chen, Manohar Paluri, Dhruv Mahajan

This paper presents a study of semi-supervised learning with large convolutional networks. We propose a pipeline, based on a teacher/student paradigm, that leverages a large collection of unlabelled images (up to 1 billion). Our main goal is to improve the performance for a given target architecture, like ResNet-50 or ResNext. We provide an extensive analysis of the success factors of our approach, which leads us to formulate some recommendations to produce high-accuracy models for image classification with semi-supervised learning. As a result, our approach brings important gains to standard architectures for image, video and fine-grained classification. For instance, by leveraging one billion unlabelled images, our learned vanilla ResNet-50 achieves 81.2% top-1 accuracy on the ImageNet benchmark.

📄 PDF Abstract BibTeX arXiv:1905.00546

Code (4)

facebookresearch/semi-supervised-ImageNet1K-models pytorch

leaderj1001/Billion-scale-semi-supervised-learning pytorch

salesforce/ensemble-of-averages pytorch

tiskw/patchcore-ad pytorch

Tasks

Classification General Classification image-classification Image Classification Object Recognition Video Classification

Methods 이 논문이 사용한 방법론

Average Pooling 설명 없음

ResNeXt Block A ResNeXt Block is a type of residual block used as part of the ResNeXt CNN…

Grouped Convolution A Grouped Convolution uses a group of convolutions - multiple kernels per layer - resulting in multiple channel outputs per layer. This leads to wider networks helping a…

Bottleneck Residual Block A Bottleneck Residual Block is a variant of the residual block that utilises 1x1 convolutions to create a bottleneck. The…

Global Average Pooling Global Average Pooling is a pooling operation designed to replace fully connected layers in classical CNNs. The idea is to generate one feature map for each corresponding…

Residual Block Residual Blocks are skip-connection blocks that learn residual functions with reference to the layer inputs, instead of learning unreferenced functions. They were introduced…

Max Pooling Max Pooling is a pooling operation that calculates the maximum value for patches of a feature map, and uses it to create a downsampled (pooled) feature map. It is usually…

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…

STAR: Noisy Semi-Supervised Transfer Learning for Visual Classification

2021-08-18 · Hasib Zunair, Yan Gobeil, Samuel Mercier, A. Ben Hamza

Semi-supervised learning (SSL) has proven to be effective at leveraging large-scale unlabeled data to mitigate the dependency on labeled data in order to learn better models for visual recognition and classification task…

Adversarial Robustness Classification Multi-class Classification Transfer Learning

Xray-Visual Models: Scaling Vision models on Industry Scale Data

2026-02-18 · Shlok Mishra, Tsung-Yu Lin, Linda Wang, Hongli Xu 외 arxiv

We present Xray-Visual, a unified vision model architecture for large-scale image and video understanding trained on industry-scale social media data. Our model leverages over 15 billion curated image-text pairs and 10 b…

Computational Efficiency Cross-Modal Retrieval Contrastive Learning Image Classification

The effectiveness of MAE pre-pretraining for billion-scale pretraining

2023-03-23 · ICCV 2023 1 · Mannat Singh, Quentin Duval, Kalyan Vasudev Alwala, Haoqi Fan 외

This paper revisits the standard pretrain-then-finetune paradigm used in computer vision for visual recognition tasks. Typically, state-of-the-art foundation models are pretrained using large scale (weakly) supervised da…

Action Classification Action Recognition Few-Shot Image Classification image-classification+6

Semi-Supervised Semantic Segmentation in Earth Observation: The MiniFrance Suite, Dataset Analysis and Multi-task Network Study

2020-10-15 · Javiera Castillo-Navarro, Bertrand Le Saux, Alexandre Boulch, Nicolas Audebert 외

The development of semi-supervised learning techniques is essential to enhance the generalization capacities of machine learning algorithms. Indeed, raw image data are abundant while labels are scarce, therefore it is cr…

Earth Observation Multi-Task Learning Semantic Segmentation Semi-Supervised Semantic Segmentation

Vehicle Trajectory Prediction by Transfer Learning of Semi-Supervised Models

2020-07-14 · Nick Lamm, Shashank Jaiprakash, Malavika Srikanth, Iddo Drori

In this work we show that semi-supervised models for vehicle trajectory prediction significantly improve performance over supervised models on state-of-the-art real-world benchmarks. Moving from supervised to semi-superv…

Contrastive Learning Trajectory Prediction Transfer Learning