Unsupervised Visual Representation Learning by Online Constrained K-Means

2021-05-24 · CVPR 2022 1 · Qi Qian, Yuanhong Xu, Juhua Hu, Hao Li, Rong Jin

Cluster discrimination is an effective pretext task for unsupervised representation learning, which often consists of two phases: clustering and discrimination. Clustering is to assign each instance a pseudo label that will be used to learn representations in discrimination. The main challenge resides in clustering since prevalent clustering methods (e.g., k-means) have to run in a batch mode. Besides, there can be a trivial solution consisting of a dominating cluster. To address these challenges, we first investigate the objective of clustering-based representation learning. Based on this, we propose a novel clustering-based pretext task with online \textbf{Co}nstrained \textbf{K}-m\textbf{e}ans (\textbf{CoKe}). Compared with the balanced clustering that each cluster has exactly the same size, we only constrain the minimal size of each cluster to flexibly capture the inherent data structure. More importantly, our online assignment method has a theoretical guarantee to approach the global optimum. By decoupling clustering and discrimination, CoKe can achieve competitive performance when optimizing with only a single view from each instance. Extensive experiments on ImageNet and other benchmark data sets verify both the efficacy and efficiency of our proposal. Code is available at \url{https://github.com/idstcv/CoKe}.

📄 PDF Abstract BibTeX arXiv:2105.11527

Code (1)

idstcv/coke 공식 구현 pytorch

Tasks

ClusteringContrastive Learning Image Clustering Metric Learning Online ClusteringPseudo LabelRepresentation Learning Self-Supervised Image Classification Unsupervised Image Classification

Representation Learning via Consistent Assignment of Views to Clusters

2021-12-31 · Thalles Silva, Adín Ramírez Rivera

We introduce Consistent Assignment for Representation Learning (CARL), an unsupervised learning method to learn visual representations by combining ideas from self-supervised contrastive learning and deep clustering. By …

Clustering Contrastive Learning Deep Clustering Linear evaluation+2

Distinguishing indistinguishable attractors: Unsupervised anomaly detection with reservoir computers

2026-06-19 · Davide Prosperino, Haochun Ma, Christoph Räth arxiv

Detecting when a nonlinear dynamical system departs from its normal regime is a recurring problem across the sciences, from cardiology to climate and energy systems. We show that a very simple Kolmogorov--Smirnov test on…

Unsupervised Anomaly Detection

An Algorithm for Online K-Means Clustering

2014-12-18 · Edo Liberty, Ram Sriharsha, Maxim Sviridenko

This paper shows that one can be competitive with the k-means objective while operating online. In this model, the algorithm receives vectors v_1,...,v_n one by one in an arbitrary order. For each vector the algorithm ou…

Clustering

MIX'EM: Unsupervised Image Classification using a Mixture of Embeddings

2020-07-18 · Ali Varamesh, Tinne Tuytelaars

We present MIX'EM, a novel solution for unsupervised image classification. MIX'EM generates representations that by themselves are sufficient to drive a general-purpose clustering algorithm to deliver high-quality classi…

Classification Clustering General Classification image-classification+3

Learning from Label Proportions in Brain-Computer Interfaces: Online Unsupervised Learning with Guarantees

2017-01-25 · D Hübner, T Verhoeven, K Schmid, K-R Müller 외

Objective: Using traditional approaches, a Brain-Computer Interface (BCI) requires the collection of calibration data for new subjects prior to online use. Calibration time can be reduced or eliminated e.g.~by transfer o…

Brain Computer Interface Decoder ERP General Classification