Attention-Based Clustering: Learning a Kernel from Context
In machine learning, no data point stands alone. We believe that context is an underappreciated concept in many machine learning methods. We propose Attention-Based Clustering (ABC), a neural architecture based on the attention mechanism, which is designed to learn latent representations that adapt to context within an input set, and which is inherently agnostic to input sizes and number of clusters. By learning a similarity kernel, our method directly combines with any out-of-the-box kernel-based clustering approach. We present competitive results for clustering Omniglot characters and include analytical evidence of the effectiveness of an attention-based approach for clustering.
Code (1)
Tasks
BIG-bench Machine LearningClusteringSimilar Papers 제목 키워드 기반
On Robustness of Kernel Clustering
Clustering is one of the most important unsupervised problems in machine learning and statistics. Among many existing algorithms, kernel k-means has drawn much research attention due to its ability to find non-linear clu…
ClusteringRobust Multiple Kernel k-means Clustering using Min-Max Optimization
Multiple kernel learning is a type of multiview learning that combines different data modalities by capturing view-specific patterns using kernels. Although supervised multiple kernel learning has been extensively studie…
ClusteringDisease PredictionMultiview LearningRecovery Guarantees for Kernel-based Clustering under Non-parametric Mixture Models
Despite the ubiquity of kernel-based clustering, surprisingly few statistical guarantees exist beyond settings that consider strong structural assumptions on the data generation process. In this work, we take a step towa…
ClusteringMulti-view Clustering via Unified Multi-kernel Learning and Matrix Factorization
Multi-view clustering has become increasingly important due to the multi-source character of real-world data. Among existing multi-view clustering methods, multi-kernel clustering and matrix factorization-based multi-vie…
ClusteringHybrid Random Features
We propose a new class of random feature methods for linearizing softmax and Gaussian kernels called hybrid random features (HRFs) that automatically adapt the quality of kernel estimation to provide most accurate approx…
Benchmarking