Sparse and geometry-aware generalisation of the mutual information for joint discriminative clustering and feature selection
Feature selection in clustering is a hard task which involves simultaneously the discovery of relevant clusters as well as relevant variables with respect to these clusters. While feature selection algorithms are often model-based through optimised model selection or strong assumptions on the data distribution, we introduce a discriminative clustering model trying to maximise a geometry-aware generalisation of the mutual information called GEMINI with a simple l1 penalty: the Sparse GEMINI. This algorithm avoids the burden of combinatorial feature subset exploration and is easily scalable to high-dimensional data and large amounts of samples while only designing a discriminative clustering model. We demonstrate the performances of Sparse GEMINI on synthetic datasets and large-scale datasets. Our results show that Sparse GEMINI is a competitive algorithm and has the ability to select relevant subsets of variables with respect to the clustering without using relevance criteria or prior hypotheses.
Code (0)
등록된 구현이 없습니다.
Tasks
Clusteringfeature selectionModel SelectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Skill-aware Mutual Information Optimisation for Generalisation in Reinforcement Learning
Meta-Reinforcement Learning (Meta-RL) agents can struggle to operate across tasks with varying environmental features that require different optimal skills (i.e., different modes of behaviour). Using context encoders bas…
Contrastive LearningMeta Reinforcement LearningMuJoCoreinforcement-learning+1Chained Generalisation Bounds
This work discusses how to derive upper bounds for the expected generalisation error of supervised learning algorithms by means of the chaining technique. By developing a general theoretical framework, we establish a dua…
From Generalisation Error to Transportation-cost Inequalities and Back
In this work, we connect the problem of bounding the expected generalisation error with transportation-cost inequalities. Exposing the underlying pattern behind both approaches we are able to generalise them and go beyon…
Generalisation and the Geometry of Class Separability
Recent results in deep learning show that considering only the capacity of machines does not adequately explain the generalisation performance we can observe. We propose that by considering the geometry of the data we ca…
Deep LearningCarré du champ flow matching: better quality-generalisation tradeoff in generative models
Deep generative models often face a fundamental tradeoff: high sample quality can come at the cost of memorisation, where the model reproduces training data rather than generalising across the underlying data geometry. W…
Point Clouds