Information-theoretical label embeddings for large-scale image classification
We present a method for training multi-label, massively multi-class image classification models, that is faster and more accurate than supervision via a sigmoid cross-entropy loss (logistic regression). Our method consists in embedding high-dimensional sparse labels onto a lower-dimensional dense sphere of unit-normed vectors, and treating the classification problem as a cosine proximity regression problem on this sphere. We test our method on a dataset of 300 million high-resolution images with 17,000 labels, where it yields considerably faster convergence, as well as a 7% higher mean average precision compared to logistic regression.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGeneral Classificationimage-classificationImage ClassificationregressionSimilar Papers 제목 키워드 기반
Breaking Secure Aggregation: Label Leakage from Aggregated Gradients in Federated Learning
Federated Learning (FL) exhibits privacy vulnerabilities under gradient inversion attacks (GIAs), which can extract private information from individual gradients. To enhance privacy, FL incorporates Secure Aggregation (S…
Federated LearningInference AttackModeling Label Space Interactions in Multi-label Classification using Box Embeddings
Multi-label classification is a challenging structured prediction task in which a set of output class labels are predicted for each input. Real-world datasets often have natural or latent taxonomic relationships between …
ClassificationHierarchical Multi-label ClassificationInductive BiasMulti-Label Classification+2Semi-supervised Clustering Through Representation Learning of Large-scale EHR Data
Electronic Health Records (EHR) offer rich real-world data for personalized medicine, providing insights into disease progression, treatment responses, and patient outcomes. However, their sparsity, heterogeneity, and hi…
Representation LearningLeveraging Distributional Semantics for Multi-Label Learning
We present a novel and scalable label embedding framework for large-scale multi-label learning a.k.a ExMLDS (Extreme Multi-Label Learning using Distributional Semantics). Our approach draws inspiration from ideas rooted …
Document EmbeddingMissing LabelsMulti-Label LearningWord EmbeddingsFast Label Embeddings for Extremely Large Output Spaces
Many modern multiclass and multilabel problems are characterized by increasingly large output spaces. For these problems, label embeddings have been shown to be a useful primitive that can improve computational and stati…