Hybrid Losses for Hierarchical Embedding Learning
In traditional supervised learning, the cross-entropy loss treats all incorrect predictions equally, ignoring the relevance or proximity of wrong labels to the correct answer. By leveraging a tree hierarchy for fine-grained labels, we investigate hybrid losses, such as generalised triplet and cross-entropy losses, to enforce similarity between labels within a multi-task learning framework. We propose metrics to evaluate the embedding space structure and assess the model's ability to generalise to unseen classes, that is, to infer similar classes for data belonging to unseen categories. Our experiments on OrchideaSOL, a four-level hierarchical instrument sound dataset with nearly 200 detailed categories, demonstrate that the proposed hybrid losses outperform previous works in classification, retrieval, embedding space structure, and generalisation.
Code (1)
Tasks
Multi-Task LearningRetrievalTripletSimilar Papers 제목 키워드 기반
CLIFFNet for Monocular Depth Estimation with Hierarchical Embedding Loss
This paper proposes a hierarchical loss for monocular depth estimation, which measures the differences between the prediction and ground truth in hierarchical embedding spaces of depth maps. In order to find an appropria…
Depth EstimationMonocular Depth EstimationLanguage Models as Hierarchy Encoders
Interpreting hierarchical structures latent in language is a key limitation of current language models (LMs). While previous research has implicitly leveraged these hierarchies to enhance LMs, approaches for their explic…
HEAL: Hierarchical Embedding Alignment Loss for Improved Retrieval and Representation Learning
Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by integrating external document retrieval to provide domain-specific or up-to-date knowledge. The effectiveness of RAG depends on the relevance …
Contrastive LearningDocument ClassificationRAGRepresentation Learning+2Learning Time-Series Representations by Hierarchical Uniformity-Tolerance Latent Balancing
We propose TimeHUT, a novel method for learning time-series representations by hierarchical uniformity-tolerance balancing of contrastive representations. Our method uses two distinct losses to learn strong representatio…
Temporal SequencesAnomaly DetectionA Hybrid Approach for Aspect-Based Sentiment Analysis Using Deep Contextual Word Embeddings and Hierarchical Attention
The Web has become the main platform where people express their opinions about entities of interest and their associated aspects. Aspect-Based Sentiment Analysis (ABSA) aims to automatically compute the sentiment towards…
Aspect-Based Sentiment AnalysisAspect-Based Sentiment Analysis (ABSA)Sentiment AnalysisWord Embeddings