paper-with-me

Papers

Improving Deep Representation Learning via Auxiliary Learnable Target Coding

2023-05-30 · KangJun Liu, Ke Chen, Kui Jia, YaoWei Wang

Deep representation learning is a subfield of machine learning that focuses on learning meaningful and useful representations of data through deep neural networks. However, existing methods for semantic classification typically employ pre-defined target codes such as the one-hot and the Hadamard codes, which can either fail or be less flexible to model inter-class correlation. In light of this, this paper introduces a novel learnable target coding as an auxiliary regularization of deep representation learning, which can not only incorporate latent dependency across classes but also impose geometric properties of target codes into representation space. Specifically, a margin-based triplet loss and a correlation consistency loss on the proposed target codes are designed to encourage more discriminative representations owing to enlarging between-class margins in representation space and favoring equal semantic correlation of learnable target codes respectively. Experimental results on several popular visual classification and retrieval benchmarks can demonstrate the effectiveness of our method on improving representation learning, especially for imbalanced data. Source codes are made publicly available at \href{https://github.com/AkonLau/LTC}{https://github.com/AkonLau/LTC}.

📄 PDF Abstract BibTeX arXiv:2305.18680

Code (1)

AkonLau/LTC 공식 구현 pytorch

Tasks

Representation LearningRetrievalTriplet

Methods 이 논문이 사용한 방법론

fail 설명 없음
Triplet Loss The goal of Triplet loss, in the context of Siamese Networks, is to maximize the joint probability among all score-pairs i.e. the product of all probabilities. By using its…

Similar Papers 제목 키워드 기반

Enhanced Standard Compatible Image Compression Framework based on Auxiliary Codec Networks

2020-09-30 · Hanbin Son, Taeoh Kim, Hyeongmin Lee, Sangyoun Lee

To enhance image compression performance, recent deep neural network-based research can be divided into three categories: a learnable codec, a postprocessing network, and a compact representation network. The learnable c…

Image Compression

Instance-Conditional Knowledge Distillation for Object Detection

2021-10-25 · NeurIPS 2021 12 · Zijian Kang, Peizhen Zhang, Xiangyu Zhang, Jian Sun 외

Knowledge distillation has shown great success in classification, however, it is still challenging for detection. In a typical image for detection, representations from different locations may have different contribution…

Image ClassificationKnowledge DistillationObjectobject-detection+1

Speculative Streaming: Fast LLM Inference without Auxiliary Models

2024-02-16 · Nikhil Bhendawade, Irina Belousova, Qichen Fu, Henry Mason 외

Speculative decoding is a prominent technique to speed up the inference of a large target language model based on predictions of an auxiliary draft model. While effective, in application-specific settings, it often invol…

Language ModelingLanguage Modelling

From Uniform to Learned Knots: A Study of Spline-Based Numerical Encodings for Tabular Deep Learning

2026-04-07 · Manish Kumar, Anton Frederik Thielmann, Christoph Weisser, Benjamin Säfken arxiv

Numerical preprocessing remains a critical component of tabular deep learning, as the representation of continuous features can strongly affect downstream performance. We systematically study spline-based numerical encod…

Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding

2021-06-05 · NeurIPS 2021 12 · Yang Li, Si Si, Gang Li, Cho-Jui Hsieh 외

Attentional mechanisms are order-invariant. Positional encoding is a crucial component to allow attention-based deep model architectures such as Transformer to address sequences or images where the position of informatio…

Position