paper-with-me

Papers

Representation Learning of Compositional Data

2018-12-01 · NeurIPS 2018 12 · Marta Avalos, Richard Nock, Cheng Soon Ong, Julien Rouar, Ke Sun

We consider the problem of learning a low dimensional representation for compositional data. Compositional data consists of a collection of nonnegative data that sum to a constant value. Since the parts of the collection are statistically dependent, many standard tools cannot be directly applied. Instead, compositional data must be first transformed before analysis. Focusing on principal component analysis (PCA), we propose an approach that allows low dimensional representation learning directly from the original data. Our approach combines the benefits of the log-ratio transformation from compositional data analysis and exponential family PCA. A key tool in its derivation is a generalization of the scaled Bregman theorem, that relates the perspective transform of a Bregman divergence to the Bregman divergence of a perspective transform and a remainder conformal divergence. Our proposed approach includes a convenient surrogate (upper bound) loss of the exponential family PCA which has an easy to optimize form. We also derive the corresponding form for nonlinear autoencoders. Experiments on simulated data and microbiome data show the promise of our method.

📄 PDF Abstract BibTeX

Code (2)

https://bitbucket.org/RichardNock/coda 공식 구현
sistm/CoDa-PCA

Tasks

Representation Learning

Methods 이 논문이 사용한 방법론

PCA Principle Components Analysis (PCA) is an unsupervised method primary used for dimensionality reduction within machine learning. PCA is calculated via a singular value…

Similar Papers 제목 키워드 기반

Compositional Generalization in Unsupervised Compositional Representation Learning: A Study on Disentanglement and Emergent Language

2022-10-02 · Zhenlin Xu, Marc Niethammer, Colin Raffel

Deep learning models struggle with compositional generalization, i.e. the ability to recognize or generate novel combinations of observed elementary concepts. In hopes of enabling compositional generalization, various un…

DisentanglementRepresentation Learning

Measuring Compositionality in Representation Learning

2019-02-19 · ICLR 2019 5 · Jacob Andreas

Many machine learning algorithms represent input data with vector embeddings or discrete codes. When inputs exhibit compositional structure (e.g. objects built from parts or procedures from subroutines), it is natural to…

BIG-bench Machine LearningRepresentation Learning

A Complexity-Based Theory of Compositionality

2024-10-18 · Eric Elmoznino, Thomas Jiralerspong, Yoshua Bengio, Guillaume Lajoie

Compositionality is believed to be fundamental to intelligence. In humans, it underlies the structure of thought, language, and higher-level reasoning. In AI, compositional representations can enable a powerful form of o…

Out-of-Distribution Generalization

Necessary and Sufficient Conditions for Compositional Representations

2021-01-01 · Yuanpeng Li

Humans naturally use compositional representations for flexible recognition and expression, but current machine learning lacks such ability. Despite many efforts in specific cases, there is still absence of theories and …

Towards Compositionality in Concept Learning

2024-06-26 · Adam Stein, Aaditya Naik, Yinjun Wu, Mayur Naik 외

Concept-based interpretability methods offer a lens into the internals of foundation models by decomposing their embeddings into high-level concepts. These concept representations are most useful when they are compositio…