Natural Alpha Embeddings

2019-12-04 · Riccardo Volpi, Luigi Malagò

Learning an embedding for a large collection of items is a popular approach to overcome the computational limitations associated to one-hot encodings. The aim of item embedding is to learn a low dimensional space for the representations, able to capture with its geometry relevant features or relationships for the data at hand. This can be achieved for example by exploiting adjacencies among items in large sets of unlabelled data. In this paper we interpret in an Information Geometric framework the item embeddings obtained from conditional models. By exploiting the $\alpha$-geometry of the exponential family, first introduced by Amari, we introduce a family of natural $\alpha$-embeddings represented by vectors in the tangent space of the probability simplex, which includes as a special case standard approaches available in the literature. A typical example is given by word embeddings, commonly used in natural language processing, such as Word2Vec and GloVe. In our analysis, we show how the $\alpha$-deformation parameter can impact on standard evaluation tasks.

📄 PDF Abstract BibTeX arXiv:1912.02280

Code (0)

등록된 구현이 없습니다.

Tasks

Word Embeddings

Methods 이 논문이 사용한 방법론

GloVe GloVe Embeddings are a type of word embedding that encode the co-occurrence probability ratio between two words as vector differences. GloVe uses a weighted least squares…

Evaluating Natural Alpha Embeddings on Intrinsic and Extrinsic Tasks

2020-07-01 · WS 2020 7 · Riccardo Volpi, Luigi Malag{\`o}

Skip-Gram is a simple, but effective, model to learn a word embedding mapping by estimating a conditional probability distribution for each word of the dictionary. In the context of Information Geometry, these distributi…

Word Embeddings

AlphaFuse: Learn ID Embeddings for Sequential Recommendation in Null Space of Language Embeddings

2025-04-27 · Guoqing Hu, An Zhang, Shuo Liu, Zhibo Cai 외

Recent advancements in sequential recommendation have underscored the potential of Large Language Models (LLMs) for enhancing item embeddings. However, existing approaches face three key limitations: 1) the degradation o…

Sequential Recommendation

Reverse Map Projections as Equivariant Quantum Embeddings

2024-07-29 · Max Arnott, Dimitri Papaioannou, Kieran McDowall, Phalgun Lolur 외

We introduce the novel class $(E_\alpha)_{\alpha \in [-\infty,1)}$ of reverse map projection embeddings, each one defining a unique new method of encoding classical data into quantum states. Inspired by well-known map pr…

Quantum Machine Learning

A Multimodal Human Protein Embeddings Database: DeepDrug Protein Embeddings Bank (DPEB)

2025-10-24 · Md Saiful Islam Sajol, Magesh Rajasekaran, Hayden Gemeinhardt, Adam Bess 외 arxiv

Computationally predicting protein-protein interactions (PPIs) is challenging due to the lack of integrated, multimodal protein representations. DPEB is a curated collection of 22,043 human proteins that integrates four …

Graph Neural Network

Utilizing Earth Foundation Models to Enhance the Simulation Performance of Hydrological Models with AlphaEarth Embeddings

2026-01-04 · Pengfei Qu, Wenyu Ouyang, Chi Zhang, Yikai Chai 외 arxiv

Predicting river flow in places without streamflow records is challenging because basins respond differently to climate, terrain, vegetation, and soils. Traditional basin attributes describe some of these differences, bu…