LLM Embeddings for Deep Learning on Tabular Data

2025-02-17 · Boshko Koloski, Andrei Margeloiu, Xiangjian Jiang, Blaž Škrlj, Nikola Simidjievski, Mateja Jamnik

Tabular deep-learning methods require embedding numerical and categorical input features into high-dimensional spaces before processing them. Existing methods deal with this heterogeneous nature of tabular data by employing separate type-specific encoding approaches. This limits the cross-table transfer potential and the exploitation of pre-trained knowledge. We propose a novel approach that first transforms tabular data into text, and then leverages pre-trained representations from LLMs to encode this data, resulting in a plug-and-play solution to improv ing deep-learning tabular methods. We demonstrate that our approach improves accuracy over competitive models, such as MLP, ResNet and FT-Transformer, by validating on seven classification datasets.

📄 PDF Abstract BibTeX arXiv:2502.11596

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Learning

Methods 이 논문이 사용한 방법론

Average Pooling 설명 없음

Max Pooling Max Pooling is a pooling operation that calculates the maximum value for patches of a feature map, and uses it to create a downsampled (pooled) feature map. It is usually…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Kaiming Initialization 설명 없음

FT-Transformer 설명 없음

Global Average Pooling Global Average Pooling is a pooling operation designed to replace fully connected layers in classical CNNs. The idea is to generate one feature map for each corresponding…

Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks

2026-04-23 · Liane Vogel, Kavitha Srinivas, Niharika D'Souza, Sola Shirai 외 arxiv

Tabular foundation models aim to learn universal representations of tabular data that transfer across tasks and domains, enabling applications such as table retrieval, semantic search and table-based prediction. Despite …

Representation Learning Table Retrieval

Universal Embeddings of Tabular Data

2025-07-08 · Astrid Franz, Frederik Hoppe, Marianne Michaelis, Udo Göbel

Tabular data in relational databases represents a significant portion of industrial data. Hence, analyzing and interpreting tabular data is of utmost importance. Application tasks on tabular data are manifold and are oft…

Entity Embeddings Outlier Detection

SuperTML: Two-Dimensional Word Embedding for the Precognition on Structured Tabular Data

2019-02-26 · Baohua Sun, Lin Yang, Wenhan Zhang, Michael Lin 외

Tabular data is the most commonly used form of data in industry. Gradient Boosting Trees, Support Vector Machine, Random Forest, and Logistic Regression are typically used for classification tasks on tabular data. DNN mo…

General Classification Text Classification Transfer Learning Word Embeddings

SuperTML: Domain Transfer from Computer Vision to Structured Tabular Data through Two-Dimensional Word Embedding

2019-05-16 · ICML Workshop AMTL 2019 6 · Anonymous

Structured tabular data is the most commonly used form of data in industry according to a Kaggle ML and DS Survey. Gradient Boosting Trees, Support Vector Machine, Random Forest, and Logistic Regression are typically use…

Classification text-classification Text Classification Word Embeddings

Linear Dimensionality Reduction for Word Embeddings in Tabular Data Classification

2025-09-15 · Liam Ressel, Hamza A. A. Gardi arxiv

The Engineers' Salary Prediction Challenge requires classifying salary categories into three classes based on tabular data. The job description is represented as a 300-dimensional word embedding incorporated into the tab…

Dimensionality Reduction