Incorporating LLM Priors into Tabular Learners
We present a method to integrate Large Language Models (LLMs) and traditional tabular data classification techniques, addressing LLMs challenges like data serialization sensitivity and biases. We introduce two strategies utilizing LLMs for ranking categorical variables and generating priors on correlations between continuous variables and targets, enhancing performance in few-shot scenarios. We focus on Logistic Regression, introducing MonotonicLR that employs a non-linear monotonic function for mapping ordinals to cardinals while preserving LLM-determined orders. Validation against baseline models reveals the superior performance of our approach, especially in low-data scenarios, while remaining interpretable.
Code (0)
등록된 구현이 없습니다.
Tasks
regressionSensitivityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Strengthening LLMs for Tabular Prediction with Structural Priors
Tabular prediction has long been dominated by gradient-boosted decision trees and specialized deep tabular models, while large language models (LLMs) remain difficult to make competitive despite their cross-task adaptabi…
Can Graphs Improve Tabular Foundation Models?
Tabular data are central to many real-world systems. While recent tabular transformers and in-context learners such as SAINT, TP-BERTa, TabPFN, TabICL, and MITRA incorporate limited inter-row reasoning, most approaches s…
Algebraic Machine Learning for Small-to-Medium Datasets Is Competitive against Strong Standard Baselines
Symbolic methods are generally not considered competitive with strong modern learners on realistic supervised tasks. We evaluate Algebraic Machine Learning (AML), a framework that learns through subdirect decomposition o…
Enhancing Tabular Learners with Context-Aware Semantic Embeddings
While modern tabular learners excel at capturing statistical patterns, they frequently operate in a semantic vacuum, treating textual features as discrete symbols, ignoring the rich semantics inherent in feature names or…
Fine-Grain Few-Shot Vision via Domain Knowledge as Hyperspherical Priors
Prototypical networks have been shown to perform well at few-shot learning tasks in computer vision. Yet these networks struggle when classes are very similar to each other (fine-grain classification) and currently have …
ClassificationFew-Shot LearningGeneral Classification