Mixture of In-Context Prompters for Tabular PFNs
Recent benchmarks found In-Context Learning (ICL) outperforms both deep learning and tree-based algorithms on small tabular datasets. However, on larger datasets, ICL for tabular learning cannot run without severely compromising performance, due to its quadratic space and time complexity w.r.t. dataset size. We propose MIXTUREPFN, which both extends nearest-neighbor sampling to the state-of-the-art ICL for tabular learning model and uses bootstrapping to finetune said model on the inference-time dataset. MIXTUREPFN is the Condorcet winner across 36 diverse tabular datasets against 19 strong deep learning and tree-based baselines, achieving the highest mean rank among Top-10 aforementioned algorithms with statistical significance.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningIn-Context LearningSimilar Papers 제목 키워드 기반
TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks
While tabular classification has traditionally relied on from-scratch training, a recent breakthrough called prior-data fitted networks (PFNs) challenges this approach. Similar to large language models, PFNs make use of …
FairnessIn-Context Learningparameter-efficient fine-tuningtabular-classificationCorrecting Class Imbalance in Prior-Data Fitted Networks for Tabular Classification
Prior-data fitted networks (PFNs) have achieved exceptional performance on tabular classification tasks. However, like other classifiers, their performance can suffer under the effect of class imbalance, resulting in poo…
Transformers Can Do Bayesian Inference
Currently, it is hard to reap the benefits of deep learning for Bayesian methods, which allow the explicit specification of prior knowledge and accurately capture model uncertainty. We present Prior-Data Fitted Networks …
AutoMLBayesian InferenceFew-Shot Image ClassificationGaussian Processes+3Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networks
Tabular classification has traditionally relied on supervised algorithms, which estimate the parameters of a prediction model using its training data. Recently, Prior-Data Fitted Networks (PFNs) such as TabPFN have succe…
feature selectiontabular-classificationPrior-Fitted Networks Scale to Larger Datasets When Treated as Weak Learners
Prior-Fitted Networks (PFNs) have recently been proposed to efficiently perform tabular classification tasks. Although they achieve good performance on small datasets, they encounter limitations with larger datasets. The…
AutoMLtabular-classification