Transfer Learning with Deep Tabular Models
Recent work on deep learning for tabular data demonstrates the strong performance of deep tabular models, often bridging the gap between gradient boosted decision trees and neural networks. Accuracy aside, a major advantage of neural models is that they learn reusable features and are easily fine-tuned in new domains. This property is often exploited in computer vision and natural language applications, where transfer learning is indispensable when task-specific training data is scarce. In this work, we demonstrate that upstream data gives tabular neural networks a decisive advantage over widely used GBDT models. We propose a realistic medical diagnosis benchmark for tabular transfer learning, and we present a how-to guide for using upstream data to boost performance with a variety of tabular neural network architectures. Finally, we propose a pseudo-feature method for cases where the upstream and downstream feature sets differ, a tabular-specific problem widespread in real-world applications. Our code is available at https://github.com/LevinRoman/tabular-transfer-learning .
Code (1)
Tasks
Medical DiagnosisTransfer LearningSimilar Papers 제목 키워드 기반
LATTLE: LLM Attention Transplant for Transfer Learning of Tabular Data Across Disparate Domains
Transfer learning on tabular data is challenging due to disparate feature spaces across domains, in contrast to the homogeneous structures of image and text. Large language models (LLMs) offer a knowledge base to improve…
Prompt EngineeringTransfer LearningVisTabNet: Adapting Vision Transformers for Tabular Data
Although deep learning models have had great success in natural language processing and computer vision, we do not observe comparable improvements in the case of tabular data, which is still the most common data type use…
Transfer LearningTransfer Learning of Tabular Data by Finetuning Large Language Models
Despite the artificial intelligence (AI) revolution, deep learning has yet to achieve much success with tabular data due to heterogeneous feature space and limited sample sizes without viable transfer learning. The new e…
Deep LearningTransfer LearningTransferable Adversarial Robustness for Categorical Data via Universal Robust Embeddings
Research on adversarial robustness is primarily focused on image and text data. Yet, many scenarios in which lack of robustness can result in serious risks, such as fraud detection, medical diagnosis, or recommender syst…
Adversarial RobustnessFraud DetectionMedical DiagnosisRecommendation SystemsUnleashing the Power of Image-Tabular Self-Supervised Learning via Breaking Cross-Tabular Barriers
Multi-modal learning integrating medical images and tabular data has significantly advanced clinical decision-making in recent years. Self-Supervised Learning (SSL) has emerged as a powerful paradigm for pretraining thes…
Self-Supervised LearningRepresentation Learning