Transfer Learning of Tabular Data by Finetuning Large Language Models
Despite the artificial intelligence (AI) revolution, deep learning has yet to achieve much success with tabular data due to heterogeneous feature space and limited sample sizes without viable transfer learning. The new era of generative AI, powered by large language models (LLM), brings unprecedented learning opportunities to diverse data and domains. This paper investigates the effectiveness of an LLM application programming interface (API) and transfer learning of LLM in tabular data classification. LLM APIs respond to input text prompts with tokenized data and instructions, whereas transfer learning finetunes an LLM for a target classification task. This paper proposes an end-to-end finetuning of LLM to demonstrate cross-data transfer learning on ten benchmark data sets when large pre-trained tabular data models do not exist to facilitate transfer learning. The proposed LLM finetuning method outperforms state-of-the-art machine and deep learning methods on tabular data with less than ten features - a standard feature size for tabular data sets. The transfer learning approach uses a fraction of the computational cost of other deep learning or API-based solutions while ensuring competitive or superior classification performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningTransfer LearningSimilar Papers 제목 키워드 기반
Transfer Learning for Finetuning Large Language Models
As the landscape of large language models expands, efficiently finetuning for specific tasks becomes increasingly crucial. At the same time, the landscape of parameter-efficient finetuning methods rapidly expands. Conseq…
Bayesian OptimizationMeta-LearningTransfer LearningUtilizing Training Data to Improve LLM Reasoning for Tabular Understanding
Automated tabular understanding and reasoning are essential tasks for data scientists. Recently, Large language models (LLMs) have become increasingly prevalent in tabular reasoning tasks. Previous work focuses on (1) fi…
LATTLE: LLM Attention Transplant for Transfer Learning of Tabular Data Across Disparate Domains
Transfer learning on tabular data is challenging due to disparate feature spaces across domains, in contrast to the homogeneous structures of image and text. Large language models (LLMs) offer a knowledge base to improve…
Prompt EngineeringTransfer LearningTabular Embedding Model (TEM): Finetuning Embedding Models For Tabular RAG Applications
In recent times Large Language Models have exhibited tremendous capabilities, especially in the areas of mathematics, code generation and general-purpose reasoning. However for specialized domains especially in applicati…
Code GenerationRAGRetrievalCross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation
Recent generative large language models (LLMs) show remarkable performance in non-English languages, but when prompted in those languages they tend to express higher harmful social biases and toxicity levels. Prior work …
Cross-Lingual TransferText Generation