A Kernel-Based View of Language Model Fine-Tuning

2022-10-11 · Sadhika Malladi, Alexander Wettig, Dingli Yu, Danqi Chen, Sanjeev Arora

It has become standard to solve NLP tasks by fine-tuning pre-trained language models (LMs), especially in low-data settings. There is minimal theoretical understanding of empirical success, e.g., why fine-tuning a model with $10^8$ or more parameters on a couple dozen training points does not result in overfitting. We investigate whether the Neural Tangent Kernel (NTK) - which originated as a model to study the gradient descent dynamics of infinitely wide networks with suitable random initialization - describes fine-tuning of pre-trained LMs. This study was inspired by the decent performance of NTK for computer vision tasks (Wei et al., 2022). We extend the NTK formalism to Adam and use Tensor Programs (Yang, 2020) to characterize conditions under which the NTK lens may describe fine-tuning updates to pre-trained language models. Extensive experiments on 14 NLP tasks validate our theory and show that formulating the downstream task as a masked word prediction problem through prompting often induces kernel-based dynamics during fine-tuning. Finally, we use this kernel view to propose an explanation for the success of parameter-efficient subspace-based fine-tuning methods.

📄 PDF Abstract BibTeX arXiv:2210.05643

Code (1)

princeton-nlp/lm-kernel-ft 공식 구현 pytorch

Tasks

Language ModelingLanguage Modelling

Methods 이 논문이 사용한 방법론

SGD Stochastic Gradient Descent is an iterative optimization technique that uses minibatches of data to form an expectation of the gradient, rather than the full gradient using…

NTK 설명 없음

Adam 설명 없음

Inducer-tuning: Connecting Prefix-tuning and Adapter-tuning

2022-10-26 · Yifan Chen, Devamanyu Hazarika, Mahdi Namazifar, Yang Liu 외

Prefix-tuning, or more generally continuous prompt tuning, has become an essential paradigm of parameter-efficient transfer learning. Using a large pre-trained language model (PLM), prefix-tuning can obtain strong perfor…

Language Modeling Language Modelling Natural Language Understanding Transfer Learning

Empowering parameter-efficient transfer learning by recognizing the kernel structure in self-attention

2022-05-07 · Findings (NAACL) 2022 7 · Yifan Chen, Devamanyu Hazarika, Mahdi Namazifar, Yang Liu 외

The massive amount of trainable parameters in the pre-trained language models (PLMs) makes them hard to be deployed to multiple downstream tasks. To address this issue, parameter-efficient transfer learning methods have …

Text Generation Transfer Learning

A Theoretical Framework for LLM Fine-tuning Using Early Stopping for Non-random Initialization

2026-02-15 · Zexuan Sun, Garvesh Raskutti arxiv

In the era of large language models (LLMs), fine-tuning pretrained models has become ubiquitous. Yet the theoretical underpinning remains an open question. A central question is why only a few epochs of fine-tuning are t…

RoCoFT: Efficient Finetuning of Large Language Models with Row-Column Updates

2024-10-14 · Md Kowsher, Tara Esmaeilbeig, Chun-Nam Yu, Mojtaba Soltanalian 외

We propose RoCoFT, a parameter-efficient fine-tuning method for large-scale language models (LMs) based on updating only a few rows and columns of the weight matrices in transformers. Through extensive experiments with m…

parameter-efficient fine-tuning

Fine-Tuning GPT-5 for GPU Kernel Generation

2026-02-11 · Ali Tehrani, Yahya Emara, Essam Wissam, Wojciech Paluch 외 arxiv

Developing efficient GPU kernels is essential for scaling modern AI systems, yet it remains a complex task due to intricate hardware architectures and the need for specialized optimization expertise. Although Large Langu…

Reinforcement Learning Code Generation