How to Fine-Tune BERT for Text Classification?

2019-05-14 · Chi Sun, Xipeng Qiu, Yige Xu, Xuanjing Huang

Language model pre-training has proven to be useful in learning universal language representations. As a state-of-the-art language model pre-training model, BERT (Bidirectional Encoder Representations from Transformers) has achieved amazing results in many language understanding tasks. In this paper, we conduct exhaustive experiments to investigate different fine-tuning methods of BERT on text classification task and provide a general solution for BERT fine-tuning. Finally, the proposed solution obtains new state-of-the-art results on eight widely-studied text classification datasets.

📄 PDF Abstract BibTeX arXiv:1905.05583

Code (15)

xuyige/BERT4doc-Classification 공식 구현 pytorch

Derposoft/ai-educator

Domminique/Deploy-BERT-for-Sentiment-Analysis-with-FastAPI- pytorch

GeorgeLuImmortal/Hierarchical-BERT-Model-with-Limited-Labelled-Data pytorch

arctic-yen/Google_QUEST_Q-A_Labeling tf

bcaitech1/p4-dkt-no_caffeine_no_gain pytorch

helmy-elrais/RoBERT_Recurrence_over_BERT pytorch

heraclex12/VLSP2020-Fake-News-Detection pytorch

jyp1111/sentiment_analysis pytorch

ongunuzaymacar/comparatively-finetuning-bert pytorch

qinhanmin2014/fine-tune-bert-for-text-classification pytorch

sahil00199/KYC pytorch

saproovarun/Google-Quest-Q-A tf

soarsmu/BiasFinder pytorch

uzaymacar/comparatively-finetuning-bert pytorch

Tasks

General ClassificationLanguage ModelingLanguage Modelling Sentiment Analysis Text Classification

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Slanted Triangular Learning Rates Slanted Triangular Learning Rates (STLR) is a learning rate schedule which first linearly increases the learning rate and then linearly decays it, which can be seen in Figure…

Residual Connection 설명 없음

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Weight Decay 설명 없음

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Improved Text Classification via Contrastive Adversarial Training

2021-07-21 · Lin Pan, Chung-Wei Hang, Avirup Sil, Saloni Potdar

We propose a simple and general method to regularize the fine-tuning of Transformer-based encoders for text classification tasks. Specifically, during fine-tuning we generate adversarial examples by perturbing the word e…

Classification Contrastive Learning intent-classification Intent Classification+3

State of the Art in Text Classification for South Slavic Languages: Fine-Tuning or Prompting?

2025-11-11 · Taja Kuzman Pungeršek, Peter Rupnik, Ivan Porupski, Vuk Dinić 외 arxiv

Until recently, fine-tuned BERT-like models provided state-of-the-art performance on text classification tasks. With the rise of instruction-tuned decoder-only models, commonly known as large language models (LLMs), the …

Text Classification

Advancing Single and Multi-task Text Classification through Large Language Model Fine-tuning

2024-12-11 · Hang Zhao, Qile P. Chen, Yijing Barry Zhang, Gang Yang

Both encoder-only models (e.g., BERT, RoBERTa) and large language models (LLMs, e.g., Llama3) have been widely used for text classification tasks. However, there is a lack of systematic studies comparing the performance …

Classification Decoder Intent Detection Language Modeling+6

Imbalanced Multi-label Classification for Business-related Text with Moderately Large Label Spaces

2023-06-12 · Muhammad Arslan, Christophe Cruz

In this study, we compared the performance of four different methods for multi label text classification using a specific imbalanced business dataset. The four methods we evaluated were fine tuned BERT, Binary Relevance,…

Multi-Label Classification MUlTI-LABEL-ClASSIFICATION Multi Label Text Classification Multi-Label Text Classification+2

A Hybrid Classical-Quantum Fine Tuned BERT for Text Classification

2025-11-21 · Abu Kaisar Mohammad Masum, Naveed Mahmud, M. Hassan Najafi, Sercan Aygun arxiv

Fine-tuning BERT for text classification can be computationally challenging and requires careful hyper-parameter tuning. Recent studies have highlighted the potential of quantum algorithms to outperform conventional meth…