NEFTune: Noisy Embeddings Improve Instruction Finetuning
We show that language model finetuning can be improved, sometimes dramatically, with a simple augmentation. NEFTune adds noise to the embedding vectors during training. Standard finetuning of LLaMA-2-7B using Alpaca achieves 29.79% on AlpacaEval, which rises to 64.69% using noisy embeddings. NEFTune also improves over strong baselines on modern instruction datasets. Models trained with Evol-Instruct see a 10% improvement, with ShareGPT an 8% improvement, and with OpenPlatypus an 8% improvement. Even powerful models further refined with RLHF such as LLaMA-2-Chat benefit from additional training with NEFTune.
Code (4)
Tasks
Language ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
Understanding and Improving Noisy Embedding Techniques in Instruction Finetuning
Recent advancements in instructional fine-tuning have injected noise into embeddings, with NEFTune (Jain et al., 2024) setting benchmarks using uniform noise. Despite NEFTune's empirical findings that uniform noise outpe…
Instruction Finetuning DeepSeek-R1-8B Model Using LoRA and NEFTune
Financial named-entity recognition (NER) is essential for translating unstructured financial reports and news into structured knowledge graphs. However, general-purpose large language models (LLMs) often misclassify fina…
Knowledge GraphsSymNoise: Advancing Language Model Fine-tuning with Symmetric Noise
In this paper, we introduce a novel fine-tuning technique for language models, which involves incorporating symmetric noise into the embedding process. This method aims to enhance the model's function by more stringently…
Language ModelingLanguage ModellingFinetuning Generative Large Language Models with Discrimination Instructions for Knowledge Graph Completion
Traditional knowledge graph (KG) completion models learn embeddings to predict missing facts. Recent works attempt to complete KGs in a text-generation manner with large language models (LLMs). However, they need to grou…
Knowledge Graph CompletionText GenerationBaichuan2-Sum: Instruction Finetune Baichuan2-7B Model for Dialogue Summarization
Large language models (LLMs) like Llama, Baichuan and Bloom models show remarkable ability with instruction fine-tuning in many natural language tasks. Nevertheless, for the dialogue summarization task, which aims to gen…