parameter-efficient fine-tuning 벤치마크
parameter-efficient fine-tuning on HellaSwag
Accuracy (% )
- 2021-06-17 — LLaMA2-7b: Accuracy (% ) 76.67
- 2024-08-27 — LLaMA2-7b: Accuracy (% ) 76.68
| Rank | Model | Accuracy (% ) | Extra Training Data | Paper | Code | Year |
|---|---|---|---|---|---|---|
| 1 | LLaMA2-7b | 76.68 | ✓ | GIFT-SW: Gaussian noise Injected Fine-Tuning of Salient Weights for LLMs | On-Point-RND/GIFT_SW | 2024 |
| 2 | LLaMA2-7b | 76.67 | ✓ | LoRA: Low-Rank Adaptation of Large Language Models | labmlai/annotated_deep_learning_paper_implementations · hiyouga/llama-efficient-tuning · tatsu-lab/stanford_alpaca · +71 | 2021 |
| 3 | LLaMA2-7b | 76.27 | ✓ | DoRA: Weight-Decomposed Low-Rank Adaptation | NVlabs/DoRA · catid/dora · seanzhang-zhichen/llama3-chinese · +2 | 2024 |