parameter-efficient fine-tuning 벤치마크
parameter-efficient fine-tuning on WinoGrande
Accuracy (% )
- 2021-06-17 — LLaMA2-7b: Accuracy (% ) 69.85
- 2024-02-14 — LLaMA2-7b: Accuracy (% ) 70.09
- 2024-08-27 — LLaMA2-7b: Accuracy (% ) 70.8
| Rank | Model | Accuracy (% ) | Extra Training Data | Paper | Code | Year |
|---|---|---|---|---|---|---|
| 1 | LLaMA2-7b | 70.80 | ✓ | GIFT-SW: Gaussian noise Injected Fine-Tuning of Salient Weights for LLMs | On-Point-RND/GIFT_SW | 2024 |
| 2 | LLaMA2-7b | 70.09 | ✓ | DoRA: Weight-Decomposed Low-Rank Adaptation | NVlabs/DoRA · catid/dora · seanzhang-zhichen/llama3-chinese · +2 | 2024 |
| 3 | LLaMA2-7b | 69.85 | ✓ | LoRA: Low-Rank Adaptation of Large Language Models | labmlai/annotated_deep_learning_paper_implementations · hiyouga/llama-efficient-tuning · tatsu-lab/stanford_alpaca · +71 | 2021 |