parameter-efficient fine-tuning 벤치마크
parameter-efficient fine-tuning on BoolQ
Accuracy (% )
- 2021-06-17 — LLaMA2-7b: Accuracy (% ) 80.28
- 2023-05-23 — LLaMA2-7b: Accuracy (% ) 82.63
| Rank | Model | Accuracy (% ) | Paper | Code | Year |
|---|---|---|---|---|---|
| 1 | LLaMA2-7b | 82.63 | QLoRA: Efficient Finetuning of Quantized LLMs | qwenlm/qwen · QwenLM/Qwen-7B · artidoro/qlora · +17 | 2023 |
| 1 | LLaMA2-7b | 82.63 | GIFT-SW: Gaussian noise Injected Fine-Tuning of Salient Weights for LLMs | On-Point-RND/GIFT_SW | 2024 |
| 3 | LLaMA2-7b | 81.93 | DoRA: Weight-Decomposed Low-Rank Adaptation | NVlabs/DoRA · catid/dora · seanzhang-zhichen/llama3-chinese · +2 | 2024 |
| 4 | LLaMA2-7b | 80.28 | LoRA: Low-Rank Adaptation of Large Language Models | labmlai/annotated_deep_learning_paper_implementations · hiyouga/llama-efficient-tuning · tatsu-lab/stanford_alpaca · +71 | 2021 |