answerability prediction on PeerQA
Macro F1
- 2020-05-28 — GPT-3.5-Turbo-0613-16k: Macro F1 0.3304
- 2023-10-10 — Mistral-IT-v02-7B-32k: Macro F1 0.4703
| Rank | Model | Macro F1 | Paper | Code | Year |
|---|---|---|---|---|---|
| 1 | Mistral-IT-v02-7B-32k | 0.4703 | Mistral 7B | mistralai/mistral-src · facebookresearch/fairseq2 · mgmalek/efficient_cross_entropy · +3 | 2023 |
| 2 | Command-R-v01-34B-128k | 0.4197 | |||
| 3 | GPT-3.5-Turbo-0613-16k | 0.3304 | Language Models are Few-Shot Learners | ggml-org/llama.cpp · ggerganov/llama.cpp · karpathy/llm.c · +64 | 2020 |
| 4 | Llama-3-IT-8B-8k | 0.3112 | The Llama 3 Herd of Models | zhuzilin/ring-flash-attention · wenet-e2e/west · zechenli03/sensorllm · +2 | 2024 |
| 5 | GPT-4o-2024-08-06 | 0.3087 | GPT-4 Technical Report | openai/evals · shmsw25/factscore · unispac/visual-adversarial-examples-jailbreak-large-language-models · +8 | 2023 |
| 6 | Llama-3-IT-8B-32k | 0.2881 | The Llama 3 Herd of Models | zhuzilin/ring-flash-attention · wenet-e2e/west · zechenli03/sensorllm · +2 | 2024 |