Linguistic Acceptability on CoLA Dev
Accuracy
- 2019-09-23 — TinyBERT-6 67M: Accuracy 54.0
- 2020-06-05 — DeBERTa (large): Accuracy 69.5
- 2022-05-19 — En-BERT + TDA: Accuracy 88.6
| Rank | Model | Accuracy | MCC | Paper | Code | Year |
|---|---|---|---|---|---|---|
| 1 | En-BERT + TDA | 88.6 | 0.725 | Acceptability Judgements via Examining the Topology of Attention Maps | danchern97/tda4la | 2022 |
| 2 | XLM-R (pre-trained) + TDA | 73 | – | Acceptability Judgements via Examining the Topology of Attention Maps | danchern97/tda4la | 2022 |
| 3 | DeBERTa (large) | 69.5 | – | DeBERTa: Decoding-enhanced BERT with Disentangled Attention | huggingface/transformers · microsoft/DeBERTa · osu-nlp-group/mind2web · +11 | 2020 |
| 4 | TinyBERT-6 67M | 54 | – | TinyBERT: Distilling BERT for Natural Language Understanding | PaddlePaddle/PaddleNLP · huawei-noah/Pretrained-Language-Model · mindspore-ai/models · +7 | 2019 |
| 5 | Synthesizer (R+V) | 53.3 | – | Synthesizer: Rethinking Self-Attention in Transformer Models | 10-zin/Synthesizer | 2020 |
| 6 | En-BERT (pre-trained) + TDA | – | 0.420 | Acceptability Judgements via Examining the Topology of Attention Maps | danchern97/tda4la | 2022 |