PARus
홈페이지 · 논문 7편
Choice of Plausible Alternatives for Russian language (PARus) evaluation provides researchers with a tool for assessing progress in open-domain commonsense causal reasoning. Each question in PARus is composed of a premise and two alternatives, where the task is to select the alternative that more plausibly has a causal relation with the premise. The correct alternative is randomized so that the expected performance of randomly guessing is 50%.
### Task Type
Evaluation of commonsense causal reasoning
Sentence Pair Classification: suitable - not suitable
### Example
``
{
"premise": "Гости вечеринки прятались за диваном.",
"choice1": "Это была вечеринка-сюрприз.",
"choice2":"Это был день рождения.",
"question": "cause",
"label": 0,
"idx": 4
}
``
### How did we collect data?
All text examples were collected from open news sources and literary magazines, then manually reviewed and supplemented by a human assessment on Yandex.Toloka