UFAL-CUNI at SemEval-2026 Task 11: An Efficient Modular Neuro-symbolic Method for Syllogistic Reasoning

2026-05-06 · Ivan Kartáč, Kristýna Onderková, Jan Bronec, Zdeněk Kasner, Mateusz Lango, Ondřej Dušek arxiv

This paper describes our system submitted to SemEval-2026 Task 11: Disentangling Content and Formal Reasoning in Large Language Models. We present an efficient modular neuro-symbolic approach, combining a symbolic prover with small reasoning LLMs (4B parameters). The system consists of an LLM-based parser that translates natural language syllogisms to a first-order logic (FOL) representation, an automated theorem prover, and two optional modules: machine translation for multilingual inputs and a symbolic retrieval component for the identification of relevant premises. The system achieves competitive accuracy and relatively low content effect on most subtasks. Our ablations show that this approach outperforms LLM-based zero-shot baselines in this parameter size range, but also reveal limited multilingual capabilities of small LLMs. Finally, we include a discussion of the task's main ranking metric and analyze its limitations.

📄 PDF Abstract BibTeX arXiv:2605.04941

Code (0)

등록된 구현이 없습니다.

Tasks

Machine Translation

UFAL at SemEval-2016 Task 5: Recurrent Neural Networks for Sentence Classification

2016-06-01 · SEMEVAL 2016 6 · Ale{\v{s}} Tamchyna, Kate{\v{r}}ina Veselovsk{\'a}

Aspect-Based Sentiment Analysis (ABSA)Feature Engineering General Classification Sentence+2

Open-Source Web Service with Morphological Dictionary-Supplemented Deep Learning for Morphosyntactic Analysis of Czech

2024-06-18 · Milan Straka, Jana Straková

We present an open-source web service for Czech morphosyntactic analysis. The system combines a deep learning model with rescoring by a high-precision morphological dictionary at inference time. We show that our hybrid m…

Deep Learning Dependency Parsing Lemmatization POS+1

A database of semantic clusters of verb usages

2012-05-01 · LREC 2012 5 · Silvie Cinkov{\'a}, Martin Holub, Adam Rambousek, Lenka Smejkalov{\'a}

We are presenting VPS-30-En, a small lexical resource that contains the following 30 English verbs: access, ally, arrive, breathe, claim, cool, crush, cry, deny, enlarge, enlist, forge, furnish, hail, halt, part, plough,…

ETHAN at SemEval-2020 Task 5: Modelling Causal Reasoning in Language Using Neuro-symbolic Cloud Computing

2020-12-01 · SEMEVAL 2020 · Len Yabloko

I present ETHAN: Experimental Testing of Hybrid AI Node implemented entirely on free cloud computing infrastructure. The ultimate goal of this research is to create modular reusable hybrid neuro-symbolic architecture for…

Cloud Computing Language Modeling Language Modelling

ETHAN at SemEval-2020 Task 5: Modelling Causal Reasoning inLanguage using neuro-symbolic cloud computing

2020-08-28 · COLING 2020 8 · Len Yabloko

Experimental Testing of Hybrid AI Node implemented entirely on free cloud computing infrastructure. The ultimate goal of this research is to create modular reusable hybrid neuro-symbolic architecture for Artificial Intel…

Cloud Computing Counterfactual Detection Language Modeling Language Modelling