Lost in Historical Time? A Polish History Matura Benchmark for Large Language Models
Language models are widely used by students as knowledge sources, yet benchmarks rarely assess their interpretative historical reasoning. We evaluate eight leading LLMs on the Polish high school exit exam (Matura) in history - three official papers from 2023-2025, comprising short-answer questions and extended essays - and compare model performance against the human examinee population. Although models score near the ceiling, aggregate scores mask distinct competency profiles: rankings are unstable across task types, source modalities, and geographical scopes, with a consistent penalty for Polish versus Global history content. Qualitative error analysis reveals two recurring failure modes - source decontextualization, when models reason from source content rather than treating it as an object of analysis, and temporal disorientation, when responses are historically misplaced. This study introduces the first LLM history benchmark grounded in the Polish national curriculum.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Towards a contextualised spatial-diachronic history of literature: mapping emotional representations of the city and the country in Polish fiction from 1864 to 1939
In this article, we discuss the conditions surrounding the building of historical and literary corpora. We describe the assumptions and method of making the original corpus of the Polish novel (1864-1939). Then, we prese…
Evaluating Polish linguistic and cultural competency in large language models
Large language models (LLMs) are becoming increasingly proficient in processing and generating multilingual texts, which allows them to address real-world problems more effectively. However, language understanding is a f…
General KnowledgeLost but not erased: Finding traces of a forgotten language in neural speech models
International adoptees retain phonological traces of a birth language they can no longer speak or comprehend, a persistence typically attributed to a biologically-timed critical period. We asked whether it could instead …
Language AcquisitionSpeech RecognitionAI Decodes Historical Chinese Archives to Reveal Lost Climate History
Historical archives contain qualitative descriptions of climate events, yet converting these into quantitative records has remained a fundamental challenge. Here we introduce a paradigm shift: a generative AI framework t…
Reconstruction of the evolutionary history of gene gains and losses since the last universal common ancestor
Gene gains and losses have shaped the gene repertoire of species since the universal last common ancestor to species today. Genes in extant species were gained at different historical times via de novo creation of new ge…