Can Large Language Models Reason about the Region Connection Calculus?
Qualitative Spatial Reasoning is a well explored area of Knowledge Representation and Reasoning and has multiple applications ranging from Geographical Information Systems to Robotics and Computer Vision. Recently, many claims have been made for the reasoning capabilities of Large Language Models (LLMs). Here, we investigate the extent to which a set of representative LLMs can perform classical qualitative spatial reasoning tasks on the mereotopological Region Connection Calculus, RCC-8. We conduct three pairs of experiments (reconstruction of composition tables, alignment to human composition preferences, conceptual neighbourhood reconstruction) using state-of-the-art LLMs; in each pair one experiment uses eponymous relations and one, anonymous relations (to test the extent to which the LLM relies on knowledge about the relation names obtained during training). All instances are repeated 30 times to measure the stochasticity of the LLMs.
Code (1)
Tasks
Spatial ReasoningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Region-Based Merging of Open-Domain Terminological Knowledge
This paper introduces a novel method for merging open-domain terminological knowledge. It takes advantage of the Region Connection Calculus (RCC5), a formalism used to represent regions in a topological space and to reas…
TranslationDon't Forget to Connect! Improving RAG with Graph-based Reranking
Retrieval Augmented Generation (RAG) has greatly improved the performance of Large Language Model (LLM) responses by grounding generation with context from existing documents. These systems work well when documents are c…
Abstract Meaning RepresentationLanguage ModelingLanguage ModellingLarge Language Model+4Realizing RCC8 networks using convex regions
RCC8 is a popular fragment of the region connection calculus, in which qualitative spatial relations between regions, such as adjacency, overlap and parthood, can be expressed. While RCC8 is essentially dimensionless, mo…
Missed Connections: Lateral Thinking Puzzles for Large Language Models
The Connections puzzle published each day by the New York Times tasks players with dividing a bank of sixteen words into four groups of four words that each relate to a common theme. Solving the puzzle requires both comm…
SentenceSentence EmbeddingSentence-EmbeddingSearch-on-Graph: Iterative Informed Navigation for Large Language Model Reasoning on Knowledge Graphs
Large language models (LLMs) augmented with knowledge graphs (KGs) offer a promising approach for knowledge-intensive reasoning. Central to this approach is the selection of appropriate reasoning paths in the KG. Yet, ex…
Graph Question AnsweringKnowledge Graphs