Speaking, Seeing, Understanding: Correlating semantic models with conceptual representation in the brain
Research in computational semantics is increasingly guided by our understanding of human semantic processing. However, semantic models are typically studied in the context of natural language processing system performance. In this paper, we present a systematic evaluation and comparison of a range of widely-used, state-of-the-art semantic models in their ability to predict patterns of conceptual representation in the human brain. Our results provide new insights both for the design of computational semantic models and for further research in cognitive neuroscience.
Code (0)
등록된 구현이 없습니다.
Tasks
Semantic Textual SimilaritySimilar Papers 제목 키워드 기반
Seeing Through Words, Speaking Through Pixels: Deep Representational Alignment Between Vision and Language Models
Recent studies show that deep vision-only and language-only models--trained on disjoint modalities--nonetheless project their inputs into a partially aligned representational space. Yet we still lack a clear picture of w…
The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open World
We present the All-Seeing (AS) project: a large-scale data and model for recognizing and understanding everything in the open world. Using a scalable data engine that incorporates human feedback and efficient models in t…
AllQuestion AnsweringRetrievalText RetrievalInvestigating Inner Properties of Multimodal Representation and Semantic Compositionality with Brain-based Componential Semantics
Multimodal models have been proven to outperform text-based approaches on learning semantic representations. However, it still remains unclear what properties are encoded in multimodal representations, in what aspects do…
Learning Semantic RepresentationsNatural Language UnderstandingConceptual transfer: Using local classifiers for transfer selection
A key challenge for Machine Translation is transfer selection, i.e. to find the right translation for a given word from a set of alternatives (1:n). This problem becomes the more important the larger the dictionary is, a…
Machine TranslationTranslationComparing Moral Values in Western English-speaking societies and LLMs with Word Associations
As the impact of large language models increases, understanding the moral values they reflect becomes ever more important. Assessing the nature of moral values as understood by these models via direct prompting is challe…