Complex Verbs are Different: Exploring the Visual Modality in Multi-Modal Models to Predict Compositionality
This paper compares a neural network DSM relying on textual co-occurrences with a multi-modal model integrating visual information. We focus on nominal vs. verbal compounds, and zoom into lexical, empirical and perceptual target properties to explore the contribution of the visual modality. Our experiments show that (i) visual features contribute differently for verbs than for nouns, and (ii) images complement textual information, if (a) the textual modality by itself is poor and appropriate image subsets are used, or (b) the textual modality by itself is rich and large (potentially noisy) images are added.
Code (0)
등록된 구현이 없습니다.
Tasks
Semantic Textual SimilaritySimilar Papers 제목 키워드 기반
Exploring the Sensory Spaces of English Perceptual Verbs in Natural Language Data
In this study, we explore how language captures the meaning of words, in particular meaning related to sensory experiences learned from statistical distributions across texts. We focus on the most frequent perception ver…
Word EmbeddingsResources in Conflict: A Bilingual Valency Lexicon vs. a Bilingual Treebank vs. a Linguistic Theory
In this paper, we would like to exemplify how a syntactically annotated bilingual treebank can help us in exploring and revising a developed linguistic theory. On the material of the Prague Czech-English Dependency Treeb…
Machine TranslationExploring Soft-Clustering for German (Particle) Verbs across Frequency Ranges
Verbs Taking Clausal and Non-Finite Arguments as Signals of Modality - Revisiting the Issue of Meaning Grounded in Syntax
We revisit Levin's theory about the correspondence of verb meaning and syntax and infer semantic classes from a large syntactic classification of more than 600 German verbs taking clausal and non-finite arguments. Graspi…
ClassificationGeneral ClassificationTranslationVROAV: Using Iconicity to Visually Represent Abstract Verbs
For a long time, philosophers, linguists and scientists have been keen on finding an answer to the mind-bending question {``}what does abstract language look like?{''}, which has also sprung from the phenomenon of mental…