Predicting human similarity judgments with distributional models: The value of word associations.
Most distributional lexico-semantic models derive their representations based on external language resources such as text corpora. In this study, we propose that internal language models, that are more closely aligned to the mental representations of words could provide important insights into cognitive science, including linguistics. Doing so allows us to reflect upon theoretical questions regarding the structure of the mental lexicon, and also puts into perspective a number of assumptions underlying recently proposed distributional text-based models. In particular, we focus on word-embedding models which have been proposed to learn aspects of word meaning in a manner similar to humans. These are contrasted with internal language models derived from a new extensive data set of word associations. Using relatedness and similarity judgments we evaluate these models and find that the word-association-based internal language models consistently outperform current state-of-the art text-based external language models, often with a large margin. These results are not just a performance improvement; they also have implications for our understanding of how distributional knowledge is used by people.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingSemantic Textual SimilaritySimilar Papers 제목 키워드 기반
Predicting Human Similarity Judgments Using Large Language Models
Similarity judgments provide a well-established method for accessing mental representations, with applications in psychology, neuroscience and machine learning. However, collecting similarity judgments can be prohibitive…
Enriching ImageNet with Human Similarity Judgments and Psychological Embeddings
Advances in object recognition flourished in part because of the availability of high-quality datasets and associated benchmarks. However, these benchmarks---such as ILSVRC---are relatively task-specific, focusing predom…
Bayesian InferenceObject RecognitionThe Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Models
People increasingly rely on Large Language Models (LLMs) for moral advice, which may influence humans' decisions. Yet, little is known about how closely LLMs align with human moral judgments. To address this, we introduc…
Improving Correlation with Human Judgments by Integrating Semantic Similarity with Second--Order Vectors
Vector space methods that measure semantic similarity and relatedness often rely on distributional information such as co--occurrence frequencies or statistical measures of association to weight the importance of particu…
Semantic SimilaritySemantic Textual SimilarityFrom Brain Space to Distributional Space: The Perilous Journeys of fMRI Decoding
Recent work in cognitive neuroscience has introduced models for predicting distributional word meaning representations from brain imaging data. Such models have great potential, but the quality of their predictions has n…
Decoder