Structured Generative Models of Continuous Features for Word Sense Induction
We propose a structured generative latent variable model that integrates information from multiple contextual representations for Word Sense Induction. Our approach jointly models global lexical, local lexical and dependency syntactic context. Each context type is associated with a latent variable and the three types of variables share a hierarchical structure. We use skip-gram based word and dependency context embeddings to construct all three types of representations, reducing the total number of parameters to be estimated and enabling better generalization. We describe an EM algorithm to efficiently estimate model parameters and use the Integrated Complete Likelihood criterion to automatically estimate the number of senses. Our model achieves state-of-the-art results on the SemEval-2010 and SemEval-2013 Word Sense Induction datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringWord EmbeddingsWord Sense DisambiguationWord Sense InductionSimilar Papers 제목 키워드 기반
Decoder-based Sense Knowledge Distillation
Large language models (LLMs) learn contextual embeddings that capture rich semantic information, yet they often overlook structured lexical knowledge such as word senses and relationships. Prior work has shown that incor…
Knowledge DistillationTowards a principled approach to sense clustering – a case study of wordnet and dictionary senses in Danish
Our aim is to develop principled methods for sense clustering which can make existing lexical resources practically useful in NLP – not too fine-grained to be operational and yet finegrained enough to be worth the troubl…
ClusteringWord sense induction using word embeddings and community detection in complex networks
Word Sense Induction (WSI) is the ability to automatically induce word senses from corpora. The WSI task was first proposed to overcome the limitations of manually annotated corpus that are required in word sense disambi…
Community DetectionWord EmbeddingsWord Sense DisambiguationWord Sense InductionUnsupervised Learning of Syntactic Structure with Invertible Neural Projections
Unsupervised learning of syntactic structure is typically performed using generative models with discrete latent variables and multinomial parameters. In most cases, these models have not leveraged continuous word repres…
Constituency Grammar InductionDependency ParsingPOSUnsupervised Dependency ParsingChainNet: Structured Metaphor and Metonymy in WordNet
The senses of a word exhibit rich internal structure. In a typical lexicon, this structure is overlooked: a word's senses are encoded as a list without inter-sense relations. We present ChainNet, a lexical resource which…