FlexiTerm: A more efficient implementation of flexible multi-word term recognition
Terms are linguistic signifiers of domain-specific concepts. Automated recognition of multi-word terms (MWT) in free text is a sequence labelling problem, which is commonly addressed using supervised machine learning methods. Their need for manual annotation of training data makes it difficult to port such methods across domains. FlexiTerm, on the other hand, is a fully unsupervised method for MWT recognition from domain-specific corpora. Originally implemented in Java as a proof of concept, it did not scale well, thus offering little practical value in the context of big data. In this paper, we describe its re-implementation in Python and compare the performance of these two implementations. The results demonstrated major improvements in terms of efficiency, which allow FlexiTerm to transition from the proof of concept to the production-grade application.
Code (1)
Similar Papers 제목 키워드 기반
Object-oriented lexical encoding of multiword expressions: Short and sweet
Multiword expressions (MWEs) exhibit both regular and idiosyncratic properties. Their idiosyncrasy requires lexical encoding in parallel with their component words. Their (at times intricate) regularity, on the other han…
SupWSD: A Flexible Toolkit for Supervised Word Sense Disambiguation
In this demonstration we present SupWSD, a Java API for supervised Word Sense Disambiguation (WSD). This toolkit includes the implementation of a state-of-the-art supervised WSD system, together with a Natural Language P…
Word Sense DisambiguationProbabilistic Embeddings with Laplacian Graph Priors
We introduce probabilistic embeddings using Laplacian priors (PELP). The proposed model enables incorporating graph side-information into static word embeddings. We theoretically show that the model unifies several previ…
Word EmbeddingsSwitch-LSTMs for Multi-Criteria Chinese Word Segmentation
Multi-criteria Chinese word segmentation is a promising but challenging task, which exploits several different segmentation criteria and mines their common underlying knowledge. In this paper, we propose a flexible multi…
Chinese Word SegmentationSegmentationPractical Approach on Implementation of WordNets for South African Languages
This paper proposes the implementation of WordNets for five South African languages, namely, Sepedi, Setswana, Tshivenda, isiZulu and isiXhosa to be added to open multilingual WordNets (OMW) on natural language toolkit (…