Enhancing BERT for Lexical Normalization
Language model-based pre-trained representations have become ubiquitous in natural language processing. They have been shown to significantly improve the performance of neural models on a great variety of tasks. However, it remains unclear how useful those general models can be in handling non-canonical text. In this article, focusing on User Generated Content (UGC), we study the ability of BERT to perform lexical normalisation. Our contribution is simple: by framing lexical normalisation as a token prediction task, by enhancing its architecture and by carefully fine-tuning it, we show that BERT can be a competitive lexical normalisation model without the need of any UGC resources aside from 3,000 training sentences. To the best of our knowledge, it is the first work done in adapting and analysing the ability of this model to handle noisy UGC data.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingLexical NormalizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DaN+: Danish Nested Named Entities and Lexical Normalization
This paper introduces DaN+, a new multi-domain corpus and annotation guidelines for Danish nested named entities (NEs) and lexical normalization to support research on cross-lingual cross-domain learning for a less-resou…
Cross-Lingual TransferLexical NormalizationMulti-Task Learningnamed-entity-recognition+3Sesame Street to Mount Sinai: BERT-constrained character-level Moses models for multilingual lexical normalization
This paper describes the HEL-LJU submissions to the MultiLexNorm shared task on multilingual lexical normalization. Our system is based on a BERT token classification preprocessing step, where for each token the type of …
Lexical Normalizationtoken-classificationToken ClassificationFrom Lemmas to Dependencies: What Signals Drive Light Verbs Classification?
Light verb constructions (LVCs) are a challenging class of verbal multiword expressions, especially in Turkish, where rich morphology and productive complex predicates create minimal contrasts between idiomatic predicate…
Norm It! Lexical Normalization for Italian and Its Downstream Effects for Dependency Parsing
Lexical normalization is the task of translating non-standard social media data to a standard form. Previous work has shown that this is beneficial for many downstream tasks in multiple languages. However, for Italian, t…
Dependency ParsingLexical NormalizationOCHADAI-KYOTO at SemEval-2021 Task 1: Enhancing Model Generalization and Robustness for Lexical Complexity Prediction
We propose an ensemble model for predicting the lexical complexity of words and multiword expressions (MWEs). The model receives as input a sentence with a target word or MWEand outputs its complexity score. Given that a…
Lexical Complexity PredictionMulti-Task LearningSentence