paper-with-me

홈 › Papers

OCHADAI-KYOTO at SemEval-2021 Task 1: Enhancing Model Generalization and Robustness for Lexical Complexity Prediction

2021-05-12 · SEMEVAL 2021 · Yuki Taya, Lis Kanashiro Pereira, Fei Cheng, Ichiro Kobayashi

We propose an ensemble model for predicting the lexical complexity of words and multiword expressions (MWEs). The model receives as input a sentence with a target word or MWEand outputs its complexity score. Given that a key challenge with this task is the limited size of annotated data, our model relies on pretrained contextual representations from different state-of-the-art transformer-based language models (i.e., BERT and RoBERTa), and on a variety of training methods for further enhancing model generalization and robustness:multi-step fine-tuning and multi-task learning, and adversarial training. Additionally, we propose to enrich contextual representations by adding hand-crafted features during training. Our model achieved competitive results and ranked among the top-10 systems in both sub-tasks.

📄 PDF Abstract BibTeX arXiv:2105.05535

Code (0)

등록된 구현이 없습니다.

Tasks

Lexical Complexity PredictionMulti-Task LearningSentence

Methods 이 논문이 사용한 방법론

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Multi-Head Attention 설명 없음
Residual Connection 설명 없음

Similar Papers 제목 키워드 기반

OCHADAI at SemEval-2022 Task 2: Adversarial Training for Multilingual Idiomaticity Detection

2022-06-07 · SemEval (NAACL) 2022 7 · Lis Kanashiro Pereira, Ichiro Kobayashi

We propose a multilingual adversarial training model for determining whether a sentence contains an idiomatic expression. Given that a key challenge with this task is the limited size of annotated data, our model relies …

SentenceTask 2

OCHADAI at SMM4H-2021 Task 5: Classifying self-reporting tweets on potential cases of COVID-19 by ensembling pre-trained language models

2021-06-01 · NAACL (SMM4H) 2021 6 · Ying Luo, Lis Pereira, Kobayashi Ichiro

Since the outbreak of coronavirus at the end of 2019, there have been numerous studies on coro- navirus in the NLP arena. Meanwhile, Twitter has been a valuable source of news and a pub- lic medium for the conveyance of …

Meta Ensemble for Japanese-Chinese Neural Machine Translation: Kyoto-U+ECNU Participation to WAT 2020

2020-12-01 · AACL (WAT) 2020 12 · Zhuoyuan Mao, Yibin Shen, Chenhui Chu, Sadao Kurohashi 외

This paper describes the Japanese-Chinese Neural Machine Translation (NMT) system submitted by the joint team of Kyoto University and East China Normal University (Kyoto-U+ECNU) to WAT 2020 (Nakazawa et al.,2020). We par…

DenoisingMachine TranslationNMTTranslation

Kyoto-NMT: a Neural Machine Translation implementation in Chainer

2016-12-01 · COLING 2016 12 · Fabien Cromi{\`e}res

We present Kyoto-NMT, an open-source implementation of the Neural Machine Translation paradigm. This implementation is done in Python and Chainer, an easy-to-use Deep Learning Framework.

Deep LearningLanguage ModelingLanguage ModellingMachine Translation+2

UCAS-IIE-NLP at SemEval-2023 Task 12: Enhancing Generalization of Multilingual BERT for Low-resource Sentiment Analysis

2023-06-01 · Dou Hu, Lingwei Wei, Yaxin Liu, Wei Zhou 외

This paper describes our system designed for SemEval-2023 Task 12: Sentiment analysis for African languages. The challenge faced by this task is the scarcity of labeled data and linguistic resources in low-resource setti…

Contrastive LearningRepresentation LearningSentiment AnalysisSentiment Classification+3