홈 › Papers

ERNIE at SemEval-2020 Task 10: Learning Word Emphasis Selection by Pre-trained Language Model

2020-09-08 · SEMEVAL 2020 · Zhengjie Huang, Shikun Feng, Weiyue Su, Xuyi Chen, Shuohuan Wang, Jiaxiang Liu, Xuan Ouyang, Yu Sun

This paper describes the system designed by ERNIE Team which achieved the first place in SemEval-2020 Task 10: Emphasis Selection For Written Text in Visual Media. Given a sentence, we are asked to find out the most important words as the suggestion for automated design. We leverage the unsupervised pre-training model and finetune these models on our task. After our investigation, we found that the following models achieved an excellent performance in this task: ERNIE 2.0, XLM-ROBERTA, ROBERTA and ALBERT. We combine a pointwise regression loss and a pairwise ranking loss which is more close to the final M atchm metric to finetune our models. And we also find that additional feature engineering and data augmentation can help improve the performance. Our best model achieves the highest score of 0.823 and ranks first for all kinds of metrics

📄 PDF Abstract BibTeX arXiv:2009.03706

Code (0)

등록된 구현이 없습니다.

Tasks

Data Augmentation Feature EngineeringLanguage ModelingLanguage ModellingSentenceUnsupervised Pre-training

Methods 이 논문이 사용한 방법론

ERNIE ERNIE is a transformer-based model consisting of two stacked modules: 1) textual encoder and 2) knowledgeable encoder, which is responsible to integrate extra token-oriented…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Weight Decay 설명 없음

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

WordPiece 설명 없음

FPAI at SemEval-2020 Task 10: A Query Enhanced Model with RoBERTa for Emphasis Selection

2020-12-01 · SEMEVAL 2020 · Chenyang Guo, Xiaolong Hou, Junsong Ren, Lianxin Jiang 외

This paper describes the model we apply in the SemEval-2020 Task 10. We formalize the task of emphasis selection as a simplified query-based machine reading comprehension (MRC) task, i.e. answering a fixed question of {`…

Machine Reading Comprehension Reading Comprehension

IITK at SemEval-2020 Task 10: Transformers for Emphasis Selection

2020-07-21 · SEMEVAL 2020 · Vipul Singhal, Sahil Dhull, Rishabh Agarwal, Ashutosh Modi

This paper describes the system proposed for addressing the research problem posed in Task 10 of SemEval-2020: Emphasis Selection For Written Text in Visual Media. We propose an end-to-end model that takes as input the t…

EL-BERT at SemEval-2020 Task 10: A Multi-Embedding Ensemble Based Approach for Emphasis Selection in Visual Media

2020-12-01 · SEMEVAL 2020 · Chandresh Kanani, Sriparna Saha, Pushpak Bhattacharyya

In visual media, text emphasis is the strengthening of words in a text to convey the intent of the author. Text emphasis in visual media is generally done by using different colors, backgrounds, or font for the text; it …

valid

MIDAS at SemEval-2020 Task 10: Emphasis Selection using Label Distribution Learning and Contextual Embeddings

2020-09-06 · SEMEVAL 2020 · Sarthak Anand, Pradyumna Gupta, Hemant Yadav, Debanjan Mahata 외

This paper presents our submission to the SemEval 2020 - Task 10 on emphasis selection in written text. We approach this emphasis selection problem as a sequence labeling task where we represent the underlying text with …

Sentence

TextLearner at SemEval-2020 Task 10: A Contextualized Ranking System in Solving Emphasis Selection in Text

2020-12-01 · SEMEVAL 2020 · Zhishen Yang, Lars Wolfsteller, Naoaki Okazaki

This paper describes the emphasis selection system of the team TextLearner for SemEval 2020 Task 10: Emphasis Selection For Written Text in Visual Media. The system aims to learn the emphasis selection distribution using…

Language Modeling Language Modelling