GAPX: Generalized Autoregressive Paraphrase-Identification X
Paraphrase Identification is a fundamental task in Natural Language Processing. While much progress has been made in the field, the performance of many state-of-the-art models often suffer from distribution shift during inference time. We verify that a major source of this performance drop comes from biases introduced by negative examples. To overcome these biases, we propose in this paper to train two separate models, one that only utilizes the positive pairs and the other the negative pairs. This enables us the option of deciding how much to utilize the negative model, for which we introduce a perplexity based out-of-distribution metric that we show can effectively and automatically determine how much weight it should be given during inference. We support our findings with strong empirical results.
Code (1)
Tasks
Paraphrase IdentificationSimilar Papers 제목 키워드 기반
SMCLM: Semantically Meaningful Causal Language Modeling for Autoregressive Paraphrase Generation
This article introduces semantically meaningful causal language modeling (SMCLM), a selfsupervised method of training autoregressive models to generate semantically equivalent text. Our approach involves using semantical…
Paraphrase GenerationImproving Large-scale Paraphrase Acquisition and Generation
This paper addresses the quality issues in existing Twitter-based paraphrase datasets, and discusses the necessity of using two separate definitions of paraphrase for identification and generation tasks. We present a new…
Language ModelingLanguage ModellingParaphrase GenerationParaphrase Identification+1Discriminative Phrase Embedding for Paraphrase Identification
This work, concerning paraphrase identification task, on one hand contributes to expanding deep learning embeddings to include continuous and discontinuous linguistic phrases. On the other hand, it comes up with a new sc…
Paraphrase IdentificationRuPAWS: A Russian Adversarial Dataset for Paraphrase Identification
Paraphrase identification task can be easily challenged by changing word order, e.g. as in “Can a good person become bad?”. While for English this problem was tackled by the PAWS dataset (Zhang et al., 2019), datasets fo…
Paraphrase IdentificationOn Paraphrase Identification Corpora
We analyze in this paper a number of data sets proposed over the last decade or so for the task of paraphrase identification. The goal of the analysis is to identify the advantages as well as shortcomings of the previous…
DiversityNatural Language InferenceParaphrase IdentificationSemantic Similarity+1