Style-transfer and Paraphrase: Looking for a Sensible Semantic Similarity Metric
The rapid development of such natural language processing tasks as style transfer, paraphrase, and machine translation often calls for the use of semantic similarity metrics. In recent years a lot of methods to measure the semantic similarity of two short texts were developed. This paper provides a comprehensive analysis for more than a dozen of such methods. Using a new dataset of fourteen thousand sentence pairs human-labeled according to their semantic similarity, we demonstrate that none of the metrics widely used in the literature is close enough to human judgment in these tasks. A number of recently proposed metrics provide comparable results, yet Word Mover Distance is shown to be the most reasonable solution to measure semantic similarity in reformulated texts at the moment.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationSemantic SimilaritySemantic Textual SimilaritySentenceStyle TransferTranslationSimilar Papers 제목 키워드 기반
Reformulating Unsupervised Style Transfer as Paraphrase Generation
Modern NLP defines the task of style transfer as modifying the style of a given sentence without appreciably changing its semantics, which implies that the outputs of style transfer systems should be paraphrases of their…
AttributeParaphrase GenerationSentenceStyle TransferParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer
Textual style transfer is the task of transforming stylistic properties of text while preserving meaning. Target "styles" can be defined in numerous ways, ranging from single attributes (e.g, formality) to authorship (e.…
Style TransferTowards Actual (Not Operational) Textual Style Transfer Auto-Evaluation
Regarding the problem of automatically generating paraphrases with modified styles or attributes, the difficulty lies in the lack of parallel corpora. Numerous advances have been proposed for the generation. However, sig…
Semantic SimilaritySemantic Textual SimilarityStyle TransferMahaParaphrase: A Marathi Paraphrase Detection Corpus and BERT-based Models
Paraphrases are a vital tool to assist language understanding tasks such as question answering, style transfer, semantic parsing, and data augmentation tasks. Indic languages are complex in natural language processing (N…
Question AnsweringData AugmentationSemantic ParsingStyle TransferAn Empirical Study on Multi-Task Learning for Text Style Transfer and Paraphrase Generation
The topic of this paper is neural multi-task training for text style transfer. We present an efficient method for neutral-to-style transformation using the transformer framework. We demonstrate how to prepare a robust mo…
Multi-Task LearningParaphrase GenerationStyle TransferText Style Transfer