SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation
Semantic Textual Similarity (STS) measures the meaning similarity of sentences. Applications include machine translation (MT), summarization, generation, question answering (QA), short answer grading, semantic search, dialog and conversational systems. The STS shared task is a venue for assessing the current state-of-the-art. The 2017 task focuses on multilingual and cross-lingual pairs with one sub-track exploring MT quality estimation (MTQE) data. The task obtained strong participation from 31 teams, with 17 participating in all language tracks. We summarize performance and review a selection of well performing methods. Analysis highlights common errors, providing insight into the limitations of existing models. To support ongoing work on semantic representations, the STS Benchmark is introduced as a new shared training and evaluation set carefully selected from the corpus of English STS shared task data (2012-2017).
Code (3)
Tasks
Machine TranslationQuestion AnsweringSemantic Textual SimilaritySTSSTS BenchmarkTranslationSimilar Papers 제목 키워드 기반
SemEval-2014 Task 10: Multilingual Semantic Textual Similarity
EMBEDDIA at SemEval-2022 Task 8: Investigating Sentence, Image, and Knowledge Graph Representations for Multilingual News Article Similarity
In this paper, we present the participation of the EMBEDDIA team in the SemEval-2022 Task 8 (Multilingual News Article Similarity). We cover several techniques and propose different methods for finding the multilingual n…
ArticlesSemantic SimilaritySemantic Textual SimilaritySentenceBL.Research at SemEval-2022 Task 8: Using various Semantic Information to evaluate document-level Semantic Textual Similarity
This paper presents our system for document-level semantic textual similarity (STS) evaluation at SemEval-2022 Task 8: “Multilingual News Article Similarity”. The semantic information used is obtained by using different …
Document ClassificationSemantic Textual SimilaritySTSL2F/INESC-ID at SemEval-2017 Tasks 1 and 2: Lexical and semantic features in word and textual similarity
This paper describes our approach to the SemEval-2017 {``}Semantic Textual Similarity{''} and {``}Multilingual Word Similarity{''} tasks. In the former, we test our approach in both English and Spanish, and use a linguis…
Abstract Meaning RepresentationSemantic Textual SimilarityWord EmbeddingsWord SimilarityNeobility at SemEval-2017 Task 1: An Attention-based Sentence Similarity Model
This paper describes a neural-network model which performed competitively (top 6) at the SemEval 2017 cross-lingual Semantic Textual Similarity (STS) task. Our system employs an attention-based recurrent neural network m…
Cross-Lingual Semantic Textual SimilaritySemantic Textual SimilaritySentenceSentence Similarity+1