Structure-aware Sentence Encoder in Bert-Based Siamese Network

2021-08-01 · ACL (RepL4NLP) 2021 8 · Qiwei Peng, David Weir, Julie Weeds

Recently, impressive performance on various natural language understanding tasks has been achieved by explicitly incorporating syntax and semantic information into pre-trained models, such as BERT and RoBERTa. However, this approach depends on problem-specific fine-tuning, and as widely noted, BERT-like models exhibit weak performance, and are inefficient, when applied to unsupervised similarity comparison tasks. Sentence-BERT (SBERT) has been proposed as a general-purpose sentence embedding method, suited to both similarity comparison and downstream tasks. In this work, we show that by incorporating structural information into SBERT, the resulting model outperforms SBERT and previous general sentence encoders on unsupervised semantic textual similarity (STS) datasets and transfer classification tasks.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Natural Language Understanding Semantic Textual SimilaritySentenceSentence EmbeddingSentence-EmbeddingSTS

Methods 이 논문이 사용한 방법론

SBERT 설명 없음

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

2019-08-27 · IJCNLP 2019 11 · Nils Reimers, Iryna Gurevych

BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019) has set a new state-of-the-art performance on sentence-pair regression tasks like semantic textual similarity (STS). However, it requires that both sentences are …

Clustering Linear-Probe Classification Semantic Similarity Semantic Textual Similarity+6

Dual-View Distilled BERT for Sentence Embedding

2021-04-18 · Xingyi Cheng

Recently, BERT realized significant progress for sentence matching via word-level cross sentence attention. However, the performance significantly drops when using siamese BERT-networks to derive two sentence embeddings,…

Sentence Sentence Embedding Sentence-Embedding Sentence Embeddings+1

Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Tasks

2021-01-26 · Hyunjin Choi, Judong Kim, Seongho Joe, Youngjune Gwon

Contextualized representations from a pre-trained language model are central to achieve a high performance on downstream NLP task. The pre-trained BERT and A Lite BERT (ALBERT) models can be fine-tuned to give state-ofth…

Language Modelling Natural Language Inference Semantic Textual Similarity Sentence+5

Query Focused Multi-document Summarisation of Biomedical Texts

2020-08-27 · Diego Molla, Christopher Jones, Vincent Nguyen

This paper presents the participation of Macquarie University and the Australian National University for Task B Phase B of the 2020 BioASQ Challenge (BioASQ8b). Our overall framework implements Query focused multi-docume…

regression reinforcement-learning Reinforcement Learning (RL)Sentence+2

Extractive Summarization for Arabic Documents Using SAraBERT with a Semantic Siamese Similarity Evaluation Metric

2026-08-21 · Sami Shames El Deen, Mariette Awad arxiv

In this research, we introduce SAraBERT, an enhanced version of AraBERT which proposes inter-sentence transformer layers for extractive summarization tasks. To ensure that the summaries generated by SAraBERT achieve a hi…