Sentence Embeddings in NLI with Iterative Refinement Encoders

2018-08-27 · Aarne Talman, Anssi Yli-Jyrä, Jörg Tiedemann

Sentence-level representations are necessary for various NLP tasks. Recurrent neural networks have proven to be very effective in learning distributed representations and can be trained efficiently on natural language inference tasks. We build on top of one such model and propose a hierarchy of BiLSTM and max pooling layers that implements an iterative refinement strategy and yields state of the art results on the SciTail dataset as well as strong results for SNLI and MultiNLI. We can show that the sentence embeddings learned in this way can be utilized in a wide variety of transfer learning tasks, outperforming InferSent on 7 out of 10 and SkipThought on 8 out of 9 SentEval sentence embedding evaluation tasks. Furthermore, our model beats the InferSent model in 8 out of 10 recently published SentEval probing tasks designed to evaluate sentence embeddings' ability to capture some of the important linguistic properties of sentences.

📄 PDF Abstract BibTeX arXiv:1808.08762

Code (1)

Helsinki-NLP/HBMP 공식 구현 pytorch

Tasks

Natural Language InferenceSentenceSentence EmbeddingSentence-EmbeddingSentence EmbeddingsTransfer Learning

Methods 이 논문이 사용한 방법론

HBMP 설명 없음

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

BiLSTM A Bidirectional LSTM, or biLSTM, is a sequence processing model that consists of two LSTMs: one taking the input in a forward direction, and the other in a backwards…

Max Pooling Max Pooling is a pooling operation that calculates the maximum value for patches of a feature map, and uses it to create a downsampled (pooled) feature map. It is usually…

Let LLMs Speak Embedding Languages: Generative Text Embeddings via Iterative Contrastive Refinement

2025-09-29 · Yu-Che Tsai, Kuan-Yu Chen, Yuan-Chi Li, Yuan-Hao Chen 외 arxiv

Existing large language model (LLM)-based embeddings typically adopt an encoder-only paradigm, treating LLMs as static feature extractors and overlooking their core generative strengths. We introduce GIRCSE (Generative I…

Representation Learning

Capturing Argument Interaction in Semantic Role Labeling with Capsule Networks

2019-10-07 · IJCNLP 2019 11 · Xinchi Chen, Chunchuan Lyu, Ivan Titov

Semantic role labeling (SRL) involves extracting propositions (i.e. predicates and their typed arguments) from natural language sentences. State-of-the-art SRL models rely on powerful encoders (e.g., LSTMs) and do not mo…

Semantic Role Labeling Sentence

Semantic Role Labeling with Iterative Structure Refinement

2019-09-07 · IJCNLP 2019 11 · Chunchuan Lyu, Shay B. Cohen, Ivan Titov

Modern state-of-the-art Semantic Role Labeling (SRL) methods rely on expressive sentence encoders (e.g., multi-layer LSTMs) but tend to model only local (if any) interactions between individual argument labeling decision…

Semantic Role Labeling Sentence

Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic Representations

2023-11-07 · Sihao Chen, Hongming Zhang, Tong Chen, Ben Zhou 외

We introduce sub-sentence encoder, a contrastively-learned contextual embedding model for fine-grained semantic representation of text. In contrast to the standard practice with sentence embeddings, where the meaning of …

Contrastive Learning Semantic Similarity Semantic Textual Similarity Sentence+1

Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity

2019-11-09 · ACL 2020 6 · Nina Poerner, Ulli Waltinger, Hinrich Schütze

We address the task of unsupervised Semantic Textual Similarity (STS) by ensembling diverse pre-trained sentence encoders into sentence meta-embeddings. We apply, extend and evaluate different meta-embedding methods from…

Dimensionality Reduction Semantic Textual Similarity Sentence STS+1