paper-with-me

홈 › Papers

Get the gist? Using large language models for few-shot decontextualization

2023-10-10 · Benjamin Kane, Lenhart Schubert

In many NLP applications that involve interpreting sentences within a rich context -- for instance, information retrieval systems or dialogue systems -- it is desirable to be able to preserve the sentence in a form that can be readily understood without context, for later reuse -- a process known as ``decontextualization''. While previous work demonstrated that generative Seq2Seq models could effectively perform decontextualization after being fine-tuned on a specific dataset, this approach requires expensive human annotations and may not transfer to other domains. We propose a few-shot method of decontextualization using a large language model, and present preliminary results showing that this method achieves viable performance on multiple domains using only a small set of examples.

📄 PDF Abstract BibTeX arXiv:2310.06254

Code (0)

등록된 구현이 없습니다.

Tasks

Information RetrievalLanguage ModelingLanguage ModellingLarge Language ModelRetrievalSentence

Methods 이 논문이 사용한 방법론

Tanh Activation 설명 없음
Sigmoid Activation 설명 없음
LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…
Seq2Seq Seq2Seq, or Sequence To Sequence, is a model used in sequence prediction tasks, such as language modelling and machine translation. The idea is to use one…

Similar Papers 제목 키워드 기반

DnDScore: Decontextualization and Decomposition for Factuality Verification in Long-Form Text Generation

2024-12-17 · Miriam Wanner, Benjamin Van Durme, Mark Dredze

The decompose-then-verify strategy for verification of Large Language Model (LLM) generations decomposes claims that are then independently verified. Decontextualization augments text (claims) to ensure it can be verifie…

FormLanguage ModelingLanguage ModellingLarge Language Model+1

Molecular Facts: Desiderata for Decontextualization in LLM Fact Verification

2024-06-28 · Anisha Gunjal, Greg Durrett

Automatic factuality verification of large language model (LLM) generations is becoming more and more widely used to combat hallucinations. A major point of tension in the literature is the granularity of this fact-check…

Fact CheckingFact VerificationLanguage ModelingLanguage Modelling+1

Decontextualization: Making Sentences Stand-Alone

2021-02-09 · Eunsol Choi, Jennimaria Palomaki, Matthew Lamm, Tom Kwiatkowski 외

Models for question answering, dialogue agents, and summarization often interpret the meaning of a sentence in a rich context and use that meaning in a new context. Taking excerpts of text can be problematic, as key piec…

document understandingQuestion AnsweringSentence

Learning to Reject with a Fixed Predictor: Application to Decontextualization

2023-01-22 · Christopher Mohri, Daniel Andor, Eunsol Choi, Michael Collins

We study the problem of classification with a reject option for a fixed predictor, applicable in natural language processing. We introduce a new problem formulation for this scenario, and an algorithm minimizing a new su…

SLICER: Sliced Fine-Tuning for Low-Resource Cross-Lingual Transfer for Named Entity Recognition

2022-10-01 · Proceedings of the Conference on Empirical Methods in Natural Language Processing 2022 10 · Fabian David Schmidt, Ivan Vulić, Goran Glavaš

Large multilingual language models generally demonstrate impressive results in zero-shot cross-lingual transfer, yet often fail to successfully transfer to low-resource languages, even for token-level prediction tasks li…

Cross-Lingual TransferMultilingual text classificationnamed-entity-recognitionNamed Entity Recognition+3