Robust Retrieval Augmented Generation for Zero-shot Slot Filling

2021-08-31 · EMNLP 2021 11 · Michael Glass, Gaetano Rossiello, Md Faisal Mahbub Chowdhury, Alfio Gliozzo

Automatically inducing high quality knowledge graphs from a given collection of documents still remains a challenging problem in AI. One way to make headway for this problem is through advancements in a related task known as slot filling. In this task, given an entity query in form of [Entity, Slot, ?], a system is asked to fill the slot by generating or extracting the missing value exploiting evidence extracted from relevant passage(s) in the given document collection. The recent works in the field try to solve this task in an end-to-end fashion using retrieval-based language models. In this paper, we present a novel approach to zero-shot slot filling that extends dense passage retrieval with hard negatives and robust training procedures for retrieval augmented generation models. Our model reports large improvements on both T-REx and zsRE slot filling datasets, improving both passage retrieval and slot value generation, and ranking at the top-1 position in the KILT leaderboard. Moreover, we demonstrate the robustness of our system showing its domain adaptation capability on a new variant of the TACRED dataset for slot filling, through a combination of zero/few-shot learning. We release the source code and pre-trained models.

📄 PDF Abstract BibTeX arXiv:2108.13934

Code (2)

ibm/kgi-slot-filling 공식 구현 pytorch

IBM/retrieve-write-slot-filling pytorch

Tasks

Domain Adaptation Few-Shot Learning Knowledge Graphs Passage Retrieval RetrievalRetrieval-augmented Generationslot-filling Slot Filling Zero-shot Slot Filling

Audiobox TTA-RAG: Improving Zero-Shot and Few-Shot Text-To-Audio with Retrieval-Augmented Generation

2024-11-07 · Mu Yang, Bowen Shi, Matthew Le, Wei-Ning Hsu 외

This work focuses on improving Text-To-Audio (TTA) generation on zero-shot and few-shot settings (i.e. generating unseen or uncommon audio events). Inspired by the success of Retrieval-Augmented Generation (RAG) in Large…

Audio Generation Large Language Model RAG Retrieval+1

Zero-shot Slot Filling with DPR and RAG

2021-04-17 · Michael Glass, Gaetano Rossiello, Alfio Gliozzo

The ability to automatically extract Knowledge Graphs (KG) from a given collection of documents is a long-standing problem in Artificial Intelligence. One way to assess this capability is through the task of slot filling…

Knowledge Base Population Knowledge Graphs RAG Retrieval+4

Slot Dependency Modeling for Zero-Shot Cross-Domain Dialogue State Tracking

2022-10-01 · COLING 2022 10 · Qingyue Wang, Yanan Cao, Piji Li, Yanhe Fu 외

Zero-shot learning for Dialogue State Tracking (DST) focuses on generalizing to an unseen domain without the expense of collecting in domain data. However, previous zero-shot DST methods ignore the slot dependencies in a…

Dialogue State Tracking Zero-Shot Learning

GAR-meets-RAG Paradigm for Zero-Shot Information Retrieval

2023-10-31 · Daman Arora, Anush Kini, Sayak Ray Chowdhury, Nagarajan Natarajan 외

Given a query and a document corpus, the information retrieval (IR) task is to output a ranked list of relevant documents. Combining large language models (LLMs) with embedding-based retrieval models, recent work shows p…

Information Retrieval Passage Retrieval RAG Re-Ranking+2

Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation

2024-02-28 · Shicheng Xu, Liang Pang, Mo Yu, Fandong Meng 외

Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating additional information from retrieval. However, studies have shown that LLMs still face challenges in effectively using the retr…

Code Generation In-Context Learning Language Modeling Language Modelling+6