Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion

2022-03-24 · Kurt Shuster, Mojtaba Komeili, Leonard Adolphs, Stephen Roller, Arthur Szlam, Jason Weston

Language models (LMs) have recently been shown to generate more factual responses by employing modularity (Zhou et al., 2021) in combination with retrieval (Adolphs et al., 2021). We extend the recent approach of Adolphs et al. (2021) to include internet search as a module. Our SeeKeR (Search engine->Knowledge->Response) method thus applies a single LM to three modular tasks in succession: search, generating knowledge, and generating a final response. We show that, when using SeeKeR as a dialogue model, it outperforms the state-of-the-art model BlenderBot 2 (Chen et al., 2021) on open-domain knowledge-grounded conversations for the same number of parameters, in terms of consistency, knowledge and per-turn engagingness. SeeKeR applied to topical prompt completions as a standard language model outperforms GPT2 (Radford et al., 2019) and GPT3 (Brown et al., 2020) in terms of factuality and topicality, despite GPT3 being a vastly larger model. Our code and models are made publicly available.

📄 PDF Abstract BibTeX arXiv:2203.13224

Code (1)

facebookresearch/ParlAI pytorch

Tasks

Language ModelingLanguage Modelling Retrieval

RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation

2024-08-21 · Xuanwang Zhang, Yunze Song, Yidong Wang, Shuyun Tang 외

Large Language Models (LLMs) demonstrate human-level capabilities in dialogue, reasoning, and knowledge retention. However, even the most advanced LLMs face challenges such as hallucinations and real-time updating of the…

RAG Retrieval Retrieval-augmented Generation

Repo0: Design-Driven Zero-to-All Code Generation

2026-08-20 · Silin Chen, Haoyi Teng, Xiaodong Gu, Yuling Shi 외 arxiv

Large language model agents have made substantial progress in code generation, yet most existing systems assume a predefined repository architecture. This assumption does not hold in zero-to-all code generation, where an…

Code Generation

SEEK: Steering LLM Reasoning for RAG via Internal Reasoning Sketches

2026-01-14 · Xinze Li, Yuqing Lan, Zhenghao Liu, Haidong Xin 외 arxiv

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by incorporating external knowledge into the generation process. Benefiting from the reasoning capabilities of LLMs, existing methods have levera…

Answer Generation

UltraRAG: A Modular and Automated Toolkit for Adaptive Retrieval-Augmented Generation

2025-03-31 · Yuxuan Chen, Dewen Guo, Sen Mei, Xinze Li 외

Retrieval-Augmented Generation (RAG) significantly enhances the performance of large language models (LLMs) in downstream tasks by integrating external knowledge. To facilitate researchers in deploying RAG systems, vario…

RAG Retrieval Retrieval-augmented Generation

Distilling Reasoning Without Knowledge: A Framework for Reliable LLMs

2026-03-15 · Auksarapak Kietkajornrit, Jad Tarifi, Nima Asgharbeygi arxiv

Fact-seeking question answering with large language models (LLMs) remains unreliable when answers depend on up-to-date or conflicting information. Although retrieval-augmented and tool-using LLMs reduce hallucinations, t…

Question Answering