Joint Generator-Ranker Learning for Natural Language Generation

2022-06-28 · Weizhou Shen, Yeyun Gong, Yelong Shen, Song Wang, Xiaojun Quan, Nan Duan, Weizhu Chen

Generate-then-rank is a widely used mechanism for text generation, where a generator produces multiple text candidates and a ranker chooses the best one among the text candidates. However, existing methods usually train the generator and the ranker individually, neglecting the mutual feedback that could further enhance the generation quality. To tackle this limitation, we propose JGR, a novel joint training algorithm that integrates the generator and the ranker in a single framework. JGR optimizes the generator with a hybrid objective that combines data likelihood and ranker reward, and trains the ranker with a contrastive loss that compares the generator outputs. By iteratively updating the generator and the ranker, JGR can effectively harmonize their learning and enhance their quality jointly. We evaluate JGR on various text generation tasks and demonstrate that it surpasses existing methods on four public datasets across three common generation scenarios. Our code and models are publicly available at https://github.com/microsoft/ProphetNet/tree/master/JGR.

📄 PDF Abstract BibTeX arXiv:2206.13974

Code (2)

microsoft/ProphetNet 공식 구현 pytorch

microsoft/advnlg 공식 구현 pytorch

Tasks

Question Generation Question-Generation Response Generation Text Generation

Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem

2026-02-21 · Lichang Song, Ting Long, Yi Chang arxiv

Retrieval-Augmented Generation (RAG) has demonstrated strong effectiveness in knowledge-intensive tasks by grounding language generation in external evidence. Despite its success, many existing RAG systems are built base…

PairReranker: Pairwise Reranking for Natural Language Generation

2022-12-20 · Dongfu Jiang, Bill Yuchen Lin, Xiang Ren

Pre-trained language models have been successful in natural language generation (NLG) tasks. While various decoding methods have been employed, they often produce suboptimal results. We first present an empirical analysi…

Machine Translation Reranking Text Generation

EnsembleGAN: Adversarial Learning for Retrieval-Generation Ensemble Model on Short-Text Conversation

2020-04-30 · Jiayi Zhang, Chongyang Tao, Zhenjing Xu, Qiaojing Xie 외

Generating qualitative responses has always been a challenge for human-computer dialogue systems. Existing dialogue systems generally derive from either retrieval-based or generative-based approaches, both of which have …

Language Modeling Language Modelling Retrieval Short-Text Conversation

Grounded Dialogue Generation with Cross-encoding Re-ranker, Grounding Span Prediction, and Passage Dropout

2022-05-01 · dialdoc (ACL) 2022 5 · Kun Li, Tianhua Zhang, Liping Tang, Junan Li 외

MultiDoc2Dial presents an important challenge on modeling dialogues grounded with multiple documents. This paper proposes a pipeline system of “retrieve, re-rank, and generate”, where each component is individually optim…

Dialogue Generation Response Generation

CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning

2025-11-24 · Jie He, Richard He Bai, Sinead Williamson, Jeff Z. Pan 외 arxiv

Retrieval-augmented generation (RAG) enhances large language models (LLMs) with external knowledge but still suffers from long contexts and disjoint retrieval-generation optimization. In this work, we propose CLaRa (Cont…

Question Answering