Generative Bridging Network for Neural Sequence Prediction
In order to alleviate data sparsity and overfitting problems in maximum likelihood estimation (MLE) for sequence prediction tasks, we propose the Generative Bridging Network (GBN), in which a novel bridge module is introduced to assist the training of the sequence prediction model (the generator network). Unlike MLE directly maximizing the conditional likelihood, the bridge extends the point-wise ground truth to a bridge distribution conditioned on it, and the generator is optimized to minimize their KL-divergence. Three different GBNs, namely uniform GBN, language-model GBN and coaching GBN, are proposed to penalize confidence, enhance language smoothness and relieve learning burden. Experiments conducted on two recognized sequence prediction tasks (machine translation and abstractive text summarization) show that our proposed GBNs can yield significant improvements over strong baselines. Furthermore, by analyzing samples drawn from different bridges, expected influences on the generator are verified.
Code (0)
등록된 구현이 없습니다.
Tasks
Abstractive Text SummarizationImage CaptioningLanguage ModelingLanguage ModellingMachine TranslationPredictionSpeech RecognitionSpelling CorrectionText SummarizationTranslationSimilar Papers 제목 키워드 기반
Generative Bridging Network in Neural Sequence Prediction
In order to alleviate data sparsity and overfitting problems in maximum likelihood estimation (MLE) for sequence prediction tasks, we propose the Generative Bridging Network (GBN), in which a novel bridge module is intro…
Abstractive Text SummarizationLanguage ModelingLanguage ModellingMachine Translation+3A Survey on Future Frame Synthesis: Bridging Deterministic and Generative Approaches
Future Frame Synthesis (FFS, aka Video Frame Prediction) focuses on generating future frame sequences conditioned on existing content. This survey provides a comprehensive review of existing research on FFS, covering com…
SurveyVideo PredictionElectron flow matching for generative reaction mechanism prediction obeying conservation laws
Central to our understanding of chemical reactivity is the principle of mass conservation, which is fundamental for ensuring physical consistency, balancing equations, and guiding reaction design. However, data-driven co…
PredictionRynnVLA-001: Using Human Demonstrations to Improve Robot Manipulation
This paper presents RynnVLA-001, a vision-language-action(VLA) model built upon large-scale video generative pretraining from human demonstrations. We propose a novel two-stage pretraining methodology. The first stage, E…
Robot ManipulationAttention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings
In neural machine translation, a source sequence of words is encoded into a vector from which a target sequence is generated in the decoding phase. Differently from statistical machine translation, the associations betwe…
Machine TranslationSentenceTranslationWord Embeddings