Multichannel Generative Language Model: Learning All Possible Factorizations Within and Across Channels
A channel corresponds to a viewpoint or transformation of an underlying meaning. A pair of parallel sentences in English and French express the same underlying meaning, but through two separate channels corresponding to their languages. In this work, we present the Multichannel Generative Language Model (MGLM). MGLM is a generative joint distribution model over channels. MGLM marginalizes over all possible factorizations within and across all channels. MGLM endows flexible inference, including unconditional generation, conditional generation (where 1 channel is observed and other channels are generated), and partially observed generation (where incomplete observations are spread across all the channels). We experiment with the Multi30K dataset containing English, French, Czech, and German. We demonstrate experiments with unconditional, conditional, and partially conditional generation. We provide qualitative samples sampled unconditionally from the generative joint distribution. We also quantitatively analyze the quality-diversity trade-offs and find MGLM outperforms traditional bilingual discriminative models.
Code (0)
등록된 구현이 없습니다.
Tasks
AllDiversityLanguage ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
Maximal Ordinal Two-Factorizations
Given a formal context, an ordinal factor is a subset of its incidence relation that forms a chain in the concept lattice, i.e., a part of the dataset that corresponds to a linear order. To visualize the data in a formal…
RelationVocal Bursts Valence PredictionWhat is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?
This paper establishes a rigorous connection between circuit representations and tensor factorizations, two seemingly distinct yet fundamentally related areas. By connecting these fields, we highlight a series of opportu…
Factorization of Dempster-Shafer Belief Functions Based on Data
One important obstacle in applying Dempster-Shafer Theory (DST) is its relationship to frequencies. In particular, there exist serious difficulties in finding factorizations of belief functions from data. In probabilit…
On the use of generative deep neural networks to synthesize artificial multichannel EEG signals
Recent promises of generative deep learning lately brought interest to its potential uses in neural engineering. In this paper we firstly review recently emerging studies on generating artificial electroencephalography (…
EEGElectroencephalogram (EEG)Motor ImageryTime Series+1Multichannel Generative Language Models
A channel corresponds to a viewpoint or transformation of an underlying meaning. A pair of parallel sentences in English and French express the same underlying meaning but through two separate channels corresponding to t…
Machine Translation