Generative and Discriminative Text Classification with Recurrent Neural Networks

2017-03-06 · Dani Yogatama, Chris Dyer, Wang Ling, Phil Blunsom

We empirically characterize the performance of discriminative and generative LSTM models for text classification. We find that although RNN-based generative models are more powerful than their bag-of-words ancestors (e.g., they account for conditional dependencies across words in a document), they have higher asymptotic error rates than discriminatively trained RNN models. However we also find that generative models approach their asymptotic error rate more rapidly than their discriminative counterparts---the same pattern that Ng & Jordan (2001) proved holds for linear classification models that make more naive conditional independence assumptions. Building on this finding, we hypothesize that RNN-based generative classification models will be more robust to shifts in the data distribution. This hypothesis is confirmed in a series of experiments in zero-shot and continual learning settings that show that generative models substantially outperform discriminative models.

📄 PDF Abstract BibTeX arXiv:1703.01898

Code (2)

johnr0/TaleBrush-backend pytorch

salesforce/GeDi pytorch

Tasks

Classification Continual Learning General Classification text-classification Text Classification

Jointly Discriminative and Generative Recurrent Neural Networks for Learning from fMRI

2019-10-15 · Nicha C. Dvornek, Xiaoxiao Li, Juntang Zhuang, James S. Duncan

Recurrent neural networks (RNNs) were designed for dealing with time-series data and have recently been used for creating predictive models from functional magnetic resonance imaging (fMRI) data. However, gathering large…

Classification General Classification Time Series Time Series Analysis

Deep Recurrent Generative Decoder for Abstractive Text Summarization

2017-08-02 · EMNLP 2017 9 · Piji Li, Wai Lam, Lidong Bing, ZiHao Wang

We propose a new framework for abstractive text summarization based on a sequence-to-sequence oriented encoder-decoder model equipped with a deep recurrent generative decoder (DRGN). Latent structure information implie…

Abstractive Text Summarization Decoder Text Summarization Variational Inference

Generative adversarial networks for generation and classification of physical rehabilitation movement episodes

2018-12-15 · L. Li, A. Vakanski

This article proposes a method for mathematical modeling of human movements related to patient exercise episodes performed during physical therapy sessions by using artificial neural networks. The generative adversarial …

General Classification Generative Adversarial Network

Contextual Bidirectional Long Short-Term Memory Recurrent Neural Network Language Models: A Generative Approach to Sentiment Analysis

2017-04-01 · EACL 2017 4 · Amr Mousa, Bj{\"o}rn Schuller

Traditional learning-based approaches to sentiment analysis of written text use the concept of bag-of-words or bag-of-n-grams, where a document is viewed as a set of terms or short combinations of terms disregarding gram…

Decision Making Dimensionality Reduction Opinion Mining Sentence+3

Text Generation Based on Generative Adversarial Nets with Latent Variable

2017-12-01 · Heng Wang, Zengchang Qin, Tao Wan

In this paper, we propose a model using generative adversarial net (GAN) to generate realistic text. Instead of using standard GAN, we combine variational autoencoder (VAE) with generative adversarial net. The use of hig…

Language Modeling Language Modelling Text Generation