FRMDN: Flow-based Recurrent Mixture Density Network
The class of recurrent mixture density networks is an important class of probabilistic models used extensively in sequence modeling and sequence-to-sequence mapping applications. In this class of models, the density of a target sequence in each time-step is modeled by a Gaussian mixture model with the parameters given by a recurrent neural network. In this paper, we generalize recurrent mixture density networks by defining a Gaussian mixture model on a non-linearly transformed target sequence in each time-step. The non-linearly transformed space is created by normalizing flow. We observed that this model significantly improves the fit to image sequences measured by the log-likelihood. We also applied the proposed model on some speech and image data, and observed that the model has significant modeling power outperforming other state-of-the-art methods in terms of the log-likelihood.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Supervised and Semi-supervised Deep Probabilistic Models for Indoor Positioning Problems
Predicting smartphone users location with WiFi fingerprints has been a popular research topic recently. In this work, we propose two novel deep learning-based models, the convolutional mixture density recurrent neural ne…
Convolutional Mixture Density Recurrent Neural Network for Predicting User Location with WiFi Fingerprints
Predicting smartphone users activity using WiFi fingerprints has been a popular approach for indoor positioning in recent years. However, such a high dimensional time-series prediction problem can be very tricky to solve…
Time SeriesTime Series AnalysisTime Series PredictionLinear pretraining in recurrent mixture density networks
We present a method for pretraining a recurrent mixture density network (RMDN). We also propose a slight modification to the architecture of the RMDN-GARCH proposed by Nikolaev et al. [2012]. The pretraining method helps…
Attention-based Mixture Density Recurrent Networks for History-based Recommendation
The goal of personalized history-based recommendation is to automatically output a distribution over all the items given a sequence of previous purchases of a user. In this work, we present a novel approach that uses a r…
Density Deconvolution with Normalizing Flows
Density deconvolution is the task of estimating a probability density function given only noise-corrupted samples. We can fit a Gaussian mixture model to the underlying density by maximum likelihood if the noise is norma…
Density EstimationVariational Inference