How do Mixture Density RNNs Predict the Future?
Gaining a better understanding of how and what machine learning systems learn is important to increase confidence in their decisions and catalyze further research. In this paper, we analyze the predictions made by a specific type of recurrent neural network, mixture density RNNs (MD-RNNs). These networks learn to model predictions as a combination of multiple Gaussian distributions, making them particularly interesting for problems where a sequence of inputs may lead to several distinct future possibilities. An example is learning internal models of an environment, where different events may or may not occur, but where the average over different events is not meaningful. By analyzing the predictions made by trained MD-RNNs, we find that their different Gaussian components have two complementary roles: 1) Separately modeling different stochastic events and 2) Separately modeling scenarios governed by different rules. These findings increase our understanding of what is learned by predictive MD-RNNs, and open up new research directions for further understanding how we can benefit from their self-organizing model decomposition.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Density Matrix Emulation of Quantum Recurrent Neural Networks for Multivariate Time Series Prediction
Quantum Recurrent Neural Networks (QRNNs) are robust candidates for modelling and predicting future values in multivariate time series. However, the effective implementation of some QRNN models is limited by the need for…
Time SeriesTime Series PredictionSafe Reinforcement Learning with Mixture Density Network: A Case Study in Autonomous Highway Driving
This paper presents a safe reinforcement learning system for automated driving that benefits from multimodal future trajectory predictions. We propose a safety system that consists of two safety components: a heuristic s…
reinforcement-learningReinforcement Learning (RL)Safe Reinforcement LearningUse of Parallel Explanatory Models to Enhance Transparency of Neural Network Configurations for Cell Degradation Detection
In a previous paper, we have shown that a recurrent neural network (RNN) can be used to detect cellular network radio signal degradations accurately. We unexpectedly found, though, that accuracy gains diminished as we ad…
Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction
Future prediction is a fundamental principle of intelligence that helps plan actions and avoid possible dangers. As the future is uncertain to a large extent, modeling the uncertainty and multimodality of the future stat…
Future predictionPredictionProbabilistic Deep LearningLearning to Predict Diverse Human Motions from a Single Image via Mixture Density Networks
Human motion prediction, which plays a key role in computer vision, generally requires a past motion sequence as input. However, in real applications, a complete and correct past motion sequence can be too expensive to a…
DiversityHuman motion predictionmotion predictionPrediction