Hierarchical Imitation Learning with Vector Quantized Models
The ability to plan actions on multiple levels of abstraction enables intelligent agents to solve complex tasks effectively. However, learning the models for both low and high-level planning from demonstrations has proven challenging, especially with higher-dimensional inputs. To address this issue, we propose to use reinforcement learning to identify subgoals in expert trajectories by associating the magnitude of the rewards with the predictability of low-level actions given the state and the chosen subgoal. We build a vector-quantized generative model for the identified subgoals to perform subgoal-level planning. In experiments, the algorithm excels at solving complex, long-horizon decision-making problems outperforming state-of-the-art. Because of its ability to plan, our algorithm can find better trajectories than the ones in the training set
Code (1)
Tasks
Decision MakingImitation LearningSimilar Papers 제목 키워드 기반
S-HR-VQVAE: Sequential Hierarchical Residual Learning Vector Quantized Variational Autoencoder for Video Prediction
We address the video prediction task by putting forth a novel model that combines (i) a novel hierarchical residual learning vector quantized variational autoencoder (HR-VQVAE), and (ii) a novel autoregressive spatiotemp…
PredictionVideo PredictionHierarchical Vector Quantized Transformer for Multi-class Unsupervised Anomaly Detection
Unsupervised image Anomaly Detection (UAD) aims to learn robust and discriminative representations of normal samples. While separate solutions per class endow expensive computation and limited generalizability, this pape…
Anomaly DetectionMulti-class Anomaly DetectionQuantizationUnsupervised Anomaly DetectionHDT: Hierarchical Discrete Transformer for Multivariate Time Series Forecasting
Generative models have gained significant attention in multivariate time series forecasting (MTS), particularly due to their ability to generate high-fidelity samples. Forecasting the probability distribution of multivar…
Multivariate Time Series ForecastingTime SeriesTime Series ForecastingHierarchical Residual Learning Based Vector Quantized Variational Autoencoder for Image Reconstruction and Generation
We propose a multi-layer variational autoencoder method, we call HR-VQVAE, that learns hierarchical discrete representations of the data. By utilizing a novel objective function, each layer in HR-VQVAE learns a discrete …
DecoderImage GenerationImage ReconstructionHierarchical Quantized Autoencoders
Despite progress in training neural networks for lossy image compression, current approaches fail to maintain both perceptual quality and abstract features at very low bitrates. Encouraged by recent success in learning d…
Image CompressionQuantization