paper-with-me

Papers

Hierarchical Latent Structure for Multi-Modal Vehicle Trajectory Forecasting

2022-07-11 · Dooseop Choi, Kyoungwook Min

Variational autoencoder (VAE) has widely been utilized for modeling data distributions because it is theoretically elegant, easy to train, and has nice manifold representations. However, when applied to image reconstruction and synthesis tasks, VAE shows the limitation that the generated sample tends to be blurry. We observe that a similar problem, in which the generated trajectory is located between adjacent lanes, often arises in VAE-based trajectory forecasting models. To mitigate this problem, we introduce a hierarchical latent structure into the VAE-based forecasting model. Based on the assumption that the trajectory distribution can be approximated as a mixture of simple distributions (or modes), the low-level latent variable is employed to model each mode of the mixture and the high-level latent variable is employed to represent the weights for the modes. To model each mode accurately, we condition the low-level latent variable using two lane-level context vectors computed in novel ways, one corresponds to vehicle-lane interaction and the other to vehicle-vehicle interaction. The context vectors are also used to model the weights via the proposed mode selection network. To evaluate our forecasting model, we use two large-scale real-world datasets. Experimental results show that our model is not only capable of generating clear multi-modal trajectory distributions but also outperforms the state-of-the-art (SOTA) models in terms of prediction accuracy. Our code is available at https://github.com/d1024choi/HLSTrajForecast.

📄 PDF Abstract BibTeX arXiv:2207.04624

Code (1)

d1024choi/hlstrajforecast 공식 구현 pytorch

Tasks

Image ReconstructionTrajectory Forecasting

Similar Papers 제목 키워드 기반

Hierarchical Contrastive Learning for Multimodal Data

2026-04-07 · Huichao Li, Junhan Yu, Doudou Zhou arxiv

Multimodal representation learning is commonly built on a shared-private decomposition, treating latent information as either common to all modalities or specific to one. This binary view is often inadequate: many factor…

Representation LearningContrastive Learning

Hierarchical Multimodal Variational Autoencoders

2021-09-29 · Jannik Wolff, Rahul G Krishnan, Lukas Ruff, Jan Nikolas Morshuis 외

Humans find structure in natural phenomena by absorbing stimuli from multiple input sources such as vision, text, and speech. We study the use of deep generative models that generate multimodal data from latent represent…

Neural Nonnegative CP Decomposition for Hierarchical Tensor Analysis

2021-01-01 · Joshua Vendrow, Jamie Haddock, Deanna Needell

There is a significant demand for topic modeling on large-scale data with complex multi-modal structure in applications such as multi-layer network analysis, temporal document classification, and video data analysis; fre…

Document Classification

Unified Brain MR-Ultrasound Synthesis using Multi-Modal Hierarchical Representations

2023-09-15 · Reuben Dorent, Nazim Haouchine, Fryderyk Kögl, Samuel Joutard 외

We introduce MHVAE, a deep hierarchical variational auto-encoder (VAE) that synthesizes missing images from various modalities. Extending multi-modal VAEs with a hierarchical latent structure, we introduce a probabilisti…

Exploiting Data Hierarchy as a New Modality for Contrastive Learning

2024-01-06 · Arjun Bhalla, Daniel Levenson, Jan Bernhard, Anton Abilov

This work investigates how hierarchically structured data can help neural networks learn conceptual representations of cathedrals. The underlying WikiScenes dataset provides a spatially organized hierarchical structure o…

Contrastive LearningSelf-Supervised LearningTripletWeakly-supervised Learning