Multimodal sensor fusion in the latent representation space

2022-08-03 · Robert J. Piechocki, Xiaoyang Wang, Mohammud J. Bocus

A new method for multimodal sensor fusion is introduced. The technique relies on a two-stage process. In the first stage, a multimodal generative model is constructed from unlabelled training data. In the second stage, the generative model serves as a reconstruction prior and the search manifold for the sensor fusion tasks. The method also handles cases where observations are accessed only via subsampling i.e. compressed sensing. We demonstrate the effectiveness and excellent performance on a range of multimodal fusion experiments such as multisensory classification, denoising, and recovery from subsampled observations.

📄 PDF Abstract BibTeX arXiv:2208.02183

Code (0)

등록된 구현이 없습니다.

Tasks

compressed sensingDenoisingSensor Fusion

Latent Sensor Fusion: Multimedia Learning of Physiological Signals for Resource-Constrained Devices

2025-07-13 · Abdullah Ahmed, Jeremy Gummeson arxiv

Latent spaces offer an efficient and effective means of summarizing data while implicitly preserving meta-information through relational encoding. We leverage these meta-embeddings to develop a modality-agnostic, unified…

Shared Manifold Learning Using a Triplet Network for Multiple Sensor Translation and Fusion with Missing Data

2022-10-25 · Aditya Dutt, Alina Zare, Paul Gader

Heterogeneous data fusion can enhance the robustness and accuracy of an algorithm on a given task. However, due to the difference in various modalities, aligning the sensors and embedding their information into discrimin…

Contrastive Learning Triplet

Latent Variable Algorithms for Multimodal Learning and Sensor Fusion

2019-04-23 · Lijiang Guo

Multimodal learning has been lacking principled ways of combining information from different modalities and learning a low-dimensional manifold of meaningful representations. We study multimodal learning and sensor fusio…

Activity Recognition Decision Making Event Detection Sensor Fusion+2

Geometry of Uncertainty: Learning Metric Spaces for Multimodal State Estimation in RL

2026-02-12 · Alfredo Reichlin, Adriano Pacciarelli, Danica Kragic, Miguel Vasco arxiv

Estimating the state of an environment from high-dimensional, multimodal, and noisy observations is a fundamental challenge in reinforcement learning (RL). Traditional approaches rely on probabilistic models to account f…

Reinforcement Learning

MMLatch: Bottom-up Top-down Fusion for Multimodal Sentiment Analysis

2022-01-24 · Georgios Paraskevopoulos, Efthymios Georgiou, Alexandros Potamianos

Current deep learning approaches for multimodal fusion rely on bottom-up fusion of high and mid-level latent modality representations (late/mid fusion) or low level sensory inputs (early fusion). Models of human percepti…

Multimodal Sentiment Analysis Sentiment Analysis