Sparse Gaussian Process Temporal Difference Learning for Marine Robot Navigation
We present a method for Temporal Difference (TD) learning that addresses several challenges faced by robots learning to navigate in a marine environment. For improved data efficiency, our method reduces TD updates to Gaussian Process regression. To make predictions amenable to online settings, we introduce a sparse approximation with improved quality over current rejection-based sparse methods. We derive the predictive value function posterior and use the moments to obtain a new algorithm for model-free policy evaluation, SPGP-SARSA. With simple changes, we show SPGP-SARSA can be reduced to a model-based equivalent, SPGP-TD. We perform comprehensive simulation studies and also conduct physical learning trials with an underwater robot. Our results show SPGP-SARSA can outperform the state-of-the-art sparse method, replicate the prediction quality of its exact counterpart, and be applied to solve underwater navigation tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Marine Robot NavigationNavigateRobot NavigationSimilar Papers 제목 키워드 기반
Informative Planning and Online Learning with Sparse Gaussian Processes
A big challenge in environmental monitoring is the spatiotemporal variation of the phenomena to be observed. To enable persistent sensing and estimation in such a setting, it is beneficial to have a time-varying underlyi…
Gaussian ProcessesRecursive Sparse Pseudo-input Gaussian Process SARSA
The class of Gaussian Process (GP) methods for Temporal Difference learning has shown promise for data-efficient model-free Reinforcement Learning. In this paper, we consider a recent variant of the GP-SARSA algorithm, c…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Spatiotemporal Degradation-Aware 3D Gaussian Splatting for Realistic Underwater Scene Reconstruction
Reconstructing realistic underwater scenes from underwater video remains a meaningful yet challenging task in the multimedia domain. The inherent spatiotemporal degradations in underwater imaging, including caustics, fli…
Novel View Synthesis3D ReconstructionSpatio-Temporal Structured Sparse Regression with Hierarchical Gaussian Process Priors
This paper introduces a new sparse spatio-temporal structured Gaussian process regression framework for online and offline Bayesian inference. This is the first framework that gives a time-evolving representation of the …
Bayesian InferenceEEGElectroencephalogram (EEG)object-detection+2MarineFormer: A Spatio-Temporal Attention Model for USV Navigation in Dynamic Marine Environments
Navigating autonomously in marine environments including dynamic and static obstacles, and strong flow disturbances, such as in high-flow rivers, poses significant challenges for USVs. To address these challenges, we pro…
Collision AvoidanceGraph AttentionReinforcement Learning (RL)Temporal Sequences