STERLING: Self-Supervised Terrain Representation Learning from Unconstrained Robot Experience
Terrain awareness, i.e., the ability to identify and distinguish different types of terrain, is a critical ability that robots must have to succeed at autonomous off-road navigation. Current approaches that provide robots with this awareness either rely on labeled data which is expensive to collect, engineered features and cost functions that may not generalize, or expert human demonstrations which may not be available. Towards endowing robots with terrain awareness without these limitations, we introduce Self-supervised TErrain Representation LearnING (STERLING), a novel approach for learning terrain representations that relies solely on easy-to-collect, unconstrained (e.g., non-expert), and unlabelled robot experience, with no additional constraints on data collection. STERLING employs a novel multi-modal self-supervision objective through non-contrastive representation learning to learn relevant terrain representations for terrain-aware navigation. Through physical robot experiments in off-road environments, we evaluate STERLING features on the task of preference-aligned visual navigation and find that STERLING features perform on par with fully supervised approaches and outperform other state-of-the-art methods with respect to preference alignment. Additionally, we perform a large-scale experiment of autonomously hiking a 3-mile long trail which STERLING completes successfully with only two manual interventions, demonstrating its robustness to real-world off-road conditions.
Code (0)
등록된 구현이 없습니다.
Tasks
Representation LearningVisual NavigationSimilar Papers 제목 키워드 기반
STERLING: Synergistic Representation Learning on Bipartite Graphs
A fundamental challenge of bipartite graph representation learning is how to extract informative node embeddings. Self-Supervised Learning (SSL) is a promising paradigm to address this challenge. Most recent bipartite gr…
Contrastive LearningGraph Representation LearningRepresentation LearningSelf-Supervised LearningBarlowWalk: Self-supervised Representation Learning for Legged Robot Terrain-adaptive Locomotion
Reinforcement learning (RL), driven by data-driven methods, has become an effective solution for robot leg motion control problems. However, the mainstream RL methods for bipedal robot terrain traversal, such as teacher-…
Self-Supervised LearningRepresentation LearningReinforcement LearningKnowledge DistillationSelf-Supervised Visual Terrain Classification from Unsupervised Acoustic Feature Learning
Mobile robots operating in unknown urban environments encounter a wide range of complex terrains to which they must adapt their planned trajectory for safe and efficient navigation. Most existing approaches utilize super…
ClassificationGeneral ClassificationSegmentationSemantic SegmentationSelf-Supervised Traversability Prediction by Learning to Reconstruct Safe Terrain
Navigating off-road with a fast autonomous vehicle depends on a robust perception system that differentiates traversable from non-traversable terrain. Typically, this depends on a semantic understanding which is based on…
Audio-Visual Self-Supervised Terrain Type Discovery for Mobile Platforms
The ability to both recognize and discover terrain characteristics is an important function required for many autonomous ground robots such as social robots, assistive robots, autonomous vehicles, and ground exploration …
Autonomous VehiclesSelf-Supervised LearningVocal Bursts Type Prediction