SCIM: Simultaneous Clustering, Inference, and Mapping for Open-World Semantic Scene Understanding
In order to operate in human environments, a robot's semantic perception has to overcome open-world challenges such as novel objects and domain gaps. Autonomous deployment to such environments therefore requires robots to update their knowledge and learn without supervision. We investigate how a robot can autonomously discover novel semantic classes and improve accuracy on known classes when exploring an unknown environment. To this end, we develop a general framework for mapping and clustering that we then use to generate a self-supervised learning signal to update a semantic segmentation model. In particular, we show how clustering parameters can be optimized during deployment and that fusion of multiple observation modalities improves novel object discovery compared to prior work. Models, data, and implementations can be found at https://github.com/hermannsblum/scim
Code (1)
Tasks
ClusteringObject DiscoveryScene UnderstandingSelf-Supervised LearningSemantic SegmentationSimilar Papers 제목 키워드 기반
Scientific Machine Learning Seismology
Scientific machine learning (SciML) is an interdisciplinary research field that integrates machine learning, particularly deep learning, with physics theory to understand and predict complex natural phenomena. By incorpo…
Deep LearningOperator learningInclusion of Lithological terms (rocks and minerals) in The Open Wordnet for English
We extend the Open WordNet for English (OWN-EN) with rock-related and other lithological terms using the authoritative source of GBA{'}s Thesaurus. Our aim is to improve WordNet to better function within Oil {\&} Gas dom…
A Robust Learning Methodology for Uncertainty-aware Scientific Machine Learning models
Robust learning is an important issue in Scientific Machine Learning (SciML). There are several works in the literature addressing this topic. However, there is an increasing demand for methods that can simultaneously co…
LUMOS: Democratizing SciML Workflows with L0-Regularized Learning for Unified Feature and Parameter Adaptation
The rapid growth of scientific machine learning (SciML) has accelerated discovery across diverse domains, yet designing effective SciML models remains a challenging task. In practice, building such models often requires …
Introduction to optimization methods for training SciML models
Optimization is central to both modern machine learning (ML) and scientific machine learning (SciML), yet the structure of the underlying optimization problems differs substantially across these domains. Classical ML typ…