Learning active tactile perception through belief-space control
Robots operating in an open world will encounter novel objects with unknown physical properties, such as mass, friction, or size. These robots will need to sense these properties through interaction prior to performing downstream tasks with the objects. We propose a method that autonomously learns tactile exploration policies by developing a generative world model that is leveraged to 1) estimate the object's physical parameters using a differentiable Bayesian filtering algorithm and 2) develop an exploration policy using an information-gathering model predictive controller. We evaluate our method on three simulated tasks where the goal is to estimate a desired object property (mass, height or toppling height) through physical interaction. We find that our method is able to discover policies that efficiently gather information about the desired property in an intuitive manner. Finally, we validate our method on a real robot system for the height estimation task, where our method is able to successfully learn and execute an information-gathering policy from scratch.
Code (0)
등록된 구현이 없습니다.
Tasks
FrictionSimilar Papers 제목 키워드 기반
Bayesian Active Object Recognition and 6D Pose Estimation from Multimodal Contact Sensing
We present an active tactile exploration framework for joint object recognition and 6D pose estimation. The proposed method integrates wrist force/torque sensing, GelSight tactile sensing, and free-space constraints with…
Object RecognitionBayesian Inference6D Pose EstimationMotion PlanningCross-Modal Visuo-Tactile Object Perception
Estimating physical properties is critical for safe and efficient autonomous robotic manipulation, particularly during contact-rich interactions. In such settings, vision and tactile sensing provide complementary informa…
Bayesian InferenceTactile MNIST: Benchmarking Active Tactile Perception
Tactile perception has the potential to significantly enhance dexterous robotic manipulation by providing rich local information that can complement or substitute for other sensory modalities such as vision. However, bec…
BenchmarkingScene UnderstandingActive Perception for Tactile Sensing: A Task-Agnostic Attention-Based Approach
Humans make extensive use of haptic exploration to map and identify the properties of the objects that we touch. In robotics, active tactile perception has emerged as an important research domain that complements vision …
Decision MakingPose EstimationReinforcement Learning (RL)Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?
Spatial embodied intelligence requires agents to act to acquire information under partial observability. While multimodal foundation models excel at passive perception, their capacity for active, self-directed exploratio…