Introspective Perception: Learning to Predict Failures in Vision Systems
As robots aspire for long-term autonomous operations in complex dynamic environments, the ability to reliably take mission-critical decisions in ambiguous situations becomes critical. This motivates the need to build systems that have situational awareness to assess how qualified they are at that moment to make a decision. We call this self-evaluating capability as introspection. In this paper, we take a small step in this direction and propose a generic framework for introspective behavior in perception systems. Our goal is to learn a model to reliably predict failures in a given system, with respect to a task, directly from input sensor data. We present this in the context of vision-based autonomous MAV flight in outdoor natural environments, and show that it effectively handles uncertain situations.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Competence-Aware Path Planning via Introspective Perception
Robots deployed in the real world over extended periods of time need to reason about unexpected failures, learn to predict them, and to proactively take actions to avoid future failures. Existing approaches for competenc…
IV-SLAM: Introspective Vision for Simultaneous Localization and Mapping
Existing solutions to visual simultaneous localization and mapping (V-SLAM) assume that errors in feature extraction and matching are independent and identically distributed (i.i.d), but this assumption is known to not b…
Simultaneous Localization and MappingWhen to Call an Apple Red: Humans Follow Introspective Rules, VLMs Don't
Understanding when Vision-Language Models (VLMs) will behave unexpectedly, whether models can reliably predict their own behavior, and if models adhere to their introspective reasoning are central challenges for trustwor…
How Do We Fail? Stress Testing Perception in Autonomous Vehicles
Autonomous vehicles (AVs) rely on environment perception and behavior prediction to reason about agents in their surroundings. These perception systems must be robust to adverse weather such as rain, fog, and snow. Howev…
Autonomous VehiclesData AugmentationObject TrackingTrajectory PredictionNeuro-symbolic Natural Logic with Introspective Revision for Natural Language Inference
We introduce a neuro-symbolic natural logic framework based on reinforcement learning with introspective revision. The model samples and rewards specific reasoning paths through policy gradient, in which the introspectiv…
Natural Language Inferencereinforcement-learningReinforcement Learning (RL)Systematic Generalization