Look-ahead before you leap: end-to-end active recognition by forecasting the effect of motion
Visual recognition systems mounted on autonomous moving agents face the challenge of unconstrained data, but simultaneously have the opportunity to improve their performance by moving to acquire new views of test data. In this work, we first show how a recurrent neural network-based system may be trained to perform end-to-end learning of motion policies suited for this "active recognition" setting. Further, we hypothesize that active vision requires an agent to have the capacity to reason about the effects of its motions on its view of the world. To verify this hypothesis, we attempt to induce this capacity in our active recognition pipeline, by simultaneously learning to forecast the effects of the agent's motions on its internal representation of the environment conditional on all past views. Results across two challenging datasets confirm both that our end-to-end system successfully learns meaningful policies for active category recognition, and that "learning to look ahead" further boosts recognition performance.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation
Existing research studies on vision and language grounding for robot navigation focus on improving model-free deep reinforcement learning (DRL) models in synthetic environments. However, model-free DRL models do not cons…
Deep Reinforcement LearningmodelModel-based Reinforcement Learningreinforcement-learning+5Look Before Leap: Look-Ahead Planning with Uncertainty in Reinforcement Learning
Model-based reinforcement learning (MBRL) has demonstrated superior sample efficiency compared to model-free reinforcement learning (MFRL). However, the presence of inaccurate models can introduce biases during policy le…
Atari GamesModel-based Reinforcement LearningCan a Small Model Learn to Look Before It Leaps? Dynamic Learning and Proactive Correction for Hallucination Detection
Hallucination in large language models (LLMs) remains a critical barrier to their safe deployment. For hallucination detection to be practical in real-world scenarios, the use of efficient small models is essential to en…
LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detection
Diffusion Language Models (dLLMs) have garnered significant attention for their potential in highly parallel processing. The parallel capabilities of existing dLLMs stem from the assumption of conditional independence at…
Uncertainty-aware Active Learning for Optimal Bayesian Classifier
For pool-based active learning, in each iteration a candidate training sample is chosen for labeling by optimizing an acquisition function. Expected Loss Reduction~(ELR) methods maximize the expected reduction in the cla…
Active LearningClassificationGeneral Classification