Learning to Navigate for Fine-grained Classification
Fine-grained classification is challenging due to the difficulty of finding discriminative features. Finding those subtle traits that fully characterize the object is not straightforward. To handle this circumstance, we propose a novel self-supervision mechanism to effectively localize informative regions without the need of bounding-box/part annotations. Our model, termed NTS-Net for Navigator-Teacher-Scrutinizer Network, consists of a Navigator agent, a Teacher agent and a Scrutinizer agent. In consideration of intrinsic consistency between informativeness of the regions and their probability being ground-truth class, we design a novel training paradigm, which enables Navigator to detect most informative regions under the guidance from Teacher. After that, the Scrutinizer scrutinizes the proposed regions from Navigator and makes predictions. Our model can be viewed as a multi-agent cooperation, wherein agents benefit from each other, and make progress together. NTS-Net can be trained end-to-end, while provides accurate fine-grained classification predictions as well as highly informative regions during inference. We achieve state-of-the-art performance in extensive benchmark datasets.
Code (12)
Tasks
Fine-Grained Image ClassificationGeneral ClassificationInformativenessSimilar Papers 제목 키워드 기반
Fine-Grained Alignment in Vision-and-Language Navigation through Bayesian Optimization
This paper addresses the challenge of fine-grained alignment in Vision-and-Language Navigation (VLN) tasks, where robots navigate realistic 3D environments based on natural language instructions. Current approaches use c…
Bayesian OptimizationContrastive LearningNavigateVision and Language NavigationLandmark-RxR: Solving Vision-and-Language Navigation with Fine-Grained Alignment Supervision
In Vision-and-Language Navigation (VLN) task, an agent is asked to navigate inside 3D indoor environments following given instructions. Cross-modal alignment is one of the most critical challenges in VLN because the pred…
cross-modal alignmentNavigateVision and Language NavigationThink Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation
Following language instructions to navigate in unseen environments is a challenging problem for autonomous embodied agents. The agent not only needs to ground languages in visual scenes, but also should explore the envir…
Efficient ExplorationNavigateVision and Language NavigationVisual NavigationFineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigation
UAV vision-language navigation (VLN) requires an agent to navigate complex 3D environments from an egocentric perspective while following ambiguous multi-step instructions over long horizons. Existing zero-shot methods r…
Vision-Language NavigationMuGER$^2$: Multi-Granularity Evidence Retrieval and Reasoning for Hybrid Question Answering
Hybrid question answering (HQA) aims to answer questions over heterogeneous data, including tables and passages linked to table cells. The heterogeneous data can provide different granularity evidence to HQA models, e.t.…
NavigateQuestion AnsweringRetrieval