Learning What Helps: Task-Aligned Context Selection for Vision Tasks
Humans often resolve visual uncertainty by comparing an image with relevant examples, but ViTs lack the ability to identify which examples would improve their predictions. We present Task-Aligned Context Selection (TACS), a framework that learns to select paired examples which truly improve task performance rather than those that merely appear similar. TACS jointly trains a selector network with the task model through a hybrid optimization scheme combining gradient-based supervision and reinforcement learning, making retrieval part of the learning objective. By aligning selection with task rewards, TACS enables discriminative models to discover which contextual examples genuinely help. Across 18 datasets covering fine-grained recognition, medical image classification, and medical image segmentation, TACS consistently outperforms similarity-based retrieval, particularly in challenging or data-limited settings.
Code (0)
등록된 구현이 없습니다.
Tasks
Medical Image ClassificationMedical Image SegmentationReinforcement LearningSimilar Papers 제목 키워드 기반
A Good Plan is Hard to Find: Aligning Models with Preferences is Misaligned with What Helps Users
To assist users in complex tasks, LLMs generate plans: step-by-step instructions towards a goal. While alignment methods aim to ensure LLM plans are helpful, they train (RLHF) or evaluate (ChatbotArena) on what users pre…
Question SimilarityWhat to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment
We propose an end-to-end, domain-independent neural encoder-aligner-decoder model for selective generation, i.e., the joint task of content selection and surface realization. Our model first encodes a full set of over-de…
Data-to-Text GenerationDecoderTRAD: Enhancing LLM Agents with Step-Wise Thought Retrieval and Aligned Decision
Numerous large language model (LLM) agents have been built for different tasks like web navigation and online shopping due to LLM's wide knowledge and text-understanding ability. Among these works, many of them utilize i…
Language ModellingLarge Language ModelRetrievalSequential Decision MakingSimple and Effective Text Matching with Richer Alignment Features
In this paper, we present a fast and strong neural approach for general purpose text matching applications. We explore what is sufficient to build a fast and well-performed text matching model and propose to keep three k…
Answer SelectionNatural Language InferenceParaphrase IdentificationQuestion Answering+1A Reduced-Dimension fMRI Shared Response Model
Multi-subject fMRI data is critical for evaluating the generality and validity of findings across subjects, and its effective utilization helps improve analysis sensitivity. We develop a shared response model for aggrega…
modelSensitivity