Embracing Collaboration Over Competition: Condensing Multiple Prompts for Visual In-Context Learning
Visual In-Context Learning (VICL) enables adaptively solving vision tasks by leveraging pixel demonstrations, mimicking human-like task completion through analogy. Prompt selection is critical in VICL, but current methods assume the existence of a single "ideal" prompt in a pool of candidates, which in practice may not hold true. Multiple suitable prompts may exist, but individually they often fall short, leading to difficulties in selection and the exclusion of useful context. To address this, we propose a new perspective: prompt condensation. Rather than relying on a single prompt, candidate prompts collaborate to efficiently integrate informative contexts without sacrificing resolution. We devise Condenser, a lightweight external plugin that compresses relevant fine-grained context across multiple prompts. Optimized end-to-end with the backbone, Condenser ensures accurate integration of contextual cues. Experiments demonstrate Condenser outperforms state-of-the-arts across benchmark tasks, showing superior context compression, scalability with more prompts, and enhanced computational efficiency compared to ensemble methods, positioning it as a highly competitive solution for VICL. Code is open-sourced at https://github.com/gimpong/CVPR25-Condenser.
Code (1)
Tasks
Computational EfficiencyIn-Context LearningSimilar Papers 제목 키워드 기반
TeamLoRA: Boosting Low-Rank Adaptation with Expert Collaboration and Competition
While Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA have effectively addressed GPU memory constraints during fine-tuning, their performance often falls short, especially in multidimensional task scenarios. To …
GPUMulti-Task Learningparameter-efficient fine-tuningVisual Question AnsweringA reinforcement learning algorithm for building collaboration in multi-agent systems
This paper presents a proof-of concept study for demonstrating the viability of building collaboration among multiple agents through standard Q learning algorithm embedded in particle swarm optimisation. Collaboration is…
Q-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)An adaptive volatility method for probabilistic forecasting and its application to the M6 financial forecasting competition
In this paper, we address the problem of probabilistic forecasting using an adaptive volatility method rooted in classical time-varying volatility models and leveraging online stochastic optimization algorithms. These pr…
Decision MakingStochastic OptimizationA condensing approach to multiple shooting neural ordinary differential equation
Multiple-shooting is a parameter estimation approach for ordinary differential equations. In this approach, the trajectory is broken into small intervals, each of which can be integrated independently. Equality constrain…
parameter estimationCan Competition Outperform Collaboration? The Role of Misbehaving Agents
We investigate a novel approach to resilient distributed optimization with quadratic costs in a multi-agent system prone to unexpected events that make some agents misbehave. In contrast to commonly adopted filtering str…
Distributed Optimization