Learn Faster and Remember More: Balancing Exploration and Exploitation for Continual Test-time Adaptation

2025-08-18 · Pinci Yang, Peisong Wen, Ke Ma, Qianqian Xu arxiv

Continual Test-Time Adaptation (CTTA) aims to adapt a source pre-trained model to continually changing target domains during inference. As a fundamental principle, an ideal CTTA method should rapidly adapt to new domains (exploration) while retaining and exploiting knowledge from previously encountered domains to handle similar domains in the future. Despite significant advances, balancing exploration and exploitation in CTTA is still challenging: 1) Existing methods focus on adjusting predictions based on deep-layer outputs of neural networks. However, domain shifts typically affect shallow features, which are inefficient to be adjusted from deep predictions, leading to dilatory exploration; 2) A single model inevitably forgets knowledge of previous domains during the exploration, making it incapable of exploiting historical knowledge to handle similar future domains. To address these challenges, this paper proposes a mean teacher framework that strikes an appropriate Balance between Exploration and Exploitation (BEE) during the CTTA process. For the former challenge, we introduce a Multi-level Consistency Regularization (MCR) loss that aligns the intermediate features of the student and teacher models, accelerating adaptation to the current domain. For the latter challenge, we employ a Complementary Anchor Replay (CAR) mechanism to reuse historical checkpoints (anchors), recovering complementary knowledge for diverse domains. Experiments show that our method significantly outperforms state-of-the-art methods on several benchmarks, demonstrating its effectiveness for CTTA tasks.

📄 PDF Abstract BibTeX arXiv:2508.12643

Code (0)

등록된 구현이 없습니다.

Tasks

Test-time Adaptation

Reinforcement Learning with Probabilistically Complete Exploration

2020-01-20 · Philippe Morere, Gilad Francis, Tom Blau, Fabio Ramos

Balancing exploration and exploitation remains a key challenge in reinforcement learning (RL). State-of-the-art RL algorithms suffer from high sample complexity, particularly in the sparse reward case, where they can do …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Monte Carlo Tree Descent for Black-Box Optimization

2022-11-01 · Yaoguang Zhai, Sicun Gao

The key to Black-Box Optimization is to efficiently search through input regions with potentially widely-varying numerical properties, to achieve low-regret descent and fast progress toward the optima. Monte Carlo Tree S…

Bayesian Optimization Gaussian Processes

Balancing Novelty and Salience: Adaptive Learning to Rank Entities for Timeline Summarization of High-impact Events

2017-01-14 · Tuan Tran, Claudia Niederée, Nattiya Kanhabua, Ujwal Gadiraju 외

Long-running, high-impact events such as the Boston Marathon bombing often develop through many stages and involve a large number of entities in their unfolding. Timeline summarization of an event by key sentences eases …

Informativeness Learning-To-Rank Timeline Summarization

Learning to Unlearn for Robust Machine Unlearning

2024-07-15 · Mark He Huang, Lin Geng Foo, Jun Liu

Machine unlearning (MU) seeks to remove knowledge of specific data samples from trained models without the necessity for complete retraining, a task made challenging by the dual objectives of effective erasure of data an…

Machine Unlearning Meta-Learning

First return, then explore

2020-04-27 · Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O. Stanley 외

The promise of reinforcement learning is to solve complex sequential decision problems autonomously by specifying a high-level reward function only. However, reinforcement learning algorithms struggle when, as is often t…

Atari Games Montezuma's Revenge reinforcement-learning Reinforcement Learning+1