Meta-TTT: A Meta-learning Minimax Framework For Test-Time Training

2024-10-02 · Chen Tao, Li Shen, Soumik Mondal

Test-time domain adaptation is a challenging task that aims to adapt a pre-trained model to limited, unlabeled target data during inference. Current methods that rely on self-supervision and entropy minimization underperform when the self-supervised learning (SSL) task does not align well with the primary objective. Additionally, minimizing entropy can lead to suboptimal solutions when there is limited diversity within minibatches. This paper introduces a meta-learning minimax framework for test-time training on batch normalization (BN) layers, ensuring that the SSL task aligns with the primary task while addressing minibatch overfitting. We adopt a mixed-BN approach that interpolates current test batch statistics with the statistics from source domains and propose a stochastic domain synthesizing method to improve model generalization and robustness to domain shifts. Extensive experiments demonstrate that our method surpasses state-of-the-art techniques across various domain adaptation and generalization benchmarks, significantly enhancing the pre-trained model's robustness on unseen domains.

📄 PDF Abstract BibTeX arXiv:2410.01709

Code (0)

등록된 구현이 없습니다.

Tasks

DiversityDomain Adaptation Meta-Learning Self-Supervised Learning

Methods 이 논문이 사용한 방법론

Batch Normalization 설명 없음

ALIGN In the ALIGN method, visual and language representations are jointly trained from noisy image alt-text data. The image and text encoders are learned via contrastive loss…

Meta Learning with Minimax Regularization

2021-09-29 · Lianzhe Wang, Shiji Zhou, Shanghang Zhang, Wenpeng Zhang 외

Even though meta-learning has attracted research wide attention in recent years, the generalization problem of meta-learning is still not well addressed. Existing works focus on meta-generalization to unseen tasks at the…

Few-Shot Learning Meta-Learning

Improving Generalization of Meta-Learning With Inverted Regularization at Inner-Level

2023-01-01 · CVPR 2023 1 · Lianzhe Wang, Shiji Zhou, Shanghang Zhang, Xu Chu 외

Despite the broad interest in meta-learning, the generalization problem remains one of the significant challenges in this field. Existing works focus on meta-generalization to unseen tasks at the meta-level by regula…

Meta-Learning

Can Meta-Interpretive Learning outperform Deep Reinforcement Learning of Evaluable Game strategies?

2019-02-26 · Céline Hocquette, Stephen H. Muggleton

World-class human players have been outperformed in a number of complex two person games (Go, Chess, Checkers) by Deep Reinforcement Learning systems. However, owing to tractability considerations minimax regret of a lea…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Sampling Attacks on Meta Reinforcement Learning: A Minimax Formulation and Complexity Analysis

2022-07-29 · Tao Li, Haozhe Lei, Quanyan Zhu

Meta reinforcement learning (meta RL), as a combination of meta-learning ideas and reinforcement learning (RL), enables the agent to adapt to different tasks using a few samples. However, this sampling-based adaptation a…

Meta-Learning Meta Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Model-based Adversarial Meta-Reinforcement Learning

2020-06-16 · NeurIPS 2020 12 · Zichuan Lin, Garrett Thomas, Guangwen Yang, Tengyu Ma

Meta-reinforcement learning (meta-RL) aims to learn from multiple training tasks the ability to adapt efficiently to unseen test tasks. Despite the success, existing meta-RL algorithms are known to be sensitive to the ta…

continuous-control Continuous Control Meta Reinforcement Learning model+3