Task-Specific Preconditioner for Cross-Domain Few-Shot Learning
Cross-Domain Few-Shot Learning~(CDFSL) methods typically parameterize models with task-agnostic and task-specific parameters. To adapt task-specific parameters, recent approaches have utilized fixed optimization strategies, despite their potential sub-optimality across varying domains or target tasks. To address this issue, we propose a novel adaptation mechanism called Task-Specific Preconditioned gradient descent~(TSP). Our method first meta-learns Domain-Specific Preconditioners~(DSPs) that capture the characteristics of each meta-training domain, which are then linearly combined using task-coefficients to form the Task-Specific Preconditioner. The preconditioner is applied to gradient descent, making the optimization adaptive to the target task. We constrain our preconditioners to be positive definite, guiding the preconditioned gradient toward the direction of steepest descent. Empirical evaluations on the Meta-Dataset show that TSP achieves state-of-the-art performance across diverse experimental scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Domain Few-Shotcross-domain few-shot learningFew-Shot Image ClassificationFew-Shot LearningSimilar Papers 제목 키워드 기반
Meta-Learning with a Geometry-Adaptive Preconditioner
Model-agnostic meta-learning (MAML) is one of the most successful meta-learning algorithms. It has a bi-level optimization structure where the outer-loop process learns a shared initialization and the inner-loop process …
Few-Shot Image ClassificationFew-Shot LearningMeta-LearningMeta-Learning with Versatile Loss Geometries for Fast Adaptation Using Mirror Descent
Utilizing task-invariant prior knowledge extracted from related tasks, meta-learning is a principled framework that empowers learning a new task especially when data records are limited. A fundamental challenge in meta-l…
Few-Shot LearningMeta-LearningTwo-level overlapping additive Schwarz preconditioner for training scientific machine learning applications
We introduce a novel two-level overlapping additive Schwarz preconditioner for accelerating the training of scientific machine learning applications. The design of the proposed preconditioner is motivated by the nonlinea…
Operator learningPEARL: Preconditioner Enhancement through Actor-critic Reinforcement Learning
We present PEARL (Preconditioner Enhancement through Actor-critic Reinforcement Learning), a novel approach to learning matrix preconditioners. Existing preconditioners such as Jacobi, Incomplete LU, and Algebraic Multig…
reinforcement-learningReinforcement LearningNSPOD: Accelerating Krylov solvers via DeepONet-learned POD subspaces
The convergence of Krylov-based linear iterative solvers applied to parametric partial differential equations (PDEs) is often highly sensitive to the domain, its discretization, the location/values of the applied Dirichl…