On the inductive biases of deep domain adaptation
Domain alignment is currently the most prevalent solution to unsupervised domain-adaptation tasks and are often being presented as minimizers of some theoretical upper-bounds on risk in the target domain. However, further works revealed severe inadequacies between theory and practice: we consolidate this analysis and confirm that imposing domain invariance on features is neither necessary nor sufficient to obtain low target risk. We instead argue that successful deep domain adaptation rely largely on hidden inductive biases found in the common practice, such as model pre-training or design of encoder architecture. We perform various ablation experiments on popular benchmarks and our own synthetic transfers to illustrate their role in prototypical situations. To conclude our analysis, we propose to meta-learn parametric inductive biases to solve specific transfers and show their superior performance over handcrafted heuristics.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationUnsupervised Domain AdaptationSimilar Papers 제목 키워드 기반
Adapting Pretrained ViTs with Convolution Injector for Visuo-Motor Control
Vision Transformers (ViT), when paired with large-scale pretraining, have shown remarkable performance across various computer vision tasks, primarily due to their weak inductive bias. However, while such weak inductive …
Inductive BiasFast Adaptation with Linearized Neural Networks
The inductive biases of trained neural networks are difficult to understand and, consequently, to adapt to new settings. We study the inductive biases of linearizations of neural networks, which we show to be surprisingl…
Domain AdaptationGaussian Processesimage-classificationImage Classification+1Tripod: Three Complementary Inductive Biases for Disentangled Representation Learning
Inductive biases are crucial in disentangled representation learning for narrowing down an underspecified solution set. In this work, we consider endowing a neural network autoencoder with three select inductive biases f…
Data CompressionDecoderDisentanglementQuantization+1On Inductive Biases in Deep Reinforcement Learning
Many deep reinforcement learning algorithms contain inductive biases that sculpt the agent's objective and its interface to the environment. These inductive biases can take many forms, including domain knowledge and pret…
continuous-controlContinuous ControlDeep Reinforcement Learningreinforcement-learning+2Input-level Inductive Biases for 3D Reconstruction
Much of the recent progress in 3D vision has been driven by the development of specialized architectures that incorporate geometrical inductive biases. In this paper we tackle 3D reconstruction using a domain agnostic ar…
3D ReconstructionDepth Estimation