Emergent Equivariance in Deep Ensembles
We show that deep ensembles become equivariant for all inputs and at all training times by simply using data augmentation. Crucially, equivariance holds off-manifold and for any architecture in the infinite width limit. The equivariance is emergent in the sense that predictions of individual ensemble members are not equivariant but their collective prediction is. Neural tangent kernel theory is used to derive this result and we verify our theoretical insights using detailed numerical experiments.
Code (0)
등록된 구현이 없습니다.
Tasks
AllData AugmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MAEBE: Multi-Agent Emergent Behavior Framework
Traditional AI safety evaluations on isolated LLMs are insufficient as multi-agent AI ensembles become prevalent, introducing novel emergent risks. This paper introduces the Multi-Agent Emergent Behavior Evaluation (MAEB…
Ensembles provably learn equivariance through data augmentation
Recently, it was proved that group equivariance emerges in ensembles of neural networks as the result of full augmentation in the limit of infinitely wide neural networks (neural tangent kernel limit). In this paper, we …
Data AugmentationEquivariance and Augmentation for Bayesian Neural Networks
Symmetries are important for many deep learning tasks, ranging from applications in the sciences to medical imaging. However, there is an ongoing debate about whether to impose symmetry constraints on the neural network …
Data AugmentationPredictable Emergent Abilities of LLMs: Proxy Tasks Are All You Need
While scaling laws optimize training configurations for large language models (LLMs) through experiments on smaller or early-stage models, they fail to predict emergent abilities due to the absence of such capabilities i…
AllRoCoDA: Counterfactual Data Augmentation for Data-Efficient Robot Learning from Demonstrations
Imitation learning in robotics faces significant challenges in generalization due to the complexity of robotic environments and the high cost of data collection. We introduce RoCoDA, a novel method that unifies the conce…
counterfactualData AugmentationImitation Learning